I. eval() to accept custom mapping arguments for globals and locals. This makes it possible to write a smart __getitem__ method for applications like spreadsheets or case-insensitive evaluation. class LowerCaseDict(dict): def __getitem__(self, key): return dict.__getitem__(self, key.lower())
print eval(raw_input('Okay kids, type in an expression:', LowerCaseDict(globals()))
Okay kids, type in an expression: HeX(20) 0x14 class SpreadSheet: _cells = {} def __setitem__(self, key, formula): self._cells[key] = formula def __getitem__(self, key): return eval(self._cells[key], self) ss = SpreadSheet() ss['a1'] = '5' ss['a2'] = 'a1*5' ss['a2'] While this seems like a minor feature request, it presents amazing opportunities to avoid writing lexers, parsers, interpreters by transferring the work to the python interpreter. Writing little languages like the spreadsheet class becomes almost trivial. II. Timer.timeit to take an optional context argument.
Timer('f(10)', env=globals()).timeit() 1.234
The idea is to make it *much* easier to time functions or parts of existing modules. Having to write-out the definition in a long setup string is cumbersome and interferes with syntax highlighting. To instrument modules with timeit, the timer setup string currently has to re-import the whole module -- passing in an execution environment is much more straight-forward. And, as the example shows, passing in globals() makes it easier to experiment with timings using the interpreter. III. enumerate to take an optional argument: enumerate([firstnumber=0,] iterable) The idea is to make enumerate() a complete solution to the loop counter problem: for lineno, line in enumerate(1, file('hist.log')): print lineno, line IV. list.sorted() to become just sorted(). All of the common use cases read better without the "list." prefix:
Also, this Christmas wish cures the weirdness that comes from the classmethod approach:
[3,2,1].sorted('cbs') ['b', 'c', 's'].
While I worry about the total number of builtins, opposing them on principle is a recipe for putting tools where they don't belong. IMO, making list.sorted() a class method was a clumsy way of avoiding the obvious solution and left it smelling a bit hackish. Raymond Hettinger
Raymond wrote:
+0. if you can do it without affecting performance in the dictionary case, that is. -1 if you can't.
II. Timer.timeit to take an optional context argument.
+1
III. enumerate to take an optional argument:
+0 (it's not that hard or expensive to add an offset inside the loop)
IV. list.sorted() to become just sorted(). All of the common use cases read better without the "list." prefix:
+1 one more: V. a join() builtin (join(x,s) becomes s.join(x)). All of the common sep.join() use cases read better without the string literal prefix. (I'll leave the rest for later) </F>
On Sun, 2003-12-14 at 17:05, Fredrik Lundh wrote:
V. a join() builtin (join(x,s) becomes s.join(x)). All of the common sep.join() use cases read better without the string literal prefix.
Really? I predict everlasting confusion on argument order. It's not obvious from the method name whether the joining separator comes first or second. My wish list: V <wink>. Explicit global imports. I want a way to spell "import this from the global standard library even if there's a this.py in the local directory". VI. PEP 318 or something like it. -Barry
I'd much rather invent new syntax to spell *local* imports. I like from .module import something You can even generalize and write from ..module import something to reference your package's parent package. :-) --Guido van Rossum (home page: http://www.python.org/~guido/)
On Mon, 2003-12-15 at 11:03, Guido van Rossum wrote:
Actually, if truth be told, I'd love to just ban local imports. I understand that might be controversial though <wink>. The next best thing would be to force local imports to use special syntax (I was going to say "ugly" :) like what you suggest above, but I'm afraid that that might be more disruptive, requiring future imports and such. I think having a syntax for spelling global imports can be done in a backwards compatible way for 2.4. -Barry
Guido> I'd much rather invent new syntax to spell *local* imports. Barry> Actually, if truth be told, I'd love to just ban local imports. Barry> I understand that might be controversial though <wink>. Okay, so what are global and local imports? I thought I understood after Barry's first post, but not now. I see the <wink>, but figured there's still some germ of desire in Barry's "ban local imports" comment. What kind of imports do we have today? Do they relate to the position of the import statement in the code or to the location of the module being imported along sys.path? Skip
Traditionally, we've called these absolute (== global) and relative (== local) imports. The terms relate to the module's full package name, usually equivalent to its location on the filesystem. sys.path and the placement of the import statement do not enter into it, except that sys.path is the very definition of absolute import. --Guido van Rossum (home page: http://www.python.org/~guido/)
>> Okay, so what are global and local imports? Barry> Sorry, let me rephrase. I'd love to ban relative imports. Barry> Personally, I think all imports should be specified using Barry> fully-qualified absolute package paths. That's fine with me. Maybe continue to use "absolute" and "relative" (and throw in "package" somewhere) instead of "global" and "local" when referring to package import semantics? "global" and "local" have a well-established meaning in day-to-day Python programming which doesn't overlap much with the import mechanism. Skip
Barry Warsaw wrote:
If I'm understanding this right you want to ban imports that are machine independant. That can't be right, but that's what it sounds like to me. To me an "absolute import" or a "fully-qualified absolute package path" sounds like you know the exact directory that the import is occuring from. E.g.: /home/someone/projects/python/workingprogress/currentproject/thisone.py I can see the use for being able to import that, but usually I would prefer to import either from something relative to the current directory, or relative to the python site-lib, or some such. It's also quite nice to not need to bother to exactly specify just which of those I will find it in. (This is what the pythonpath is for.) Now to me an import relative to the current directory is a relative import, and I assume that this is also what is meant by a local import. By allowing files to cluster relative to the code being written one is able to keep related projects and libraries together rather than just having all the code spread out aimlessly in a huge flat directory. I don't see in what what this is inferior. Since you didn't say why you wanted to ban relative imports, I don't understand your reasons. I can't even guess at them. Even if it were for some efficiency condern about hierarchical directories, that would merely say that the compiled modules should be stuffed into a library folder. (Well, more than one...the user needs to have write permission to the libraries that he creates, but it's just as well that he not have write permissions to the system libraries...without taking special steps.)
On Wed, 2003-12-17 at 11:55, Charles Hixson wrote:
Barry Warsaw wrote:
If I'm understanding this right you want to ban imports that are machine independant. That can't be right, but that's what it sounds like to me.
Nope, because "absolute package paths" are always calculated relative to directories in sys.path when mapping a package path to a file system path. -Barry
Works for me. :-)
Well, but since you want all imports to be global, it'd be insane to introduce *new* syntax for global imports, wouldn't it? --Guido van Rossum (home page: http://www.python.org/~guido/)
I think this ought to be a *global* flag rather than a per-module flag. E.g. after setting sys.allow_relative_import = False all imports anywhere would be interpreted as absolute imports. This would mean you couldn't have some code that still uses relative imports, but the problem with the __future__ statement is that it seems so pointless: packages that only use absolute imports don't need it, and packages that use relative imports break if it is used. About the only time where the __future__ statement would make a difference is when a package defines a local module whose name is the same as that of a global module, *and* the package also wants to import the global module. I would personally solve that by renaming the local module though... --Guido van Rossum (home page: http://www.python.org/~guido/)
But there is no backwards compatibility issue. Absolute imports always work, so instead of adding a "relative imports allowed" to your package, you can just change the package to use only absolute imports. Using only absolute imports is totally backwards compatible (unless your toplevel package name is the same as the name of a submodule of the package, which seems pretty sick). --Guido van Rossum (home page: http://www.python.org/~guido/)
On Mon, 15 Dec 2003, Guido van Rossum wrote:
I guess i think there is a case to be made for relative imports, and it becomes apparent as an enterprise grows. Increasingly at Zope corp we are mixing and matching packages to deliver applications to a customer. This is very encouraging, because it means we are actually getting reuse out of the packages. Currently, we can just plop the things in place, and use them. Without relative imports, we would have to be editing the imports in the packages in each place we use it. This would increase the burden of using the packages and even more of feeding back actual fixes - which we then have to distinguish from what i would see as gratuitous changes to edit import names. If you would grant there's use to avoiding those edits, and so there's use to having relative imports, then you might see a reason to solve the problems where the names in a package conflict with those along the load path. Otherwise, if there's only absolute imports, there's no ambiguity to resolve. I'd say there's a legitimate need for relative imports, and we need some explicit gesture to disambiguate between a relative and absolute import, one way or the other. Ken
I know this line of reasoning fairly well. You are taking 3rd party packages (or perhaps internally developed packages) and copy them to a *different place in the package namespace tree*. Right? But why do you have to give those packages a different full name? That's the question that I've never seen answered adequately.
I think that moving packages around in the package namespace is a bad idea. But maybe you can give me an answer to the question above to convince me. --Guido van Rossum (home page: http://www.python.org/~guido/)
But why do you have to give those packages a different full name? That's the question that I've never seen answered adequately.
I have done this many times, so let me try to describe at least one legitimate usage case. A couple of weeks ago I wrote a software which needs a third party package to work. OTOH, since it's a small package, I don't want to force the user to go after the package, even because I'd have to ensure that my code always work with the available version of that package. Thanks to the relative module importing mechanism, solving that is no harder than copying the third party package into my own package namespace. This idea could probably be expressed in some other way, hacking sys.path or whatever, but I belive this is a fairly common pattern, and I vote for introducing a scheme to differ between local/global importing which would not break the current flexibility. -- Gustavo Niemeyer http://niemeyer.net
A lot people have presented a good case for relative imports. Nobody has argued to keep the status quo (where imports are ambiguous about whether they are meant to be absolute or relative). So I suggest that in 2.4, we introduce the leading dot notation for relative import, while still allowing relative import without a leading dot. In 2.5 we can start warning about relative import without a leading dot (although that will undoubtedly get complaints from folks who have code that needs to work with 2.3 still). In 3.0 we can retire ambiguous import. The use case for multiple dots should be obvious: inside a highly structured package, modules inside one subpackage must have a way to do relative import from another subpackage of the same parent package. There is the remaining issue of what exactly the syntax would be. I propose to extend the from clause to allow one or more dots before the dotted name, and to make the dotted name optional if at least one leading dot is found. I propose not to change from-less import. Examples: from .foo import bar from .foo.bar import xxx from . import foobar as barfoo from ..foo.bar import * from ... import foobar, barfoo Grammar change (see Grammar/Grammar): dotted_name: NAME ('.' NAME)* | '.'+ [NAME ('.' NAME)*] --Guido van Rossum (home page: http://www.python.org/~guido/)
On Tue, 16 Dec 2003, Guido van Rossum wrote:
On a related and minor note, can we please expand the grammar to allow from...import statements to continue to the next line after a trailing comma? For those of us with large, complex, and componentized projects, it would be very nice. e.g.: from OPAL.Modules.Financial.Reports.COGS import generate, webOptions, normalize, row_ops, summary instead of: from OPAL.Modules.Financial.Reports.COGS import generate, webOptions, \ normalize, row_ops, \ summary This has been a minor nit that bothers just under the threshold of fixing it myself. ;) Thanks, -Kevin -- -- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (440) 871-6725 x 19 E-mail: jacobs@theopalgroup.com Fax: (440) 871-6722 WWW: http://www.theopalgroup.com/
I guess this could be implemented by allowing [NEWLINE] after the comma in the grammar. But I'm -0 on this change; I fear that someone accidentally adding a trailing comma (or leaving it in after some rearrangements) will be very puzzled at the syntax error that comes somewhere in the middle of the next statement: from foo import bar, bufsize = 12 ^syntax error --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote:
you could require parentheses: from OPAL.Modules.Financial.Reports.COGS import ( generate, webOptions, normalize, row_ops, summary ) (after all, from-import is a kind of assignment, and "a, b = c" is the same thing as "(a, b) = c"...) </F>
Guido:
How about a "suite" of things to import? from OPAL.Modules.Financial.Reports.COGS import: generate, webOptions, normalize row_ops, summary Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+
Nah. But I liked the suggestion seen here earlier of allowing parenthesis: from OPAL.Modules.Financial.Reports.COGS import (generate, webOptions, normalize, row_ops, summary) --Guido van Rossum (home page: http://www.python.org/~guido/)
[Guido]
So do I. Something I'm still acutely aware of despite not using Emacs for years is how much trouble it would be to teach python-mode.el about a unique new way to continue a statement. Parens would "just work". Ditto for IDLE, and tokenize.py, etc etc etc etc. I bet tools parsing import statements for *content* (like pychecker, and perhaps pyclbr.py -- the others mentioned are just looking at form) would need modification regardless, though.
Second impression: "suites" contain statements, not a list.
That's why I put "suite" in quotes. It wouldn't really be a suite, just an indented block of stuff. But if you want, it could be a real suite that only happens to allow import statements... from OPAL.Modules.Financial.Reports.COGS: import generate, webOptions, normalize import row_ops, summary Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+
On Wed, Dec 17, 2003 at 02:53:35PM +1300, Greg Ewing wrote:
But this is also a very odd suite, because you can't use other statements... But it does make me think of: with OPAL.Modules.Financial.Reports.COGS: import .generate, .webOptions, .normalize import .row_ops, .summary Which is like a bastard child of the relative import proposal and the with statement proposal... <wink> -Andrew.
Hi, "Guido van Rossum" <guido@python.org> wrote
I'm +1 on it, I'd only wish an additional from __future__ import strict_import (or whatever seems a better name) to allow a module to indicate that every not explicitly relative import should be absolute. This would allow module authors to be explicit from the start and it is also a safety net against accidentially adding a relative module with the name of some absolute one. best regards Werner
On 16-dec-03, at 19:26, Guido van Rossum wrote:
Is "from . import *" allowed? Whenever I start to think about it I tend to go into oscillation (yes because within package foo its the same as "from foo import *" on the outside; no because "import *" isn't allowed today either). -- Jack Jansen, <Jack.Jansen@cwi.nl>, http://www.cwi.nl/~jack If I can't dance I don't want to be part of your revolution -- Emma Goldman
Guido:
If I understand correctly, the rule you're working towards is that if the path starts with a dot, it's relative, otherwise it's absolute. Is that correct? I'm not sure whether eliminating the "ambiguous" category of module references is the best idea. There are situations in which it's actually what you want. Suppose there are two top-level modules A and B, and A imports B using a bare name. Then someone decides to put A into a package. The reference to B still works. Then they decide to put B into the same package as well. The reference to B still works. But if only unambiguously absolute or relative references are possible, there is no way for A to refer to B that works in all these combinations. So I think we really want *three* kinds of module reference: A: Explicitly absolute B: Explicitly relative to the current module C: Searched for upwards in the package hierarchy from the current module (Note that C is a generalisation of the current "ambiguous" references which only look in two places.) Suggested syntaxes for these: A: a.b.c. Path ends with a dot B: .a.b.c Path begins with a dot C: a Path neither begins nor ends with a dot a.b.c Note that ALL current module references would be type C according to this. Yes, this is a semantic change, but one that I believe would be for the best in the long run, and don't think would cause a lot of upheaval. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+
In article <200312170022.hBH0Mne16539@oma.cosc.canterbury.ac.nz>, Greg Ewing <greg@cosc.canterbury.ac.nz> wrote:
Is funny punctuation really the right way to spell an important distinction like this? -- David Eppstein http://www.ics.uci.edu/~eppstein/ Univ. of California, Irvine, School of Information & Computer Science
David Eppstein <eppstein@ics.uci.edu>:
Is funny punctuation really the right way to spell an important distinction like this?
Maybe not, but I haven't seen anything obviously better proposed yet. I've no idea what else to to for relative paths. A leading dot doesn't seem all that bad to me for that. I concede that a trailing dot would look rather odd, though. The absolute-path case could be spelled using a special name for the root of the package hierarchy, but then we need to think of a name that's sufficiently unlikely as a real module name. I don't think I like the idea of re-using the word "global", although I'm not sure. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+
Greg Ewing wrote:
Alternate spellings (plus category D, which is "Python 2.3 semantics"): A: from __absolute__.dotted.name import foo B: from __relative__.dotted.name import bar C: from __heirarchy__.dotted.name import foobar D: from dotted.name import barfoo I believe this spelling would only require import to handle the special cases as the first 'package' name (although having the compiler recognise them may be a later optimisation), and should also work with straight "import __absolute__.dotted.name.foo" statements. Then Guido's migration plan would apply to the progression of the D semantics from the Python 2.3 model to being a shorthand for the absolute semantics. And once _that_ has happened, then we could get rid of the '__absolute__' version in the name of TOOWTDI. Regards, Nick. -- Nick Coghlan | Brisbane, Australia Email: ncoghlan@email.com | Mobile: +61 409 573 268
Just to give you a practical example, we are using python for internal tools in the electronic Company I'm working for. I needed to have a non ambiguous file lookup when using import ( We set external dependencies for the flow when opening a file, so you can't use a 'if this file exist' check, when opening a file, it must always exists) I made a special import function, that was looking for a prefix to the package name: import loc.dotted.name import glb.dotted.name import __.dotted.name import std.dotted.name import xxx.dotted.name -> raise an error "loc", look in the current directory of the module (looking to his __file__ attribute) "__ "look to the parent directory "glb" look at the root of the user working repository ( a builtins set before anything else is done , users don't call python they the program myPython ) "std" look outside the database ( in the python installation dir mainly) and set back the old import behaviour to ensure that import inside standard modules don't fail ) I know the loc,glb __ and std names are no good for a general case as modules can have these names, but this is just to give you an actual use of these relatives imports. Boris Nick Coghlan wrote:
Nick Coghlan <ncoghlan@iinet.net.au>:
That's where I disagree - I think that type C semantics should be the default/least ugly/easiest to get, not type A. But I wouldn't like any of the others to be as ugly as those, either. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+
On Tue, 2003-12-16 at 13:26, Guido van Rossum wrote:
I'll just note that where the current status quo trips /me/ up most is when I accidentally have a local module with the same name as a global module, and then I write an import statement expecting to get the standard library module, but end up getting the local module. That's why when I tend to think about this, I start wanting a way to spell "definitely give me the global one, no matter what". IOW, I feel like I want a way to bypass relative module lookups. I'm still catching up on this thread, but I wanted to throw this out there... -Barry
On Wed, Dec 17, 2003 at 09:33:43PM -0500, Barry Warsaw wrote:
Alternatively, maybe what you want is a way to say "definitely give me the standard library one, no matter what", e.g. from stdlib import codecs This would allow other packages to have their own codecs module/package without worrying about confusing import errors. To make this backwards compatible, though, we'd probably need the equivalent of C++'s "using namespace" statement, with no declaration being equivalent to: using namespace stdlib Or perhaps more python-like: __import_base__ = 'stdlib' Python 3.0 might default to __import_base__ = '', to force people to be explicit. Actually, the proposed with statement could be used here, e.g. import stdlib with stdlib: from . import codecs, user, gc looks reasonably clear to me, and for long package paths, it could be very convenient: import zope.app.interfaces with zope.app.interfaces: from .location import ILocation from .foo import IFoo, IBar # etc -Andrew.
On Wed, 2003-12-17 at 22:04, Andrew Bennetts wrote:
Interesting. I see the subtle distinction. One searches only the standard library, the other searches every directory on sys.path. I'm not sure I'd have much need to restrict the search to just the standard library. Plus, would that include site-packages? -Barry
On Wed, Dec 17, 2003 at 10:08:21PM -0500, Barry Warsaw wrote:
I should have been clearer. What I'm basically proposing is to make site-package called "stdlib", leaving a default python install with a very uncluttered top-level namespace, thus reducing unexpected name conflicts. Otherwise, this idea doesn't require any modification to import semantics, except for some magic to make stdlib the default root for imports for backwards compatibility. Thinking about it a bit harder, though, I'm not sure it's as easy as I initially thought, because this would then make importing 3rd party packages not work, i.e. currently you can do: import urllib import pygtk without problems, because they are both top-level. If urllib moved into a "stdlib" package, it's hard to imagine a clean way where old code that does those two lines would still work. I still like the idea of having the standard library in its own package, but I guess that will have to wait until Python 3.0. -Andrew.
Andrew Bennetts <andrew-pythondev@puzzling.org> writes:
I still like the idea of having the standard library in its own package, but I guess that will have to wait until Python 3.0.
Guido tends to shoot people who suggest this, watch out :-) Cheers, mwh -- Lisp nearing the age of 50 is the most modern language out there. GC, dynamic, reflective, the best OO model extant including GFs, procedural macros, and the only thing old-fashioned about it is that it is compiled and fast. -- Kenny Tilton, comp.lang.python
On Thu, Dec 18, 2003 at 02:16:45PM +1100, Andrew Bennetts wrote:
I'm at least a little confused with what people want, as far as I can tell import semantics have three things * what module to import * what name to give that import * where to look for it The 'from' keyword seems to be getting overloaded to mean both what module and where to find it, which the 'stdlib' tries to resolve. Most of the discussion has been about how to improve 'where to look' I like syntax that reads most important left-to-right, so what about from MODULE import NAMES as RENAME searching HOW or import NAMES as RENAME from MODULE searching HOW searching could be 'asbolute' 'relative' or any of the other suggested words. If you wanted to get fancy it could be a list, if it isn't a list people who truly care could cascade try/except on ImportError. -jackdied ps, I like the second version better but it is less like the current version, which is a drawback.
[Jack Diederich]
[Barry Warsaw]
I want to think about that more, but at first glance, it has some appeal. Neat!
Ditto. I think Jack's on to something there. I particularly like the second form, as it's an ongoing irritation to me that some imports begin with "from" and others with "import". In the current "from" form, the most important thing to me is almost never the module I'm importing the thing from, but (of course) the thing I'm importing. Going from import math to import sin from math (or vice versa) is also an easier edit than from import math to from math import sin (because the former doesn't require transposition). Adding a distinct clause to influence search details seems downright Pythonic. Maybe *too* Pythonic <wink>.
Jack Diederich <jack@performancedrivers.com> writes:
I like this. It feels much more Pythonic than the various suggestions around magic dots in the name. It's explicit, easy to understand, and easy to extend (extra possibilities for HOW could be added with little difficulty). Paul. -- This signature intentionally left blank
Jack Diederich wrote:
This would work really well for any uses cases I can imagine having. It also has the following benefits: 1. No magic punctuation 2. People who don't need it, just never use a 'searching' clause 3. People encountering it for the first time have a clear flag letting them know something fancy is going on 4. These people have a word they can look up in the Python docs I hadn't even thought of point 4 prior to considering Jack's proposal. If I encounter "import .fred.mary" in some python I didn't write, how the heck do I find out what it means? [...]
ps, I like the second version better but it is less like the current version, which is a drawback.
To head a little further down this sidetrack, the from version of 'import' has puzzled me for a while: A. from mypkg import foo B. import mypkg.foo as foo What, if any, are the semantic differences between A & B? Is it only that A2 works, and B2 doesn't? A2. from mymodule import * B2. import mymodule.* Cheers, Nick. -- Nick Coghlan | Brisbane, Australia Email: ncoghlan@email.com | Mobile: +61 409 573 268
This got several encouraging responses, so I have to consider it. Unfortunately it looks like 'searching' would have to become a reserved word; if we use the same trick we used to avoid making 'as' a reserved word, that would be too confusing to decode in the code generator. The HOW could be just an identifier then. I'm not sure how well it'll read in actual examples. Let's try. Adapting an example from Opal that was used here before... from COGS import generate searching relative To me, that looks a lot like someone left the commas out of from COGS import generate, searching, relative In general I'm not a big fan of "wordy" clauses like this -- they work better in languages like SQL or COBOL because those are case-sensitive so you can use a difference in capitalization there; there the former would be FROM COGS IMPORT generate SEARCHING RELATIVE (using imaginary syntax) while the latter would be FROM COGS IMPORT generate, searching, relative while in Python the long sequence of lowercase words becomes a blur. So if the two alternatives are simply 'searching absolute' and 'searching relative', with the default being 'searching absolute' in Python 3.0 (but the current ambiguous search in Python 2.x) I'd still prefer making the distinction with "from ...MODULE" vs. "from MODULE" rather than with a searching clause. And I still prefer the single-dot syntax over three dots, because it can be used to spell importing from the parent or grandparent package explicitly. A separate suggestion is to switch from "from X import Y" to "import Y from X". This seems a gratuitous change even if it reads or edits marginally better. Now's not the time to tinker with marginal stuff -- that should've been done 10 years ago. --Guido van Rossum (home page: http://www.python.org/~guido/)
On Sun, Dec 21, 2003 at 08:48:01PM -0800, Guido van Rossum wrote:
Too bad the decorator PEP (PEP 318) seems to favor 'as' in the syntax, import foo [relative] is more explicit than the dot syntax and has a nice visual seperator while avoiding the wordiness problem of 'searching'
moving the 'import' to always be first would alleviate this a bit because 'searching' would always follow a non-list. import generate from COGS searching relative or if we decorate import generate from COGS [relative] from COGS import generate [relative]
as above, moving import to be first helps but doesn't eliminate the wordiness problem[*]. 'as' and 'from' are short enough to be easilly seen as keywords, 'find' or 'look' reads less like english but would be easier to pick out visually than 'searching'
So if the two alternatives are simply 'searching absolute' and 'searching relative'
Did someone mention 'searching scan' in dusty corner somewhere? One drawback to the dot syntax is if there can be more than two ways to do the search. Adding a third way is easier w/ the searching clause instead of adding a 'bang' syntax to the 'dot' syntax . The decorator would do one better by being a placeholder for anything that modifies the import.
Damn, I really liked that part. I don't get tripped up on from X import Y anymore, but I did when I started w/ python. benign-user-for-life-ly, -jackdied [*] I use an emacs hack to display 'lambda' as 'L' to avoid the same keyword-reads-like-a-variable effect.
I just realized I've been forgetting to CC the list. It is a bit long, but I'll cut out the non-relevant portions that everyone has seen before. In chronological order below. I ask the rest of you, does Some sound so crazy? - Josiah *** message from Josiah Carlson on Sun, 21 Dec 2003 19:05:45 -0800 ***
I've read PEP 754, and yes, the author talks about having a proper Infinity, -Infinity and Not a Number (PosInf, NegInf and NaN respectively), but considering that it has been in the 'draft' state since March 28, has not been updated, and it doesn't seem like Python is any closer to having proper definitions now than it was ~9 months ago, I'm not going to put my eggs in the IEEE 754 Infinity basket. In terms of having something useful on a platform, there is only so useful that infinity can be. The below is from the windows verstion of Python 2.3.2
The real trick is that it is not so much about being able to have max(Some, a) -> Some, it is about being able to have min(Some, a) -> a. Then you don't need to worry about sys.maxint not being big enough (because there are larger numbers), nor do you need to worry about float casts of large integers producing OverflowErrors. As for an application, many path searches in graphs start out with initializing distances to infinity, Some would be suitable for this. Other algorithms require a negative infinity for this, None has been working fine. As an initialization value, I believe that Some would be quite a useful object. Certainly Some seems like a strange way to spell infinity, but it is a logical infinity, one that you can compare things against, anything, not just Python floats or things that can be cast into Python floats. In terms if PEP 754, if it were actually implemented, certainly there would be some question as to whether Some would be equal to PosInf. If so, then would -Some be equal to NegInf? And if so, would None be equal to NegInf? Maybe such things would require a float cast. I talk about all this in my PEP submission, which isn't up yet. I suppose I probably should have given it a few days before discussing it, but yeah. - Josiah *** end message from Josiah *** *** message from Guido van Rossum on Sun, 21 Dec 2003 20:45:34 -0800 ***
I expect your PEP will fare no better, and given its weirdness, probably much worse. Do you know of any language that has a similar concept? --Guido van Rossum (home page: http://www.python.org/~guido/) *** end message from Guido *** *** message from Josiah Carlson on Sun, 21 Dec 2003 21:47:34 -0800 ***
Is None so weird? All I am proposing is a symmetric value to None. Just like there is a -1 and 1, -Infinity and Infinity, None logically should have Some. Maybe not so much -Some == None, but as I propose in the PEP, possibly Some == not None. - Josiah *** end message from Josiah ***
"Josiah Carlson" <jcarlson@uci.edu> wrote in message news:20031222000819.10EC.JCARLSON@uci.edu...
Is None so weird? All I am proposing is a symmetric value to None.
To me that would be All of AllType. Some is in between. Can you think of any uses for All other than being max element? TJR
Not really. However, I was thinking of Some in the context of graph and dynamic programming algorithms. When programming some algorithms in Python, you need to either pick some large constant (that you hope will be big enough), or have a test to see if a value is None. That is ugly and very not Pythonic. I do like All and AllType though, which gives a better name, as suggested by Guido. - Josiah
Josiah Carlson <jcarlson@uci.edu> writes:
I ask the rest of you, does Some sound so crazy?
To be honest, yes. To me, None is not the low end of a scale, rather it is a unique distinguished value, more related to the concept of "uninitialised" or "not used" than to "smallest". The fact that it compares less than every other value is to me incidental, and arbitrary. I see None as inhabiting the same sort of niche as the SQL NULL value.
From the above, you'll gather that from my viewpoint, there really is no "other extreme" from None, any more than there is from zero.
Paul. -- This signature intentionally left blank
[Paul Moore]
Arbitrary?! I clearly recall when the decision was made that None would compare less than anything else. I was in the same room with Guido, and this is a verbatim transcript of the debate: "As long as we're changing everything here anyway, maybe comparing with None should yield a consistent result." "Like what, maybe smaller than anything else?" "Sure. Or larger." "Ya, one of those. It's not worth it if it just ends up in the middle somewhere." "What does Python do now?" ... [head-scratching] ... "So what did it do before?" ... [more head-scratching] ... "Well, that's no help, then." "The Reference Manual doesn't promise anything about how None will compare, just the 'consistent total ordering' bit". "So that doesn't constrain us." "No, it's neutral." "So maybe it should just be smaller than everything else." "Sure. Or larger." "If it were smaller, then when you sorted a mixed-type list, all the Nones would float to the start." "Good point. If it were larger, they'd float to the end." "I wouldn't like that much." "Me neither." "OK, it's settled." Two lines of code later, the Only Objectively Correct result was achieved. history-is-more-impressive-in-textbooks-ly y'rs - tim
On Sunday 21 December 2003 10:52 pm, Jack Diederich wrote:
I like Jack's original idea and I've been chewing on it for a few days. His was: import NAMES as RENAME from MODULE searching HOW My humble suggestion: import NAMES import NAMES in MODULE [in HOW] import NAME, OTHER in FOO in RELATIVE import NAME as RENAME in MODULE in ABSOLUTE Playing with it: import sys import path, stat in os import AClass, AValue in AModule in __search__ import AType in Package.Library in __absolute__ It emphasizes the name that's imported and it reduces+reuses a keyword. Reusing "in" is a stretch, of course, and it reading it twice in the same statement might be confusing for some. I don't really like the dot/triple dot notation... the leading punctuation smells perl-ish. :) Only-a-random-two-cents-ly yours, -- Troy Melhase, troy@gci.net -- Merry Christmas!
On Mon, 2003-12-22 at 04:03, Troy Melhase wrote:
I don't really like the dot/triple dot notation... the leading punctuation smells perl-ish. :)
I don't either, although my biggest problem with it is that it isn't clear just by looking what the punctuation actually means. I just fear this will be a permanent head scratcher. Here's my entry in the sure-to-be-shot-down-by-the-BDFL-alternative -syntax game: import foo by absolute import foo.bar by relative as baz import foo by searching as bar import foo by absolute from bar import foo as bar by absolute from baz IOW, 'by' is a reserved word in the import statement context, and it specifies how to search for modules. The question then is whether 'absolute', 'relative', and 'searching' are also reserved words or built-ins and how those names are equated with the module location algorithms. -Barry
This argument by itself has very little value. The question isn't whether it's self-describing (very few features beyond decimal integers are); the question is whether it's easy to remember. There are two subclasses of that: easy to remember what it means when you next see it, and easy to remember how to spell it when you next need it. IMO the dot syntax does fine here.
Here's my entry in the sure-to-be-shot-down-by-the-BDFL-alternative -syntax game:
There have been so many alternatives proposed this morning that I have to spend some time comparing them all... --Guido van Rossum (home page: http://www.python.org/~guido/)
On Mon, 2003-12-22 at 10:36, Guido van Rossum wrote:
Sorry, it was a pre-coffee argument. To me, the dot-syntax doesn't really tell me whether it forces a relative search or an absolute search. The dot tells me that something different is happening, so maybe when absolute is the default, it tells me there's a switch to relative searching going on. I'm not sure though, so I do think it may potentially fail the "easy to remember what it means when you next see it" test. OTOH, it would be nice to see a Greg Wilson-style newbie poll.
Hopefully, this stuff will make it into the PEP so you don't have to troll through suggestions buried in endless email threads. -Barry
On Mon, Dec 22, 2003, Barry Warsaw wrote:
That's the plan; I expect that people will review the PEP to make sure their pet proposal is listed. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization.
Thanks -- I'll stop tracking this thread closely then until I see the PEP. --Guido van Rossum (home page: http://www.python.org/~guido/)
I've been monitoring all of this discussion about relative imports, and have my own humble idea about how to spell relative imports. The first part stems from Guido's idea about using a '.' at the beginning of a relative import. It seemed a bit weird at first, but I eventually became very comfortable with the idea after thinking about it in respect to absolute imports. I'll explain what I mean. If you currently want to import a module 'a.b.c.d', you write: import a.b.c.d Now, if you're in the module 'a.b', you could still write: import a.b.c.d ... but the 'a.b' part is redundant because you're already in the module 'a.b'. If you take away 'a.b', then you end up with: import .c.d This flowed well in my head, and looks natural. However, the suggestion about '.' or '..' referring to the parent looks _ugly and unnatural_. Let's say '.' was used to spell "parent module", and let's assume that the module 'a.b.c.d' wants to import 'a.b.e'. The absolute import would be spelled: import a.b.e ... and the relative import would be spelled: import .....e # .[parent].[parent].e Yuck! (no offense :) What's needed, I believe, is an identifier/keyword/symbol that spells 'parent' that isn't '.'. Assuming that symbol is represented by '[parent]', the above relative import would look like: import .[parent].[parent].e ... which is a hell lot more clear IMHO. My initial idea is to spell parent '__parent__', and to have '__parent__' be a module-level variable in each module that refers to the parent module (where a top-level module might have '__parent__' refer to itself, or to None). Continuing with the above example, the relative import would be spelled: import .__parent__.__parent__.e ... but I'm not completely convinced that there's not a better identifier/keyword/symbol that could be substituted for '[parent]'. If I'm completely off track, then tell me and I'll continue to lurk until I get a better feel for the happenings of python-dev. -- Devin devin@whitebread.org http://www.whitebread.org/
On Mon, 22 Dec 2003, Devin wrote:
I also look at it this way, and find it both intuitive and mnemonic. My model is slighly different than yours, though, in a way that simplifies the expression for going up the relative chain. I'll explain...
Truly, yuck. But in my model, the leading '.' dot, itself, stands for the containing package, and '..' stands for the containing package's package, and so forth: import ..e # [parent-of-parent].e I don't think it's yucky at all, this way. (I also don't think it's un-pythonic - python uses punctuation all over to denote type literals ('{'/'}'), cues which delineate statement suites (':'), and so forth.) In fact, i think it's a lot more mnemonic and encompassing than the alternatives that use words (though i haven't looked at all the alternatives very carefully yet). Even using the leading '.' syntax, it would be good to have a specific identifier for the containing package, realized in each module and respected as a special identifier by the package machinery. I like '__pkg__': import __pkg__.sibling # == 'import .sibling' and import __pkg__.__pkg__.uncle # == 'import ..uncle' import __pkg__.__pkg__.uncle.cousin # == 'import ..uncle.cousin' I think the leading '.' dot forms are a lot nicer, but the presence of a module named '__pkg__' would be useful for programmatically mucking navigating the package structure. Ken Manheimer klm@zope.com
On Tue, 23 Dec 2003, Ken Manheimer wrote:
I can't quite get used to the idea of '.' having two different meanings depending on how many '.'s are found next to each other in an import statement. I believe that the notation above _is_ concise, but _is not_ very intuitive ...
... while the notation here _is_ intuitive (to a python programmer), but _is not_ concise. I still favor this syntax to the former syntax. I like '__pkg__' better than my initial suggestion ('__parent__'). It's more aesthetically pleasing. :) -- Devin devin@whitebread.org http://www.whitebread.org/
Guido van Rossum <guido@python.org> writes:
The biggest "clarity" issue I have with the dot notation is that there's no obvious keyword to look up when you want to find out what's going on. The best you could do is look up the "import" section of the manual. Heaven help you if you want to use Google. (Has anyone tried to get any meaningful information from Google on Microsoft's two technologies, COM and .NET? You'd almost think they'd done it deliberately... Seriously, the "what do I search for?" factor bugs me every time I hit it. Paul. -- This signature intentionally left blank
There's no deep reason why the language reference couldn't contain non-alphanumeric characters, although this won't help for Google. But in this case, "from" would be the thing to look up, so I'm not sure I see it as such a serious problem. --Guido van Rossum (home page: http://www.python.org/~guido/)
"Troy Melhase" <troy@gci.net> wrote in message news:200312220003.08810.troy@gci.net...
I had same idea of reusing 'in', but with relative, absolute changed to perhaps RelMode, AbsMode so as to grammatically flow better. 'in NAME' could then indicate place in package to begin search. 'in Package' and 'in Path' are clearer (more explicit) alternatives to relative and absolute. Terry J. Reedy
Summarizing what I've seen so far: Status quo: [from NAME] import NAME [, NAME] [from NAME] import NAME as NAME There also seems to be consensus on adding: [from NAME] import (NAME [, NAME]) To this, we want to as a way to spell out explicitly how the module is found; for example: -a- by searching sys.path -b- in the current package -c- in the current package or its parent package, recursively -d- among the python standard library modules (and nowhere else) -e- in the current working directory -f- in a specific directory -g- combinations of the above Jack Diederich proposed:
Guido van Rossum oppined:
(---)
Not only a gratuitous change; it was always my impression that "from" was at the beginning of the statement for exactly the reason Guido is hesitant about adding "searching" at the end of it. With this in mind, my suggestion is to add the searching syntax should before the "import" keyword rather than at the end of the statement. Reusing an existing keyword, we get: [from NAMES] [in WHERE] import ... WHERE would then be an object, or a list of objects (-g-) specifying the import mechanism: -d- __stdlib__ or maybe stdlib -b- __package__ -c- __search__ -e- '.' -f- '/home/bellman/python' -a- sys.path Without any "from" clause, "in WHERE import ..." reads a little strangely, but I think it should be possible to get used to it, just as we've gotten used to the current "from NAME import ..." Would this work ? /Paul
Guido van Rossum wrote:
Not necessarily. import could be a special function that doesn't need parentheses around it's arguments or comma separators. That could be syntax sugar for: import (names, namedAs, fromModule, searchmethod) or import (names =[theNames], namedAd=[localNames], fromModule=fromModule, searchMethod=upFromCurrent) OTOH, to me it looks more like Smalltalk than like Python. An explicit import function without special sugar looks more Pythonic. (But that would break backward compatibility.) Perhaps the current method could be deprecated, and an import function be the replacement?
Charles Hixson <charleshixsn@earthlink.net> writes:
I really don't think you could make that work (unless you want to write math = import("math") everywhere, which you can almost do today...) Cheers, mwh -- Ya, ya, ya, except ... if I were built out of KSR chips, I'd be running at 25 or 50 MHz, and would be wrong about ALMOST EVERYTHING almost ALL THE TIME just due to being a computer! -- Tim Peters, 30 Apr 97
[Barry]
Indeed, this is common on c.l.py and the Tutor lists, with a twist: someone writes a module in their *current* directory, with a name like random.py, and then "Python breaks mysteriously all over the place", because every import of random ends up with something utterly unexpected. For some reason, "random.py" seems most often to be the actual name used in these cases. Possibly related: I remember pissing away a day a year or so ago trying to track down a segfault in a Zope test that happened only when I ran the test on my McLean box. In the end, it turned out that the test in question was (after untangling the usual layers of Zopish indirection <wink/grrrr>) deliberately trying to import a module that didn't exist, with an oddball name. It just so happened that somebody had whittled down a segfaulting Python program and uploaded it to SourceForge, with the very same oddball name. I had downloaded that program into my Python build directory, and Zope kept provoking its segfault as a side effect of (unintentionally!) importing it. In large part, I've come to loathe imports -- I never have the foggiest idea how they're going to get satisfied in code I didn't write, and especially in code that plays every trick in (and out) of the book to hide what it's really doing with sys.path mutations, import hooks, poking stuff by magic into sys.modules via well-hidden side effects in scattered __init__.py files, ... it makes Java's rigid, wordy, explicit import scheme look very attractive some days. That you can't trick that into doing arbitrarily unexpected things by magic at runtime isn't entirely a bad thing <wink>.
This can be interpreted in two ways though. You could be unfortunate enough to create a module in your package that has the same name as a standard module you also need to import from inside the same package -- that's currently impossible. But the most common variant I've seen is what Tim Peters describes: having a module in the current directory that accidentally hides a standard module. The latter case does not involve packages and will not be fixed by any import syntax -- it can only be fixed by changing the default sys.path *not* to include the current directory, just like over the last decade or so Unix shells have stopped putting '.' in $PATH. But that has other downsides: it would be a lot harder to begin writing a program broken up into modules. Hmm, maybe the current directory should be last? That way you can import your own modules except if they clash with the standard library. Hm, but there are so many standard library modules that it might still cause frequent complaints from mystified beginners. I'm beginning to agree (uselessly) with Tim: import itself is a problem. :-) --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido:
Maybe in Python 3.0 the current directory should appear as a package with some standard name. Then applications could use relative imports to import their own modules, and absolute ones to import standard modules, without fear of conflict. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+
On Wed, Dec 17, 2003 at 08:55:03PM -0800, Guido van Rossum wrote: [...]
Or it can be fixed by not having the entire standard library at the toplevel, but instead inside a single "stdlib" package. That way 3rd party packages could never conflict with the standard library (unless they were crazy enough to call one of their own modules "stdlib"...). This way, no matter how large the standard library grows, the potential for module name conflicts doesn't grow. It'd probably simplify things slightly, too -- the site-packages directory could go away entirely, and all packages, standard or not, would be on an equal footing. The standard library would be no more special than, say, Zope. My /usr/lib/python would just contain stdlib/ twisted/ zope/ Numeric/ etc. The backwards compatibility problems probably means this isn't feasible until Python 3.0, though :( -Andrew.
Andrew> Or it can be fixed by not having the entire standard library at Andrew> the toplevel, but instead inside a single "stdlib" package. ... Andrew> The backwards compatibility problems probably means this isn't Andrew> feasible until Python 3.0, though :( Not necessarily. It has been proposed (more than once I think) that such a stdlib package could be created in parallel using symlinks. At its simplest, I think all you'd need would be a stdlib directory, a nearly empty __init__.py file and symlinks to all the current standard modules and packages. (The __init__.py file would take care of importing builtin modules.) I would choose a very short name for such a package ("std" seems best) even though it might create clashes because it will be used a lot. Skip
Skip Montanaro <skip@pobox.com> writes:
Are you sure that you won't get multiple copies of the same module floating around? E.g. will you have from httplib import HTTPConnection as HC1 from std.httplib import HTTPConnection as HC2 HC1 is HC2 I think in the scheme sketched above this will be false, which kills the idea stone dead. I might be wrong, though. Cheers, mwh -- Counting lines is probably a good idea if you want to print it out and are short on paper, but I fail to see the purpose otherwise. -- Erik Naggum, comp.lang.lisp
>> It has been proposed (more than once I think) that such a stdlib >> package could be created in parallel using symlinks.... Michael> Are you sure that you won't get multiple copies of the same Michael> module floating around? E.g. will you have Michael> from httplib import HTTPConnection as HC1 Michael> from std.httplib import HTTPConnection as HC2 Michael> HC1 is HC2 Michael> I think in the scheme sketched above this will be false, which Michael> kills the idea stone dead. Maybe the symlink idea won't work. It seems that it would work to import the standard modules from std/__init__.py though. I performed a simple experiment. I created the std directory, added a symlink in it for the random module and added an __init__.py file with this content: import sys import urlparse Here's the result: >>> import random >>> from std import random as random2 >>> random is random2 False >>> import urlparse >>> from std import urlparse as urlparse2 >>> urlparse is urlparse2 True >>> import sys >>> from std import sys as sys2 >>> sys is sys2 True If we added a significant number of modules to std/__init__.py, startup would slow to a crawl. I imagine some sort of delayed import mechanism could be crafted to avoid this problem though. Skip
Paolo> Symlink is one of the things that I would like to have on my XP Paolo> box... As Michael Hudson suggests, symlinks probably won't work anyway. I have something which looks like a symlink on my Win2k box. Windows calls it a Shortcut in listings. Is that not roughly the same thing as a symlink? Skip
Skip Montanaro wrote:
No, the NT file system as nothing as flexible as a symlink. If we are speaking about symlinks to files, windows shortcuts are barely usable only for pointing to executable for execution in a console (but they have a lot of redirection problems). If you *open* a shortcut you are actually opening the shortcut itself, not the thing it's pointing to ;-( Shortcuts to a directory is only usable by Explorer-like-programs, I don't know any usable operation with them from the console... The only way to use shortcut in python is to turn python shortcut-aware in the import machinery... --- Paolo Invernizzi Ps... cygwin is another story....
Skip Montanaro <skip@pobox.com>:
Only if it's transparent to the system calls which operate on files (e.g. open()ing it opens the file that it refers to, etc.) I don't believe that's the case with Windows shortcuts. Macintosh "aliases" have the same problem. (And now with MacOS X, we have both aliases *and* symlinks in the same system, just to add to the confusion...) Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+
Paolo Invernizzi <paoloinvernizzi@dmsware.com>:
Also, wouldn't that give two ways of getting each of the modules, with all the attendand Bad Things? Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+
On Wed, 2003-12-17 at 23:55, Guido van Rossum wrote:
Right, and I'd like to make that possible. The least disruptive way I see of doing that is to introduce a magic root package from which things can be imported from. E.g. import __root__.re
Yep, I've been bit by that too.
Yep. I haven't had '.' in my $PATH for a bazillion years. :)
There ya go! That's thinking outside the box: let's deprecate import altogether :) -Barry
[Guido]
Not useless at all -- that's bonding! Technically useless but humanly essential <wink>. It's no mystery *why* import is "a problem", but I'm not sure insight implies a cure: it's one of the very few places in Python where you can't explicitly say what you intend. Even in an inheritance hierarchy from hell, if I really don't want to chance that please.hit_me() will get satisfied from a class other than BlackJack, I can force it easily -- BlackJack.hit_me(please) There aren't any names to spell "the roots" of all the places Python may search to satisfy an import, so no way either to be explicit about intent. In contrast, nobody can mistake what import java.util.Random; intends. It's not a deep thing, it's just being able to name the intended context. I confess I'm keener on that than on growing more ways to spell more kinds of searches.
Barry Warsaw <barry@python.org> writes:
That issue I can understand. And I agree there should be a way to state it explicitly. One of the issues here is that this area is a bit under-documented (I know, I should read the source, but I don't have the time right now). At the moment, there are two cases: 1. Import from sys.path. This is what is being called an "absolute" import, and is nice and easy to understand. The key issue is that there is no way to *force* this interpretation in the face of option (2) below. 2. Import from "the package". This is the under-documented bit, but if I understand it correctly, it's basically that from within a module contained in a package, sys.path is conceptually *extended* to include the package's __path__ (which by default contains the directory of the package, but which can be user-modified). Now the big problem here is that behaviour (2) is useful. Simple "relative" imports of one module within a package from another module in the same package are common. Guido's (IMHO ugly) "dot" syntax handles that, by making users explicitly request option (2), and making the current import syntax *only* mean (1). But none of the proposed solutions handle the __path__ variable. I don't have any objection in principle to desupporting __path__ (heck, it would have made thinking about PEP 302 easier, if nothing else) but (a) it would need a deprecation warning, and (b) Guido himself offered a use case in <http://www.python.org/doc/essays/packages.html>. This definitely needs a PEP. If we're removing support for __path__, the implications need to be thought through (PEP 302, the pkgutil module, etc etc). If we're not, none of the proposals so far have covered how __path__ gets supported in future. A much simpler proposal, just providing an explicit way of saying "Import from sys.path *only*" may be OK without a PEP. But even then, I'd suspect we should have a PEP explaining *why* it has to be this simple. Go on, Guido. We'll be gentle if you write a PEP, and we won't set c.l.p on you :-) Paul. -- This signature intentionally left blank
[Barry]
[Paul Moore]
Right.
That's not a very useful way to think about it; when the module is found on __path__ something very *different* happens than when it is found on sys.path. The difference is noticeable if you ask the imported module for its __name__. If you the import was satisfied from __path__, then __name__ will include the package name. If it was satisfied from sys.path, it won't. And __name__ is related to module identity: all modules are stored in sys.modules using their __name__ as the key, so multiple imports of a module with the same __name__ end up referring to the same module (and only the first import causes the module's code to be executed).
Actually, what you consider useful is considered harmless by others. Many large packages, including Zope, have adopted a rule of always using absolute imports, to clarify what kind of import is being used.
__path__ has no bearing on the proposals, it is used for relative imports. I think your confusing about how the current import works caused you to think it is relevant.
We're not, but they have (by not mentioning it); but I agree that a PEP is needed.
This was where this thread started: someone proposed a way of spelling "import from sys.path only". My big problem with that is that *most* imports are intended to be satisfied by sys.path only, so this spelling should be the default, meaning the current spelling.
Go on, Guido. We'll be gentle if you write a PEP, and we won't set c.l.p on you :-)
Alas, I have no time. --Guido van Rossum (home page: http://www.python.org/~guido/)
Paul Moore <pf_moore@yahoo.co.uk>:
But none of the proposed solutions handle the __path__ variable. I don't have any objection in principle to desupporting __path__
I'll be worried if __path__ is likely to disappear as a result of all this. I'm using it in what will be the next version of my Python GUI library, to switch in different implementations of submodules depending on the platform. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+
On Tue, 2003-12-16 at 09:17, Gustavo Niemeyer wrote:
I guess the difference is when you're writing an application vs. writing a library. When writing an application (like Zope or Mailman), we already manipulate sys.path to include some non-standard locations. Incorporating a third-party package into the application then is a simple matter of arranging to have its top level directory somewhere on sys.path. In Mailman, I do this with the email package, and various codecs. When writing a library, that's harder because you're imposing requirements on the downstream consumer of your library. As an application developer, I don't have much sympathy for relative imports. As a library developer, I can see the point. -Barry
On Mon, 2003-12-15 at 14:34, Ken Manheimer wrote:
I've heard this before, but I still don't get it, so I'd like to tease this one out more. What does "plop the things in place" mean? Zope 2 has a few standard directories it adds to its PYTHONPATH. If I distribute a 3rd party package Zope wants to use, and that package uses absolute imports, you can just drop the package in one of those directories and it'll just work. Maybe you mean that you want to drop the mythical 3rd party package in somewhere other than a directory on Zope's PYTHONPATH and have it just work. I'd say that's too perverse of a use-case for Python to worry about. One problem with banning relative imports is that Python doesn't allow a top-level package name to live in more than one place on PYTHONPATH. It would be useful for Python to support such things out of the box, say if we wanted to adopt a Java style naming convention for package organizations. Jim uses the term "pure container package" to describe top-level packages that contain no real content in the package except subpackages. "zope" and "zodb" are examples of this in Zope 3. IWBNI Python had better/easier support for weaving together pure container package contents where subdirectories live in different locations on the file system. -Barry
Barry Warsaw writes:
+1 The .pth files processed by site.py support two interesting kinds of lines: - directories that should be added to sys.path, and - import statements The later alone is interesting enough, since it allows arbitrary initialization code to run when the import is executed. This still happens before all the .pth files have been processed; I suspect we really want the container package to be a little more flexible than that, at least allowing their __path__ to be computed after all .pth files have been processed. Perhaps it would be nice to have a hook (probably in sys) that gets called after site/sitecustomize are done? A "pure container package" could use a .pth file using "import mypackage", and the package's __init__.py would register a hook function in sys that searches sys.path for whatever directories should be used in mypackage.__path__. This also avoids the code being too specific to one way of thinking about packages. -Fred -- Fred L. Drake, Jr. <fred at zope.com> PythonLabs at Zope Corporation
Ken Manheimer writes:
I'd agree with Ken here. I think the importance of being able to do relative imports grows as well with the ability to compile Python into foreign frameworks, like Jython or .NET. I'd invent an ugly syntax for relative imports, and let the default be absolute. Bill
Why? I'd really like to see some concrete use cases for relative imports, other than just saving some typing.
Suppose you've developed a Python top-level application, composed of a package 'foo' containing a number of submodules and subpackages. You'd now like to add this to a Jython application as a library, part of a larger application. It's nice to be able to just add a higher-level package structure, com.mycompany.myapp, to the Python package, to fit in with the style of your application. Yet you don't want to edit the Python code to 'import com.mycompany.myapp.foo.submodule' everywhere; you'd like 'import submodule' or 'import foo.submodule' to continue working. Often the code has to work in both the stand-alone and the federated case. This is just a special case of the federation problem that we've seen for years in distributed computing; previously top-level names get federated when new higher-level constructs are formed. Ken mentioned enterprise programming; it's another case of this. Two companies merge and start merging their code. It's much better to be able to write (in the new higher-level module bletch.py) import foo import bar and have the subpackages under foo and bar continue to work without rewrites, than have to go through all the code under foo and bar and change it to know about the new higher-level bletch namespace. Bill
On Mon, 15 Dec 2003, Guido van Rossum wrote:
As i've said before (though it's been a while) i think people want this, it would be useful, and i think it would work well. Barry wrote, in response to guido's message: < Actually, if truth be told, I'd love to just ban local imports. I < understand that might be controversial though <wink>. I don't understand the desire to totally prevent relative imports. It would be quite useful in avoiding growing module-naming problems, where you have to be careful about shadowing a global module with one in your package. I *can* understand objecting to the ambiguity in the current situation, where you don't know whether a module import will resolve to a sibling module or one in higher up in the hierarchy. That is why i strongly prefer having a leading dot to explicitly signify a relative import, and originally proposed it that way. I expect that the elimination of the ambiguity would mitigate the objection to relative imports - and we would do away with the shadowing problems, and even have easy relocation of modules, if there are cases where software does need to move around. Ken klm@zope.com
I don't understand the desire to totally prevent relative imports.
It's because of TOOWTDI.
I don't see the problem, or I misuderstand what "it" refers to; it seems you have this backwards if it refers to relative imports. Say my package P defines a submodule sys. If we require absolute imports, there is no ambiguity: the submodule sys must be imported as P.sys while the standard sys module can be imported as simply sys. --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote:
If absolute imports were to be the only type allowed, then it would seem that the only possible location for naming conflicts is in the _first_ element. So if I wanted to use two different third party modules, both of which have unfortunately chosen the same name for the top-level package, the only way to let them co-exist is to redo _all_ of the imports in one or the other of them. Whereas, if relative pathing is possible, I believe that all I have to do is push them one level down in the package heirarchy (using distinct names that I invent), and neither of them ever even knows about the other's existence. I can get at both of them unambiguously, by using my new top=level names, and neither package even knows that it is no longer starting at the top of the import heirarchy. Or is there some other solution being proposed to this problem, and I just haven't understood it? Cheers, Nick. -- Nick Coghlan | Brisbane, Australia Email: ncoghlan@email.com | Mobile: +61 409 573 268
On Tue, 2003-12-16 at 05:32, Nick Coghlan wrote:
True.
Has this ever happened to you in practice? It seems like the way out would be to start adopting a Java-like convention for package names. The problem with that in current Python is that you can't (easily) weave a package's contents from different locations in the file system. -Barry
Barry Warsaw wrote:
Well, no. But others seemed concerned about it, so I thought I'd try to clarify the problem - the scripts I write don't need anything more than the standard library and win32all.
I still like the various suggestions for _giving_ those parts of the system special names. That's what I was trying for, even if the names I chame up with weren't any good. Cheers, Nick. -- Nick Coghlan | Brisbane, Australia Email: ncoghlan@email.com | Mobile: +61 409 573 268
On Sun, Dec 14, 2003 at 04:31:09PM -0500, Raymond Hettinger wrote:
http://mail.python.org/pipermail/python-dev/2002-October/029770.html [and the rest of that thread] A 3% slowdown in the common case was considered too big to allow a feature like this to be included. The slowdown might be smaller if the builtin module wasn't allowed to be anything but a dict instance, or might disappear if there were two versions of ceval.c were included, and one used PyObject_ APIs and the other used PyDict_ APIs (as long as only one was fighting for space in the cpu cache, main memory, etc) Jeff
At 02:26 PM 12/15/03 -0500, Raymond Hettinger wrote:
There is a workaround for this in CPython, which used to be used by Zope, and which I have used on occasion: scan the code object for LOAD_NAME opcodes, and note what names are used by the code block. Then, look up those names in your non-dictionary dictionary, and copy them into a real dictionary. So, it's got to be something pretty specialized to really need PyObject_* APIs instead of PyDict_* APIs in CPython at present. Why penalize *everything* for such an specialized need?
Raymond wrote:
+0. if you can do it without affecting performance in the dictionary case, that is. -1 if you can't.
II. Timer.timeit to take an optional context argument.
+1
III. enumerate to take an optional argument:
+0 (it's not that hard or expensive to add an offset inside the loop)
IV. list.sorted() to become just sorted(). All of the common use cases read better without the "list." prefix:
+1 one more: V. a join() builtin (join(x,s) becomes s.join(x)). All of the common sep.join() use cases read better without the string literal prefix. (I'll leave the rest for later) </F>
On Sun, 2003-12-14 at 17:05, Fredrik Lundh wrote:
V. a join() builtin (join(x,s) becomes s.join(x)). All of the common sep.join() use cases read better without the string literal prefix.
Really? I predict everlasting confusion on argument order. It's not obvious from the method name whether the joining separator comes first or second. My wish list: V <wink>. Explicit global imports. I want a way to spell "import this from the global standard library even if there's a this.py in the local directory". VI. PEP 318 or something like it. -Barry
I'd much rather invent new syntax to spell *local* imports. I like from .module import something You can even generalize and write from ..module import something to reference your package's parent package. :-) --Guido van Rossum (home page: http://www.python.org/~guido/)
On Mon, 2003-12-15 at 11:03, Guido van Rossum wrote:
Actually, if truth be told, I'd love to just ban local imports. I understand that might be controversial though <wink>. The next best thing would be to force local imports to use special syntax (I was going to say "ugly" :) like what you suggest above, but I'm afraid that that might be more disruptive, requiring future imports and such. I think having a syntax for spelling global imports can be done in a backwards compatible way for 2.4. -Barry
Guido> I'd much rather invent new syntax to spell *local* imports. Barry> Actually, if truth be told, I'd love to just ban local imports. Barry> I understand that might be controversial though <wink>. Okay, so what are global and local imports? I thought I understood after Barry's first post, but not now. I see the <wink>, but figured there's still some germ of desire in Barry's "ban local imports" comment. What kind of imports do we have today? Do they relate to the position of the import statement in the code or to the location of the module being imported along sys.path? Skip
Traditionally, we've called these absolute (== global) and relative (== local) imports. The terms relate to the module's full package name, usually equivalent to its location on the filesystem. sys.path and the placement of the import statement do not enter into it, except that sys.path is the very definition of absolute import. --Guido van Rossum (home page: http://www.python.org/~guido/)
>> Okay, so what are global and local imports? Barry> Sorry, let me rephrase. I'd love to ban relative imports. Barry> Personally, I think all imports should be specified using Barry> fully-qualified absolute package paths. That's fine with me. Maybe continue to use "absolute" and "relative" (and throw in "package" somewhere) instead of "global" and "local" when referring to package import semantics? "global" and "local" have a well-established meaning in day-to-day Python programming which doesn't overlap much with the import mechanism. Skip
Barry Warsaw wrote:
If I'm understanding this right you want to ban imports that are machine independant. That can't be right, but that's what it sounds like to me. To me an "absolute import" or a "fully-qualified absolute package path" sounds like you know the exact directory that the import is occuring from. E.g.: /home/someone/projects/python/workingprogress/currentproject/thisone.py I can see the use for being able to import that, but usually I would prefer to import either from something relative to the current directory, or relative to the python site-lib, or some such. It's also quite nice to not need to bother to exactly specify just which of those I will find it in. (This is what the pythonpath is for.) Now to me an import relative to the current directory is a relative import, and I assume that this is also what is meant by a local import. By allowing files to cluster relative to the code being written one is able to keep related projects and libraries together rather than just having all the code spread out aimlessly in a huge flat directory. I don't see in what what this is inferior. Since you didn't say why you wanted to ban relative imports, I don't understand your reasons. I can't even guess at them. Even if it were for some efficiency condern about hierarchical directories, that would merely say that the compiled modules should be stuffed into a library folder. (Well, more than one...the user needs to have write permission to the libraries that he creates, but it's just as well that he not have write permissions to the system libraries...without taking special steps.)
On Wed, 2003-12-17 at 11:55, Charles Hixson wrote:
Barry Warsaw wrote:
If I'm understanding this right you want to ban imports that are machine independant. That can't be right, but that's what it sounds like to me.
Nope, because "absolute package paths" are always calculated relative to directories in sys.path when mapping a package path to a file system path. -Barry
Works for me. :-)
Well, but since you want all imports to be global, it'd be insane to introduce *new* syntax for global imports, wouldn't it? --Guido van Rossum (home page: http://www.python.org/~guido/)
I think this ought to be a *global* flag rather than a per-module flag. E.g. after setting sys.allow_relative_import = False all imports anywhere would be interpreted as absolute imports. This would mean you couldn't have some code that still uses relative imports, but the problem with the __future__ statement is that it seems so pointless: packages that only use absolute imports don't need it, and packages that use relative imports break if it is used. About the only time where the __future__ statement would make a difference is when a package defines a local module whose name is the same as that of a global module, *and* the package also wants to import the global module. I would personally solve that by renaming the local module though... --Guido van Rossum (home page: http://www.python.org/~guido/)
But there is no backwards compatibility issue. Absolute imports always work, so instead of adding a "relative imports allowed" to your package, you can just change the package to use only absolute imports. Using only absolute imports is totally backwards compatible (unless your toplevel package name is the same as the name of a submodule of the package, which seems pretty sick). --Guido van Rossum (home page: http://www.python.org/~guido/)
On Mon, 15 Dec 2003, Guido van Rossum wrote:
I guess i think there is a case to be made for relative imports, and it becomes apparent as an enterprise grows. Increasingly at Zope corp we are mixing and matching packages to deliver applications to a customer. This is very encouraging, because it means we are actually getting reuse out of the packages. Currently, we can just plop the things in place, and use them. Without relative imports, we would have to be editing the imports in the packages in each place we use it. This would increase the burden of using the packages and even more of feeding back actual fixes - which we then have to distinguish from what i would see as gratuitous changes to edit import names. If you would grant there's use to avoiding those edits, and so there's use to having relative imports, then you might see a reason to solve the problems where the names in a package conflict with those along the load path. Otherwise, if there's only absolute imports, there's no ambiguity to resolve. I'd say there's a legitimate need for relative imports, and we need some explicit gesture to disambiguate between a relative and absolute import, one way or the other. Ken
I know this line of reasoning fairly well. You are taking 3rd party packages (or perhaps internally developed packages) and copy them to a *different place in the package namespace tree*. Right? But why do you have to give those packages a different full name? That's the question that I've never seen answered adequately.
I think that moving packages around in the package namespace is a bad idea. But maybe you can give me an answer to the question above to convince me. --Guido van Rossum (home page: http://www.python.org/~guido/)
But why do you have to give those packages a different full name? That's the question that I've never seen answered adequately.
I have done this many times, so let me try to describe at least one legitimate usage case. A couple of weeks ago I wrote a software which needs a third party package to work. OTOH, since it's a small package, I don't want to force the user to go after the package, even because I'd have to ensure that my code always work with the available version of that package. Thanks to the relative module importing mechanism, solving that is no harder than copying the third party package into my own package namespace. This idea could probably be expressed in some other way, hacking sys.path or whatever, but I belive this is a fairly common pattern, and I vote for introducing a scheme to differ between local/global importing which would not break the current flexibility. -- Gustavo Niemeyer http://niemeyer.net
A lot people have presented a good case for relative imports. Nobody has argued to keep the status quo (where imports are ambiguous about whether they are meant to be absolute or relative). So I suggest that in 2.4, we introduce the leading dot notation for relative import, while still allowing relative import without a leading dot. In 2.5 we can start warning about relative import without a leading dot (although that will undoubtedly get complaints from folks who have code that needs to work with 2.3 still). In 3.0 we can retire ambiguous import. The use case for multiple dots should be obvious: inside a highly structured package, modules inside one subpackage must have a way to do relative import from another subpackage of the same parent package. There is the remaining issue of what exactly the syntax would be. I propose to extend the from clause to allow one or more dots before the dotted name, and to make the dotted name optional if at least one leading dot is found. I propose not to change from-less import. Examples: from .foo import bar from .foo.bar import xxx from . import foobar as barfoo from ..foo.bar import * from ... import foobar, barfoo Grammar change (see Grammar/Grammar): dotted_name: NAME ('.' NAME)* | '.'+ [NAME ('.' NAME)*] --Guido van Rossum (home page: http://www.python.org/~guido/)
On Tue, 16 Dec 2003, Guido van Rossum wrote:
On a related and minor note, can we please expand the grammar to allow from...import statements to continue to the next line after a trailing comma? For those of us with large, complex, and componentized projects, it would be very nice. e.g.: from OPAL.Modules.Financial.Reports.COGS import generate, webOptions, normalize, row_ops, summary instead of: from OPAL.Modules.Financial.Reports.COGS import generate, webOptions, \ normalize, row_ops, \ summary This has been a minor nit that bothers just under the threshold of fixing it myself. ;) Thanks, -Kevin -- -- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (440) 871-6725 x 19 E-mail: jacobs@theopalgroup.com Fax: (440) 871-6722 WWW: http://www.theopalgroup.com/
I guess this could be implemented by allowing [NEWLINE] after the comma in the grammar. But I'm -0 on this change; I fear that someone accidentally adding a trailing comma (or leaving it in after some rearrangements) will be very puzzled at the syntax error that comes somewhere in the middle of the next statement: from foo import bar, bufsize = 12 ^syntax error --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote:
you could require parentheses: from OPAL.Modules.Financial.Reports.COGS import ( generate, webOptions, normalize, row_ops, summary ) (after all, from-import is a kind of assignment, and "a, b = c" is the same thing as "(a, b) = c"...) </F>
Guido:
How about a "suite" of things to import? from OPAL.Modules.Financial.Reports.COGS import: generate, webOptions, normalize row_ops, summary Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+
Nah. But I liked the suggestion seen here earlier of allowing parenthesis: from OPAL.Modules.Financial.Reports.COGS import (generate, webOptions, normalize, row_ops, summary) --Guido van Rossum (home page: http://www.python.org/~guido/)
[Guido]
So do I. Something I'm still acutely aware of despite not using Emacs for years is how much trouble it would be to teach python-mode.el about a unique new way to continue a statement. Parens would "just work". Ditto for IDLE, and tokenize.py, etc etc etc etc. I bet tools parsing import statements for *content* (like pychecker, and perhaps pyclbr.py -- the others mentioned are just looking at form) would need modification regardless, though.
Second impression: "suites" contain statements, not a list.
That's why I put "suite" in quotes. It wouldn't really be a suite, just an indented block of stuff. But if you want, it could be a real suite that only happens to allow import statements... from OPAL.Modules.Financial.Reports.COGS: import generate, webOptions, normalize import row_ops, summary Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+
On Wed, Dec 17, 2003 at 02:53:35PM +1300, Greg Ewing wrote:
But this is also a very odd suite, because you can't use other statements... But it does make me think of: with OPAL.Modules.Financial.Reports.COGS: import .generate, .webOptions, .normalize import .row_ops, .summary Which is like a bastard child of the relative import proposal and the with statement proposal... <wink> -Andrew.
Hi, "Guido van Rossum" <guido@python.org> wrote
I'm +1 on it, I'd only wish an additional from __future__ import strict_import (or whatever seems a better name) to allow a module to indicate that every not explicitly relative import should be absolute. This would allow module authors to be explicit from the start and it is also a safety net against accidentially adding a relative module with the name of some absolute one. best regards Werner
On 16-dec-03, at 19:26, Guido van Rossum wrote:
Is "from . import *" allowed? Whenever I start to think about it I tend to go into oscillation (yes because within package foo its the same as "from foo import *" on the outside; no because "import *" isn't allowed today either). -- Jack Jansen, <Jack.Jansen@cwi.nl>, http://www.cwi.nl/~jack If I can't dance I don't want to be part of your revolution -- Emma Goldman
Guido:
If I understand correctly, the rule you're working towards is that if the path starts with a dot, it's relative, otherwise it's absolute. Is that correct? I'm not sure whether eliminating the "ambiguous" category of module references is the best idea. There are situations in which it's actually what you want. Suppose there are two top-level modules A and B, and A imports B using a bare name. Then someone decides to put A into a package. The reference to B still works. Then they decide to put B into the same package as well. The reference to B still works. But if only unambiguously absolute or relative references are possible, there is no way for A to refer to B that works in all these combinations. So I think we really want *three* kinds of module reference: A: Explicitly absolute B: Explicitly relative to the current module C: Searched for upwards in the package hierarchy from the current module (Note that C is a generalisation of the current "ambiguous" references which only look in two places.) Suggested syntaxes for these: A: a.b.c. Path ends with a dot B: .a.b.c Path begins with a dot C: a Path neither begins nor ends with a dot a.b.c Note that ALL current module references would be type C according to this. Yes, this is a semantic change, but one that I believe would be for the best in the long run, and don't think would cause a lot of upheaval. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+
In article <200312170022.hBH0Mne16539@oma.cosc.canterbury.ac.nz>, Greg Ewing <greg@cosc.canterbury.ac.nz> wrote:
Is funny punctuation really the right way to spell an important distinction like this? -- David Eppstein http://www.ics.uci.edu/~eppstein/ Univ. of California, Irvine, School of Information & Computer Science
David Eppstein <eppstein@ics.uci.edu>:
Is funny punctuation really the right way to spell an important distinction like this?
Maybe not, but I haven't seen anything obviously better proposed yet. I've no idea what else to to for relative paths. A leading dot doesn't seem all that bad to me for that. I concede that a trailing dot would look rather odd, though. The absolute-path case could be spelled using a special name for the root of the package hierarchy, but then we need to think of a name that's sufficiently unlikely as a real module name. I don't think I like the idea of re-using the word "global", although I'm not sure. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+
Greg Ewing wrote:
Alternate spellings (plus category D, which is "Python 2.3 semantics"): A: from __absolute__.dotted.name import foo B: from __relative__.dotted.name import bar C: from __heirarchy__.dotted.name import foobar D: from dotted.name import barfoo I believe this spelling would only require import to handle the special cases as the first 'package' name (although having the compiler recognise them may be a later optimisation), and should also work with straight "import __absolute__.dotted.name.foo" statements. Then Guido's migration plan would apply to the progression of the D semantics from the Python 2.3 model to being a shorthand for the absolute semantics. And once _that_ has happened, then we could get rid of the '__absolute__' version in the name of TOOWTDI. Regards, Nick. -- Nick Coghlan | Brisbane, Australia Email: ncoghlan@email.com | Mobile: +61 409 573 268
Just to give you a practical example, we are using python for internal tools in the electronic Company I'm working for. I needed to have a non ambiguous file lookup when using import ( We set external dependencies for the flow when opening a file, so you can't use a 'if this file exist' check, when opening a file, it must always exists) I made a special import function, that was looking for a prefix to the package name: import loc.dotted.name import glb.dotted.name import __.dotted.name import std.dotted.name import xxx.dotted.name -> raise an error "loc", look in the current directory of the module (looking to his __file__ attribute) "__ "look to the parent directory "glb" look at the root of the user working repository ( a builtins set before anything else is done , users don't call python they the program myPython ) "std" look outside the database ( in the python installation dir mainly) and set back the old import behaviour to ensure that import inside standard modules don't fail ) I know the loc,glb __ and std names are no good for a general case as modules can have these names, but this is just to give you an actual use of these relatives imports. Boris Nick Coghlan wrote:
Nick Coghlan <ncoghlan@iinet.net.au>:
That's where I disagree - I think that type C semantics should be the default/least ugly/easiest to get, not type A. But I wouldn't like any of the others to be as ugly as those, either. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+
On Tue, 2003-12-16 at 13:26, Guido van Rossum wrote:
I'll just note that where the current status quo trips /me/ up most is when I accidentally have a local module with the same name as a global module, and then I write an import statement expecting to get the standard library module, but end up getting the local module. That's why when I tend to think about this, I start wanting a way to spell "definitely give me the global one, no matter what". IOW, I feel like I want a way to bypass relative module lookups. I'm still catching up on this thread, but I wanted to throw this out there... -Barry
On Wed, Dec 17, 2003 at 09:33:43PM -0500, Barry Warsaw wrote:
Alternatively, maybe what you want is a way to say "definitely give me the standard library one, no matter what", e.g. from stdlib import codecs This would allow other packages to have their own codecs module/package without worrying about confusing import errors. To make this backwards compatible, though, we'd probably need the equivalent of C++'s "using namespace" statement, with no declaration being equivalent to: using namespace stdlib Or perhaps more python-like: __import_base__ = 'stdlib' Python 3.0 might default to __import_base__ = '', to force people to be explicit. Actually, the proposed with statement could be used here, e.g. import stdlib with stdlib: from . import codecs, user, gc looks reasonably clear to me, and for long package paths, it could be very convenient: import zope.app.interfaces with zope.app.interfaces: from .location import ILocation from .foo import IFoo, IBar # etc -Andrew.
On Wed, 2003-12-17 at 22:04, Andrew Bennetts wrote:
Interesting. I see the subtle distinction. One searches only the standard library, the other searches every directory on sys.path. I'm not sure I'd have much need to restrict the search to just the standard library. Plus, would that include site-packages? -Barry
On Wed, Dec 17, 2003 at 10:08:21PM -0500, Barry Warsaw wrote:
I should have been clearer. What I'm basically proposing is to make site-package called "stdlib", leaving a default python install with a very uncluttered top-level namespace, thus reducing unexpected name conflicts. Otherwise, this idea doesn't require any modification to import semantics, except for some magic to make stdlib the default root for imports for backwards compatibility. Thinking about it a bit harder, though, I'm not sure it's as easy as I initially thought, because this would then make importing 3rd party packages not work, i.e. currently you can do: import urllib import pygtk without problems, because they are both top-level. If urllib moved into a "stdlib" package, it's hard to imagine a clean way where old code that does those two lines would still work. I still like the idea of having the standard library in its own package, but I guess that will have to wait until Python 3.0. -Andrew.
Andrew Bennetts <andrew-pythondev@puzzling.org> writes:
I still like the idea of having the standard library in its own package, but I guess that will have to wait until Python 3.0.
Guido tends to shoot people who suggest this, watch out :-) Cheers, mwh -- Lisp nearing the age of 50 is the most modern language out there. GC, dynamic, reflective, the best OO model extant including GFs, procedural macros, and the only thing old-fashioned about it is that it is compiled and fast. -- Kenny Tilton, comp.lang.python
On Thu, Dec 18, 2003 at 02:16:45PM +1100, Andrew Bennetts wrote:
I'm at least a little confused with what people want, as far as I can tell import semantics have three things * what module to import * what name to give that import * where to look for it The 'from' keyword seems to be getting overloaded to mean both what module and where to find it, which the 'stdlib' tries to resolve. Most of the discussion has been about how to improve 'where to look' I like syntax that reads most important left-to-right, so what about from MODULE import NAMES as RENAME searching HOW or import NAMES as RENAME from MODULE searching HOW searching could be 'asbolute' 'relative' or any of the other suggested words. If you wanted to get fancy it could be a list, if it isn't a list people who truly care could cascade try/except on ImportError. -jackdied ps, I like the second version better but it is less like the current version, which is a drawback.
[Jack Diederich]
[Barry Warsaw]
I want to think about that more, but at first glance, it has some appeal. Neat!
Ditto. I think Jack's on to something there. I particularly like the second form, as it's an ongoing irritation to me that some imports begin with "from" and others with "import". In the current "from" form, the most important thing to me is almost never the module I'm importing the thing from, but (of course) the thing I'm importing. Going from import math to import sin from math (or vice versa) is also an easier edit than from import math to from math import sin (because the former doesn't require transposition). Adding a distinct clause to influence search details seems downright Pythonic. Maybe *too* Pythonic <wink>.
Jack Diederich <jack@performancedrivers.com> writes:
I like this. It feels much more Pythonic than the various suggestions around magic dots in the name. It's explicit, easy to understand, and easy to extend (extra possibilities for HOW could be added with little difficulty). Paul. -- This signature intentionally left blank
Jack Diederich wrote:
This would work really well for any uses cases I can imagine having. It also has the following benefits: 1. No magic punctuation 2. People who don't need it, just never use a 'searching' clause 3. People encountering it for the first time have a clear flag letting them know something fancy is going on 4. These people have a word they can look up in the Python docs I hadn't even thought of point 4 prior to considering Jack's proposal. If I encounter "import .fred.mary" in some python I didn't write, how the heck do I find out what it means? [...]
ps, I like the second version better but it is less like the current version, which is a drawback.
To head a little further down this sidetrack, the from version of 'import' has puzzled me for a while: A. from mypkg import foo B. import mypkg.foo as foo What, if any, are the semantic differences between A & B? Is it only that A2 works, and B2 doesn't? A2. from mymodule import * B2. import mymodule.* Cheers, Nick. -- Nick Coghlan | Brisbane, Australia Email: ncoghlan@email.com | Mobile: +61 409 573 268
This got several encouraging responses, so I have to consider it. Unfortunately it looks like 'searching' would have to become a reserved word; if we use the same trick we used to avoid making 'as' a reserved word, that would be too confusing to decode in the code generator. The HOW could be just an identifier then. I'm not sure how well it'll read in actual examples. Let's try. Adapting an example from Opal that was used here before... from COGS import generate searching relative To me, that looks a lot like someone left the commas out of from COGS import generate, searching, relative In general I'm not a big fan of "wordy" clauses like this -- they work better in languages like SQL or COBOL because those are case-sensitive so you can use a difference in capitalization there; there the former would be FROM COGS IMPORT generate SEARCHING RELATIVE (using imaginary syntax) while the latter would be FROM COGS IMPORT generate, searching, relative while in Python the long sequence of lowercase words becomes a blur. So if the two alternatives are simply 'searching absolute' and 'searching relative', with the default being 'searching absolute' in Python 3.0 (but the current ambiguous search in Python 2.x) I'd still prefer making the distinction with "from ...MODULE" vs. "from MODULE" rather than with a searching clause. And I still prefer the single-dot syntax over three dots, because it can be used to spell importing from the parent or grandparent package explicitly. A separate suggestion is to switch from "from X import Y" to "import Y from X". This seems a gratuitous change even if it reads or edits marginally better. Now's not the time to tinker with marginal stuff -- that should've been done 10 years ago. --Guido van Rossum (home page: http://www.python.org/~guido/)
On Sun, Dec 21, 2003 at 08:48:01PM -0800, Guido van Rossum wrote:
Too bad the decorator PEP (PEP 318) seems to favor 'as' in the syntax, import foo [relative] is more explicit than the dot syntax and has a nice visual seperator while avoiding the wordiness problem of 'searching'
moving the 'import' to always be first would alleviate this a bit because 'searching' would always follow a non-list. import generate from COGS searching relative or if we decorate import generate from COGS [relative] from COGS import generate [relative]
as above, moving import to be first helps but doesn't eliminate the wordiness problem[*]. 'as' and 'from' are short enough to be easilly seen as keywords, 'find' or 'look' reads less like english but would be easier to pick out visually than 'searching'
So if the two alternatives are simply 'searching absolute' and 'searching relative'
Did someone mention 'searching scan' in dusty corner somewhere? One drawback to the dot syntax is if there can be more than two ways to do the search. Adding a third way is easier w/ the searching clause instead of adding a 'bang' syntax to the 'dot' syntax . The decorator would do one better by being a placeholder for anything that modifies the import.
Damn, I really liked that part. I don't get tripped up on from X import Y anymore, but I did when I started w/ python. benign-user-for-life-ly, -jackdied [*] I use an emacs hack to display 'lambda' as 'L' to avoid the same keyword-reads-like-a-variable effect.
I just realized I've been forgetting to CC the list. It is a bit long, but I'll cut out the non-relevant portions that everyone has seen before. In chronological order below. I ask the rest of you, does Some sound so crazy? - Josiah *** message from Josiah Carlson on Sun, 21 Dec 2003 19:05:45 -0800 ***
I've read PEP 754, and yes, the author talks about having a proper Infinity, -Infinity and Not a Number (PosInf, NegInf and NaN respectively), but considering that it has been in the 'draft' state since March 28, has not been updated, and it doesn't seem like Python is any closer to having proper definitions now than it was ~9 months ago, I'm not going to put my eggs in the IEEE 754 Infinity basket. In terms of having something useful on a platform, there is only so useful that infinity can be. The below is from the windows verstion of Python 2.3.2
The real trick is that it is not so much about being able to have max(Some, a) -> Some, it is about being able to have min(Some, a) -> a. Then you don't need to worry about sys.maxint not being big enough (because there are larger numbers), nor do you need to worry about float casts of large integers producing OverflowErrors. As for an application, many path searches in graphs start out with initializing distances to infinity, Some would be suitable for this. Other algorithms require a negative infinity for this, None has been working fine. As an initialization value, I believe that Some would be quite a useful object. Certainly Some seems like a strange way to spell infinity, but it is a logical infinity, one that you can compare things against, anything, not just Python floats or things that can be cast into Python floats. In terms if PEP 754, if it were actually implemented, certainly there would be some question as to whether Some would be equal to PosInf. If so, then would -Some be equal to NegInf? And if so, would None be equal to NegInf? Maybe such things would require a float cast. I talk about all this in my PEP submission, which isn't up yet. I suppose I probably should have given it a few days before discussing it, but yeah. - Josiah *** end message from Josiah *** *** message from Guido van Rossum on Sun, 21 Dec 2003 20:45:34 -0800 ***
I expect your PEP will fare no better, and given its weirdness, probably much worse. Do you know of any language that has a similar concept? --Guido van Rossum (home page: http://www.python.org/~guido/) *** end message from Guido *** *** message from Josiah Carlson on Sun, 21 Dec 2003 21:47:34 -0800 ***
Is None so weird? All I am proposing is a symmetric value to None. Just like there is a -1 and 1, -Infinity and Infinity, None logically should have Some. Maybe not so much -Some == None, but as I propose in the PEP, possibly Some == not None. - Josiah *** end message from Josiah ***
"Josiah Carlson" <jcarlson@uci.edu> wrote in message news:20031222000819.10EC.JCARLSON@uci.edu...
Is None so weird? All I am proposing is a symmetric value to None.
To me that would be All of AllType. Some is in between. Can you think of any uses for All other than being max element? TJR
Not really. However, I was thinking of Some in the context of graph and dynamic programming algorithms. When programming some algorithms in Python, you need to either pick some large constant (that you hope will be big enough), or have a test to see if a value is None. That is ugly and very not Pythonic. I do like All and AllType though, which gives a better name, as suggested by Guido. - Josiah
Josiah Carlson <jcarlson@uci.edu> writes:
I ask the rest of you, does Some sound so crazy?
To be honest, yes. To me, None is not the low end of a scale, rather it is a unique distinguished value, more related to the concept of "uninitialised" or "not used" than to "smallest". The fact that it compares less than every other value is to me incidental, and arbitrary. I see None as inhabiting the same sort of niche as the SQL NULL value.
From the above, you'll gather that from my viewpoint, there really is no "other extreme" from None, any more than there is from zero.
Paul. -- This signature intentionally left blank
[Paul Moore]
Arbitrary?! I clearly recall when the decision was made that None would compare less than anything else. I was in the same room with Guido, and this is a verbatim transcript of the debate: "As long as we're changing everything here anyway, maybe comparing with None should yield a consistent result." "Like what, maybe smaller than anything else?" "Sure. Or larger." "Ya, one of those. It's not worth it if it just ends up in the middle somewhere." "What does Python do now?" ... [head-scratching] ... "So what did it do before?" ... [more head-scratching] ... "Well, that's no help, then." "The Reference Manual doesn't promise anything about how None will compare, just the 'consistent total ordering' bit". "So that doesn't constrain us." "No, it's neutral." "So maybe it should just be smaller than everything else." "Sure. Or larger." "If it were smaller, then when you sorted a mixed-type list, all the Nones would float to the start." "Good point. If it were larger, they'd float to the end." "I wouldn't like that much." "Me neither." "OK, it's settled." Two lines of code later, the Only Objectively Correct result was achieved. history-is-more-impressive-in-textbooks-ly y'rs - tim
On Sunday 21 December 2003 10:52 pm, Jack Diederich wrote:
I like Jack's original idea and I've been chewing on it for a few days. His was: import NAMES as RENAME from MODULE searching HOW My humble suggestion: import NAMES import NAMES in MODULE [in HOW] import NAME, OTHER in FOO in RELATIVE import NAME as RENAME in MODULE in ABSOLUTE Playing with it: import sys import path, stat in os import AClass, AValue in AModule in __search__ import AType in Package.Library in __absolute__ It emphasizes the name that's imported and it reduces+reuses a keyword. Reusing "in" is a stretch, of course, and it reading it twice in the same statement might be confusing for some. I don't really like the dot/triple dot notation... the leading punctuation smells perl-ish. :) Only-a-random-two-cents-ly yours, -- Troy Melhase, troy@gci.net -- Merry Christmas!
On Mon, 2003-12-22 at 04:03, Troy Melhase wrote:
I don't really like the dot/triple dot notation... the leading punctuation smells perl-ish. :)
I don't either, although my biggest problem with it is that it isn't clear just by looking what the punctuation actually means. I just fear this will be a permanent head scratcher. Here's my entry in the sure-to-be-shot-down-by-the-BDFL-alternative -syntax game: import foo by absolute import foo.bar by relative as baz import foo by searching as bar import foo by absolute from bar import foo as bar by absolute from baz IOW, 'by' is a reserved word in the import statement context, and it specifies how to search for modules. The question then is whether 'absolute', 'relative', and 'searching' are also reserved words or built-ins and how those names are equated with the module location algorithms. -Barry
This argument by itself has very little value. The question isn't whether it's self-describing (very few features beyond decimal integers are); the question is whether it's easy to remember. There are two subclasses of that: easy to remember what it means when you next see it, and easy to remember how to spell it when you next need it. IMO the dot syntax does fine here.
Here's my entry in the sure-to-be-shot-down-by-the-BDFL-alternative -syntax game:
There have been so many alternatives proposed this morning that I have to spend some time comparing them all... --Guido van Rossum (home page: http://www.python.org/~guido/)
On Mon, 2003-12-22 at 10:36, Guido van Rossum wrote:
Sorry, it was a pre-coffee argument. To me, the dot-syntax doesn't really tell me whether it forces a relative search or an absolute search. The dot tells me that something different is happening, so maybe when absolute is the default, it tells me there's a switch to relative searching going on. I'm not sure though, so I do think it may potentially fail the "easy to remember what it means when you next see it" test. OTOH, it would be nice to see a Greg Wilson-style newbie poll.
Hopefully, this stuff will make it into the PEP so you don't have to troll through suggestions buried in endless email threads. -Barry
On Mon, Dec 22, 2003, Barry Warsaw wrote:
That's the plan; I expect that people will review the PEP to make sure their pet proposal is listed. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization.
Thanks -- I'll stop tracking this thread closely then until I see the PEP. --Guido van Rossum (home page: http://www.python.org/~guido/)
I've been monitoring all of this discussion about relative imports, and have my own humble idea about how to spell relative imports. The first part stems from Guido's idea about using a '.' at the beginning of a relative import. It seemed a bit weird at first, but I eventually became very comfortable with the idea after thinking about it in respect to absolute imports. I'll explain what I mean. If you currently want to import a module 'a.b.c.d', you write: import a.b.c.d Now, if you're in the module 'a.b', you could still write: import a.b.c.d ... but the 'a.b' part is redundant because you're already in the module 'a.b'. If you take away 'a.b', then you end up with: import .c.d This flowed well in my head, and looks natural. However, the suggestion about '.' or '..' referring to the parent looks _ugly and unnatural_. Let's say '.' was used to spell "parent module", and let's assume that the module 'a.b.c.d' wants to import 'a.b.e'. The absolute import would be spelled: import a.b.e ... and the relative import would be spelled: import .....e # .[parent].[parent].e Yuck! (no offense :) What's needed, I believe, is an identifier/keyword/symbol that spells 'parent' that isn't '.'. Assuming that symbol is represented by '[parent]', the above relative import would look like: import .[parent].[parent].e ... which is a hell lot more clear IMHO. My initial idea is to spell parent '__parent__', and to have '__parent__' be a module-level variable in each module that refers to the parent module (where a top-level module might have '__parent__' refer to itself, or to None). Continuing with the above example, the relative import would be spelled: import .__parent__.__parent__.e ... but I'm not completely convinced that there's not a better identifier/keyword/symbol that could be substituted for '[parent]'. If I'm completely off track, then tell me and I'll continue to lurk until I get a better feel for the happenings of python-dev. -- Devin devin@whitebread.org http://www.whitebread.org/
On Mon, 22 Dec 2003, Devin wrote:
I also look at it this way, and find it both intuitive and mnemonic. My model is slighly different than yours, though, in a way that simplifies the expression for going up the relative chain. I'll explain...
Truly, yuck. But in my model, the leading '.' dot, itself, stands for the containing package, and '..' stands for the containing package's package, and so forth: import ..e # [parent-of-parent].e I don't think it's yucky at all, this way. (I also don't think it's un-pythonic - python uses punctuation all over to denote type literals ('{'/'}'), cues which delineate statement suites (':'), and so forth.) In fact, i think it's a lot more mnemonic and encompassing than the alternatives that use words (though i haven't looked at all the alternatives very carefully yet). Even using the leading '.' syntax, it would be good to have a specific identifier for the containing package, realized in each module and respected as a special identifier by the package machinery. I like '__pkg__': import __pkg__.sibling # == 'import .sibling' and import __pkg__.__pkg__.uncle # == 'import ..uncle' import __pkg__.__pkg__.uncle.cousin # == 'import ..uncle.cousin' I think the leading '.' dot forms are a lot nicer, but the presence of a module named '__pkg__' would be useful for programmatically mucking navigating the package structure. Ken Manheimer klm@zope.com
On Tue, 23 Dec 2003, Ken Manheimer wrote:
I can't quite get used to the idea of '.' having two different meanings depending on how many '.'s are found next to each other in an import statement. I believe that the notation above _is_ concise, but _is not_ very intuitive ...
... while the notation here _is_ intuitive (to a python programmer), but _is not_ concise. I still favor this syntax to the former syntax. I like '__pkg__' better than my initial suggestion ('__parent__'). It's more aesthetically pleasing. :) -- Devin devin@whitebread.org http://www.whitebread.org/
Guido van Rossum <guido@python.org> writes:
The biggest "clarity" issue I have with the dot notation is that there's no obvious keyword to look up when you want to find out what's going on. The best you could do is look up the "import" section of the manual. Heaven help you if you want to use Google. (Has anyone tried to get any meaningful information from Google on Microsoft's two technologies, COM and .NET? You'd almost think they'd done it deliberately... Seriously, the "what do I search for?" factor bugs me every time I hit it. Paul. -- This signature intentionally left blank
There's no deep reason why the language reference couldn't contain non-alphanumeric characters, although this won't help for Google. But in this case, "from" would be the thing to look up, so I'm not sure I see it as such a serious problem. --Guido van Rossum (home page: http://www.python.org/~guido/)
"Troy Melhase" <troy@gci.net> wrote in message news:200312220003.08810.troy@gci.net...
I had same idea of reusing 'in', but with relative, absolute changed to perhaps RelMode, AbsMode so as to grammatically flow better. 'in NAME' could then indicate place in package to begin search. 'in Package' and 'in Path' are clearer (more explicit) alternatives to relative and absolute. Terry J. Reedy
Summarizing what I've seen so far: Status quo: [from NAME] import NAME [, NAME] [from NAME] import NAME as NAME There also seems to be consensus on adding: [from NAME] import (NAME [, NAME]) To this, we want to as a way to spell out explicitly how the module is found; for example: -a- by searching sys.path -b- in the current package -c- in the current package or its parent package, recursively -d- among the python standard library modules (and nowhere else) -e- in the current working directory -f- in a specific directory -g- combinations of the above Jack Diederich proposed:
Guido van Rossum oppined:
(---)
Not only a gratuitous change; it was always my impression that "from" was at the beginning of the statement for exactly the reason Guido is hesitant about adding "searching" at the end of it. With this in mind, my suggestion is to add the searching syntax should before the "import" keyword rather than at the end of the statement. Reusing an existing keyword, we get: [from NAMES] [in WHERE] import ... WHERE would then be an object, or a list of objects (-g-) specifying the import mechanism: -d- __stdlib__ or maybe stdlib -b- __package__ -c- __search__ -e- '.' -f- '/home/bellman/python' -a- sys.path Without any "from" clause, "in WHERE import ..." reads a little strangely, but I think it should be possible to get used to it, just as we've gotten used to the current "from NAME import ..." Would this work ? /Paul
Guido van Rossum wrote:
Not necessarily. import could be a special function that doesn't need parentheses around it's arguments or comma separators. That could be syntax sugar for: import (names, namedAs, fromModule, searchmethod) or import (names =[theNames], namedAd=[localNames], fromModule=fromModule, searchMethod=upFromCurrent) OTOH, to me it looks more like Smalltalk than like Python. An explicit import function without special sugar looks more Pythonic. (But that would break backward compatibility.) Perhaps the current method could be deprecated, and an import function be the replacement?
Charles Hixson <charleshixsn@earthlink.net> writes:
I really don't think you could make that work (unless you want to write math = import("math") everywhere, which you can almost do today...) Cheers, mwh -- Ya, ya, ya, except ... if I were built out of KSR chips, I'd be running at 25 or 50 MHz, and would be wrong about ALMOST EVERYTHING almost ALL THE TIME just due to being a computer! -- Tim Peters, 30 Apr 97
[Barry]
Indeed, this is common on c.l.py and the Tutor lists, with a twist: someone writes a module in their *current* directory, with a name like random.py, and then "Python breaks mysteriously all over the place", because every import of random ends up with something utterly unexpected. For some reason, "random.py" seems most often to be the actual name used in these cases. Possibly related: I remember pissing away a day a year or so ago trying to track down a segfault in a Zope test that happened only when I ran the test on my McLean box. In the end, it turned out that the test in question was (after untangling the usual layers of Zopish indirection <wink/grrrr>) deliberately trying to import a module that didn't exist, with an oddball name. It just so happened that somebody had whittled down a segfaulting Python program and uploaded it to SourceForge, with the very same oddball name. I had downloaded that program into my Python build directory, and Zope kept provoking its segfault as a side effect of (unintentionally!) importing it. In large part, I've come to loathe imports -- I never have the foggiest idea how they're going to get satisfied in code I didn't write, and especially in code that plays every trick in (and out) of the book to hide what it's really doing with sys.path mutations, import hooks, poking stuff by magic into sys.modules via well-hidden side effects in scattered __init__.py files, ... it makes Java's rigid, wordy, explicit import scheme look very attractive some days. That you can't trick that into doing arbitrarily unexpected things by magic at runtime isn't entirely a bad thing <wink>.
This can be interpreted in two ways though. You could be unfortunate enough to create a module in your package that has the same name as a standard module you also need to import from inside the same package -- that's currently impossible. But the most common variant I've seen is what Tim Peters describes: having a module in the current directory that accidentally hides a standard module. The latter case does not involve packages and will not be fixed by any import syntax -- it can only be fixed by changing the default sys.path *not* to include the current directory, just like over the last decade or so Unix shells have stopped putting '.' in $PATH. But that has other downsides: it would be a lot harder to begin writing a program broken up into modules. Hmm, maybe the current directory should be last? That way you can import your own modules except if they clash with the standard library. Hm, but there are so many standard library modules that it might still cause frequent complaints from mystified beginners. I'm beginning to agree (uselessly) with Tim: import itself is a problem. :-) --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido:
Maybe in Python 3.0 the current directory should appear as a package with some standard name. Then applications could use relative imports to import their own modules, and absolute ones to import standard modules, without fear of conflict. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+
On Wed, Dec 17, 2003 at 08:55:03PM -0800, Guido van Rossum wrote: [...]
Or it can be fixed by not having the entire standard library at the toplevel, but instead inside a single "stdlib" package. That way 3rd party packages could never conflict with the standard library (unless they were crazy enough to call one of their own modules "stdlib"...). This way, no matter how large the standard library grows, the potential for module name conflicts doesn't grow. It'd probably simplify things slightly, too -- the site-packages directory could go away entirely, and all packages, standard or not, would be on an equal footing. The standard library would be no more special than, say, Zope. My /usr/lib/python would just contain stdlib/ twisted/ zope/ Numeric/ etc. The backwards compatibility problems probably means this isn't feasible until Python 3.0, though :( -Andrew.
Andrew> Or it can be fixed by not having the entire standard library at Andrew> the toplevel, but instead inside a single "stdlib" package. ... Andrew> The backwards compatibility problems probably means this isn't Andrew> feasible until Python 3.0, though :( Not necessarily. It has been proposed (more than once I think) that such a stdlib package could be created in parallel using symlinks. At its simplest, I think all you'd need would be a stdlib directory, a nearly empty __init__.py file and symlinks to all the current standard modules and packages. (The __init__.py file would take care of importing builtin modules.) I would choose a very short name for such a package ("std" seems best) even though it might create clashes because it will be used a lot. Skip
Skip Montanaro <skip@pobox.com> writes:
Are you sure that you won't get multiple copies of the same module floating around? E.g. will you have from httplib import HTTPConnection as HC1 from std.httplib import HTTPConnection as HC2 HC1 is HC2 I think in the scheme sketched above this will be false, which kills the idea stone dead. I might be wrong, though. Cheers, mwh -- Counting lines is probably a good idea if you want to print it out and are short on paper, but I fail to see the purpose otherwise. -- Erik Naggum, comp.lang.lisp
>> It has been proposed (more than once I think) that such a stdlib >> package could be created in parallel using symlinks.... Michael> Are you sure that you won't get multiple copies of the same Michael> module floating around? E.g. will you have Michael> from httplib import HTTPConnection as HC1 Michael> from std.httplib import HTTPConnection as HC2 Michael> HC1 is HC2 Michael> I think in the scheme sketched above this will be false, which Michael> kills the idea stone dead. Maybe the symlink idea won't work. It seems that it would work to import the standard modules from std/__init__.py though. I performed a simple experiment. I created the std directory, added a symlink in it for the random module and added an __init__.py file with this content: import sys import urlparse Here's the result: >>> import random >>> from std import random as random2 >>> random is random2 False >>> import urlparse >>> from std import urlparse as urlparse2 >>> urlparse is urlparse2 True >>> import sys >>> from std import sys as sys2 >>> sys is sys2 True If we added a significant number of modules to std/__init__.py, startup would slow to a crawl. I imagine some sort of delayed import mechanism could be crafted to avoid this problem though. Skip
Paolo> Symlink is one of the things that I would like to have on my XP Paolo> box... As Michael Hudson suggests, symlinks probably won't work anyway. I have something which looks like a symlink on my Win2k box. Windows calls it a Shortcut in listings. Is that not roughly the same thing as a symlink? Skip
Skip Montanaro wrote:
No, the NT file system as nothing as flexible as a symlink. If we are speaking about symlinks to files, windows shortcuts are barely usable only for pointing to executable for execution in a console (but they have a lot of redirection problems). If you *open* a shortcut you are actually opening the shortcut itself, not the thing it's pointing to ;-( Shortcuts to a directory is only usable by Explorer-like-programs, I don't know any usable operation with them from the console... The only way to use shortcut in python is to turn python shortcut-aware in the import machinery... --- Paolo Invernizzi Ps... cygwin is another story....
Skip Montanaro <skip@pobox.com>:
Only if it's transparent to the system calls which operate on files (e.g. open()ing it opens the file that it refers to, etc.) I don't believe that's the case with Windows shortcuts. Macintosh "aliases" have the same problem. (And now with MacOS X, we have both aliases *and* symlinks in the same system, just to add to the confusion...) Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+
Paolo Invernizzi <paoloinvernizzi@dmsware.com>:
Also, wouldn't that give two ways of getting each of the modules, with all the attendand Bad Things? Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+
On Wed, 2003-12-17 at 23:55, Guido van Rossum wrote:
Right, and I'd like to make that possible. The least disruptive way I see of doing that is to introduce a magic root package from which things can be imported from. E.g. import __root__.re
Yep, I've been bit by that too.
Yep. I haven't had '.' in my $PATH for a bazillion years. :)
There ya go! That's thinking outside the box: let's deprecate import altogether :) -Barry
[Guido]
Not useless at all -- that's bonding! Technically useless but humanly essential <wink>. It's no mystery *why* import is "a problem", but I'm not sure insight implies a cure: it's one of the very few places in Python where you can't explicitly say what you intend. Even in an inheritance hierarchy from hell, if I really don't want to chance that please.hit_me() will get satisfied from a class other than BlackJack, I can force it easily -- BlackJack.hit_me(please) There aren't any names to spell "the roots" of all the places Python may search to satisfy an import, so no way either to be explicit about intent. In contrast, nobody can mistake what import java.util.Random; intends. It's not a deep thing, it's just being able to name the intended context. I confess I'm keener on that than on growing more ways to spell more kinds of searches.
Barry Warsaw <barry@python.org> writes:
That issue I can understand. And I agree there should be a way to state it explicitly. One of the issues here is that this area is a bit under-documented (I know, I should read the source, but I don't have the time right now). At the moment, there are two cases: 1. Import from sys.path. This is what is being called an "absolute" import, and is nice and easy to understand. The key issue is that there is no way to *force* this interpretation in the face of option (2) below. 2. Import from "the package". This is the under-documented bit, but if I understand it correctly, it's basically that from within a module contained in a package, sys.path is conceptually *extended* to include the package's __path__ (which by default contains the directory of the package, but which can be user-modified). Now the big problem here is that behaviour (2) is useful. Simple "relative" imports of one module within a package from another module in the same package are common. Guido's (IMHO ugly) "dot" syntax handles that, by making users explicitly request option (2), and making the current import syntax *only* mean (1). But none of the proposed solutions handle the __path__ variable. I don't have any objection in principle to desupporting __path__ (heck, it would have made thinking about PEP 302 easier, if nothing else) but (a) it would need a deprecation warning, and (b) Guido himself offered a use case in <http://www.python.org/doc/essays/packages.html>. This definitely needs a PEP. If we're removing support for __path__, the implications need to be thought through (PEP 302, the pkgutil module, etc etc). If we're not, none of the proposals so far have covered how __path__ gets supported in future. A much simpler proposal, just providing an explicit way of saying "Import from sys.path *only*" may be OK without a PEP. But even then, I'd suspect we should have a PEP explaining *why* it has to be this simple. Go on, Guido. We'll be gentle if you write a PEP, and we won't set c.l.p on you :-) Paul. -- This signature intentionally left blank
[Barry]
[Paul Moore]
Right.
That's not a very useful way to think about it; when the module is found on __path__ something very *different* happens than when it is found on sys.path. The difference is noticeable if you ask the imported module for its __name__. If you the import was satisfied from __path__, then __name__ will include the package name. If it was satisfied from sys.path, it won't. And __name__ is related to module identity: all modules are stored in sys.modules using their __name__ as the key, so multiple imports of a module with the same __name__ end up referring to the same module (and only the first import causes the module's code to be executed).
Actually, what you consider useful is considered harmless by others. Many large packages, including Zope, have adopted a rule of always using absolute imports, to clarify what kind of import is being used.
__path__ has no bearing on the proposals, it is used for relative imports. I think your confusing about how the current import works caused you to think it is relevant.
We're not, but they have (by not mentioning it); but I agree that a PEP is needed.
This was where this thread started: someone proposed a way of spelling "import from sys.path only". My big problem with that is that *most* imports are intended to be satisfied by sys.path only, so this spelling should be the default, meaning the current spelling.
Go on, Guido. We'll be gentle if you write a PEP, and we won't set c.l.p on you :-)
Alas, I have no time. --Guido van Rossum (home page: http://www.python.org/~guido/)
Paul Moore <pf_moore@yahoo.co.uk>:
But none of the proposed solutions handle the __path__ variable. I don't have any objection in principle to desupporting __path__
I'll be worried if __path__ is likely to disappear as a result of all this. I'm using it in what will be the next version of my Python GUI library, to switch in different implementations of submodules depending on the platform. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+
On Tue, 2003-12-16 at 09:17, Gustavo Niemeyer wrote:
I guess the difference is when you're writing an application vs. writing a library. When writing an application (like Zope or Mailman), we already manipulate sys.path to include some non-standard locations. Incorporating a third-party package into the application then is a simple matter of arranging to have its top level directory somewhere on sys.path. In Mailman, I do this with the email package, and various codecs. When writing a library, that's harder because you're imposing requirements on the downstream consumer of your library. As an application developer, I don't have much sympathy for relative imports. As a library developer, I can see the point. -Barry
On Mon, 2003-12-15 at 14:34, Ken Manheimer wrote:
I've heard this before, but I still don't get it, so I'd like to tease this one out more. What does "plop the things in place" mean? Zope 2 has a few standard directories it adds to its PYTHONPATH. If I distribute a 3rd party package Zope wants to use, and that package uses absolute imports, you can just drop the package in one of those directories and it'll just work. Maybe you mean that you want to drop the mythical 3rd party package in somewhere other than a directory on Zope's PYTHONPATH and have it just work. I'd say that's too perverse of a use-case for Python to worry about. One problem with banning relative imports is that Python doesn't allow a top-level package name to live in more than one place on PYTHONPATH. It would be useful for Python to support such things out of the box, say if we wanted to adopt a Java style naming convention for package organizations. Jim uses the term "pure container package" to describe top-level packages that contain no real content in the package except subpackages. "zope" and "zodb" are examples of this in Zope 3. IWBNI Python had better/easier support for weaving together pure container package contents where subdirectories live in different locations on the file system. -Barry
Barry Warsaw writes:
+1 The .pth files processed by site.py support two interesting kinds of lines: - directories that should be added to sys.path, and - import statements The later alone is interesting enough, since it allows arbitrary initialization code to run when the import is executed. This still happens before all the .pth files have been processed; I suspect we really want the container package to be a little more flexible than that, at least allowing their __path__ to be computed after all .pth files have been processed. Perhaps it would be nice to have a hook (probably in sys) that gets called after site/sitecustomize are done? A "pure container package" could use a .pth file using "import mypackage", and the package's __init__.py would register a hook function in sys that searches sys.path for whatever directories should be used in mypackage.__path__. This also avoids the code being too specific to one way of thinking about packages. -Fred -- Fred L. Drake, Jr. <fred at zope.com> PythonLabs at Zope Corporation
Ken Manheimer writes:
I'd agree with Ken here. I think the importance of being able to do relative imports grows as well with the ability to compile Python into foreign frameworks, like Jython or .NET. I'd invent an ugly syntax for relative imports, and let the default be absolute. Bill
Why? I'd really like to see some concrete use cases for relative imports, other than just saving some typing.
Suppose you've developed a Python top-level application, composed of a package 'foo' containing a number of submodules and subpackages. You'd now like to add this to a Jython application as a library, part of a larger application. It's nice to be able to just add a higher-level package structure, com.mycompany.myapp, to the Python package, to fit in with the style of your application. Yet you don't want to edit the Python code to 'import com.mycompany.myapp.foo.submodule' everywhere; you'd like 'import submodule' or 'import foo.submodule' to continue working. Often the code has to work in both the stand-alone and the federated case. This is just a special case of the federation problem that we've seen for years in distributed computing; previously top-level names get federated when new higher-level constructs are formed. Ken mentioned enterprise programming; it's another case of this. Two companies merge and start merging their code. It's much better to be able to write (in the new higher-level module bletch.py) import foo import bar and have the subpackages under foo and bar continue to work without rewrites, than have to go through all the code under foo and bar and change it to know about the new higher-level bletch namespace. Bill
On Mon, 15 Dec 2003, Guido van Rossum wrote:
As i've said before (though it's been a while) i think people want this, it would be useful, and i think it would work well. Barry wrote, in response to guido's message: < Actually, if truth be told, I'd love to just ban local imports. I < understand that might be controversial though <wink>. I don't understand the desire to totally prevent relative imports. It would be quite useful in avoiding growing module-naming problems, where you have to be careful about shadowing a global module with one in your package. I *can* understand objecting to the ambiguity in the current situation, where you don't know whether a module import will resolve to a sibling module or one in higher up in the hierarchy. That is why i strongly prefer having a leading dot to explicitly signify a relative import, and originally proposed it that way. I expect that the elimination of the ambiguity would mitigate the objection to relative imports - and we would do away with the shadowing problems, and even have easy relocation of modules, if there are cases where software does need to move around. Ken klm@zope.com
I don't understand the desire to totally prevent relative imports.
It's because of TOOWTDI.
I don't see the problem, or I misuderstand what "it" refers to; it seems you have this backwards if it refers to relative imports. Say my package P defines a submodule sys. If we require absolute imports, there is no ambiguity: the submodule sys must be imported as P.sys while the standard sys module can be imported as simply sys. --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote:
If absolute imports were to be the only type allowed, then it would seem that the only possible location for naming conflicts is in the _first_ element. So if I wanted to use two different third party modules, both of which have unfortunately chosen the same name for the top-level package, the only way to let them co-exist is to redo _all_ of the imports in one or the other of them. Whereas, if relative pathing is possible, I believe that all I have to do is push them one level down in the package heirarchy (using distinct names that I invent), and neither of them ever even knows about the other's existence. I can get at both of them unambiguously, by using my new top=level names, and neither package even knows that it is no longer starting at the top of the import heirarchy. Or is there some other solution being proposed to this problem, and I just haven't understood it? Cheers, Nick. -- Nick Coghlan | Brisbane, Australia Email: ncoghlan@email.com | Mobile: +61 409 573 268
On Tue, 2003-12-16 at 05:32, Nick Coghlan wrote:
True.
Has this ever happened to you in practice? It seems like the way out would be to start adopting a Java-like convention for package names. The problem with that in current Python is that you can't (easily) weave a package's contents from different locations in the file system. -Barry
Barry Warsaw wrote:
Well, no. But others seemed concerned about it, so I thought I'd try to clarify the problem - the scripts I write don't need anything more than the standard library and win32all.
I still like the various suggestions for _giving_ those parts of the system special names. That's what I was trying for, even if the names I chame up with weren't any good. Cheers, Nick. -- Nick Coghlan | Brisbane, Australia Email: ncoghlan@email.com | Mobile: +61 409 573 268
On Sun, Dec 14, 2003 at 04:31:09PM -0500, Raymond Hettinger wrote:
http://mail.python.org/pipermail/python-dev/2002-October/029770.html [and the rest of that thread] A 3% slowdown in the common case was considered too big to allow a feature like this to be included. The slowdown might be smaller if the builtin module wasn't allowed to be anything but a dict instance, or might disappear if there were two versions of ceval.c were included, and one used PyObject_ APIs and the other used PyDict_ APIs (as long as only one was fighting for space in the cpu cache, main memory, etc) Jeff
At 02:26 PM 12/15/03 -0500, Raymond Hettinger wrote:
There is a workaround for this in CPython, which used to be used by Zope, and which I have used on occasion: scan the code object for LOAD_NAME opcodes, and note what names are used by the code block. Then, look up those names in your non-dictionary dictionary, and copy them into a real dictionary. So, it's got to be something pretty specialized to really need PyObject_* APIs instead of PyDict_* APIs in CPython at present. Why penalize *everything* for such an specialized need?
participants (35)
-
Aahz -
Andrew Bennetts -
Barry Warsaw -
Bill Janssen -
Boris Boutillier -
Charles Hixson -
David Ascher -
David Eppstein -
David LeBlanc -
Devin -
Fred L. Drake, Jr. -
Fred L. Drake, Jr. -
Fredrik Lundh -
Greg Ewing -
Guido van Rossum -
Gustavo Niemeyer -
Jack Diederich -
Jack Jansen -
Jeff Epler -
Josiah Carlson -
Ken Manheimer -
Kevin Jacobs -
Michael Hudson -
Nick Coghlan -
Paolo Invernizzi -
Paul Moore -
Paul Svensson -
Phillip J. Eby -
Raymond Hettinger -
Raymond Hettinger -
Skip Montanaro -
Terry Reedy -
Tim Peters -
Troy Melhase -
Werner Schiendl