Why does PEP 8 advise against explicit relative imports?
PEP 8 advises developers to use absolute imports rather than explicit relative imports. Why? Using absolute imports couple the internal implementation of a package to its public name - you can't just change the top level directory name any more, you have to go through and change all the absolute imports as well. You also can't easily vendor a package that uses absolute imports inside another project either, since all the absolute imports will break. What's the *concrete* benefit of using absolute imports that overcomes these significant shortcomings in maintainability and composability? Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Tue, Jul 16, 2013 at 1:40 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
The problem with relative imports (both explicit and implicit) is that it makes it less obvious when you are importing a module under the wrong name. If a package ends up in sys.path directly (for example, by executing something that lives inside it directly) then an explicit relative import directly in the package will fail, but an explicit relative import in a sub-package won't, and you can end up with the subtly confusing mess of multiple imports of the same .py file under different names.
What's the *concrete* benefit of using absolute imports that overcomes these significant shortcomings in maintainability and composability?
In my experience changing the name of a package is problematic for more reasons than just 'import'. We used to do the put-in-a-package thing for third-party packages at Google, and we quit doing it because it was just too painful: it's hard to find all the places that reference the package by name (think of __import__, or mucking in sys.modules), and it's extra problematic with packages that contain extension modules. You can't rename a package, even for 'vendoring a package', without carefully checking whether it'll work -- unless you don't care if it'll work, of course :) The alternative is to include a package *without* changing its name (by adding the right directory, without __init__.py, to sys.path) and that has worked out a lot better for us. Even so, there's no reason for *the standard library* to use explicit relative imports, and that's what PEP 8 is supposed to cover, right? :) -- Thomas Wouters <thomas@python.org> Hi! I'm an email virus! Think twice before sending your email to help me spread!
On 16 July 2013 22:02, Thomas Wouters <thomas@python.org> wrote:
These are good reasons, but they're not the reasons the PEP currently gives. "Better failure modes" is a good practical benefit :)
Even so, there's no reason for *the standard library* to use explicit relative imports, and that's what PEP 8 is supposed to cover, right? :)
I realised in re-reading the whole thing that we actually admitted defeat on that front back during the last function annotations discussion :) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 16 July 2013 22:27, Nick Coghlan <ncoghlan@gmail.com> wrote:
These are good reasons, but they're not the reasons the PEP currently gives. "Better failure modes" is a good practical benefit :)
I have now included an updated rationale for this rule in the patch attached to http://bugs.python.org/issue18472 (the recommendation to use absolute imports remains unchanged). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Jul 16, 2013, at 02:02 PM, Thomas Wouters wrote:
Vendoring also gives no end of headaches to distro packagers because we have to un-vendor that stuff, which can lead to a mess of patches on top of upstream.
Even so, there's no reason for *the standard library* to use explicit relative imports, and that's what PEP 8 is supposed to cover, right? :)
Right! It doesn't bother me at all that this particular recommendation may be followed by third parties too. :) -Barry
On 16/07/13 22:02, Thomas Wouters wrote:
I don't understand this objection. What do you mean by "importing a module under the wrong name"? As I see it, if relative imports are a bad idea for the std lib, they're a bad idea for everyone. (What am I missing?) So why do we have relative imports? Was it a mistake? I don't think so. It seems to me that you're giving some fairly obscure edge cases, if I've understood you correctly. If adding a package directory to sys.path breaks relative imports, isn't that an argument against adding the package directory to sys.path rather than relative imports?
Again, these are some fairly odd edge cases. I don't mean to imply that they never occur in practice, obviously they do, but perhaps you're overly sensitive to them. It seems strange to me that PEP 8 should recommend against using relative imports for the 49 packages that would benefit from it just to avoid problems with the 1 that does weird things with __import__ or manually mucks about with sys.modules. Have I missed something?
Even so, there's no reason for *the standard library* to use explicit relative imports, and that's what PEP 8 is supposed to cover, right? :)
If a third-party package uses relative imports, are you suggesting that those rel imports should be turned into absolute imports as a precondition of it being moved into the std lib? -- Steven
I have to say that a long time ago, when you couldn't tell a relative import from a top-level import, the ban on relative imports made more sense. -- --Guido van Rossum (python.org/~guido)
On 17 July 2013 11:55, Steven D'Aprano <steve@pearwood.info> wrote:
What Thomas is saying is that, given the current state of the import system, using absolute imports usually give you better behaviour (or at least better error messages) than explicit relative imports do when things go wrong. At present, there are still too many ways to get your import configuration into a bad state (most notably running a module inside a package by filepath rather than the -m switch), and if that happens, using absolute imports is likely to make it a bit easier to find your way out again. I actually agree with that rationale - while I prefer explicit relative imports myself, I'm also more experienced than most when it comes to identifying and resolving import configuration problems (see http://python-notes.curiousefficiency.org/en/latest/python_concepts/import_t...) So, for now, avoiding even explicit relative imports is still a good default position. People who feel qualified to reject PEP 8's advice on this topic should also know enough to deal with the potential consequences. I'll be proposing a rewording of this admonition to give a better rationale, but I won't be proposing that the recommendation be changed :) Once we get sys.path initialisation to a better place (amongst other things), then it will be time to revisit the recommendation itself. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
If sounds like the main problem with relative imports (even explicit) is caused by ignoring the package structure and running modules inside a package as a script, without using -m. Maybe we should update the PEP to make this abundantly clear? I note that an additional problem with informational PEPs like PEP 8 is that they aren't tied to a particular Python version, so even if we solved the above issue perfectly in Python 3.4 there'd still be tons of people applying PEP 8 to older code who would benefit from the warning against relative imports. So maybe we should also add that to the PEP. -- --Guido van Rossum (python.org/~guido)
On 16 Jul, 2013, at 14:02, Thomas Wouters <thomas@python.org> wrote:
That's only a problem for subpackages (that is, for example when distutils.commands ends up being importable as commands), because explicit relative imports (``from .other import attr``) don't work in toplevel modules or scripts because the leading dot means "the current package". I tend to use explicit relative imports in my code when that increases readability by reducing unnecessary duplication. That is, readability tends to suffer when you have a number of lines like with "from package.module1 import name". Ronald
On 7/16/2013 7:40 AM, Nick Coghlan wrote:
From my viewpoint, absolute imports always work, on all python versions, whereas people have reported various problems with relative imports on python-list. At least some of the complaints were valid, at least at the time, but I have not paid too much attention. Even if idlelib files used relative imports, 'idlelib' cannot change and I see no reason to ever change 'idle_test'. I would like to change camelCaseFileNames.py, but if ever done, that would affect both types of imports. -- Terry Jan Reedy
On Tue, Jul 16, 2013 at 1:40 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
The problem with relative imports (both explicit and implicit) is that it makes it less obvious when you are importing a module under the wrong name. If a package ends up in sys.path directly (for example, by executing something that lives inside it directly) then an explicit relative import directly in the package will fail, but an explicit relative import in a sub-package won't, and you can end up with the subtly confusing mess of multiple imports of the same .py file under different names.
What's the *concrete* benefit of using absolute imports that overcomes these significant shortcomings in maintainability and composability?
In my experience changing the name of a package is problematic for more reasons than just 'import'. We used to do the put-in-a-package thing for third-party packages at Google, and we quit doing it because it was just too painful: it's hard to find all the places that reference the package by name (think of __import__, or mucking in sys.modules), and it's extra problematic with packages that contain extension modules. You can't rename a package, even for 'vendoring a package', without carefully checking whether it'll work -- unless you don't care if it'll work, of course :) The alternative is to include a package *without* changing its name (by adding the right directory, without __init__.py, to sys.path) and that has worked out a lot better for us. Even so, there's no reason for *the standard library* to use explicit relative imports, and that's what PEP 8 is supposed to cover, right? :) -- Thomas Wouters <thomas@python.org> Hi! I'm an email virus! Think twice before sending your email to help me spread!
On 16 July 2013 22:02, Thomas Wouters <thomas@python.org> wrote:
These are good reasons, but they're not the reasons the PEP currently gives. "Better failure modes" is a good practical benefit :)
Even so, there's no reason for *the standard library* to use explicit relative imports, and that's what PEP 8 is supposed to cover, right? :)
I realised in re-reading the whole thing that we actually admitted defeat on that front back during the last function annotations discussion :) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 16 July 2013 22:27, Nick Coghlan <ncoghlan@gmail.com> wrote:
These are good reasons, but they're not the reasons the PEP currently gives. "Better failure modes" is a good practical benefit :)
I have now included an updated rationale for this rule in the patch attached to http://bugs.python.org/issue18472 (the recommendation to use absolute imports remains unchanged). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Jul 16, 2013, at 02:02 PM, Thomas Wouters wrote:
Vendoring also gives no end of headaches to distro packagers because we have to un-vendor that stuff, which can lead to a mess of patches on top of upstream.
Even so, there's no reason for *the standard library* to use explicit relative imports, and that's what PEP 8 is supposed to cover, right? :)
Right! It doesn't bother me at all that this particular recommendation may be followed by third parties too. :) -Barry
On 16/07/13 22:02, Thomas Wouters wrote:
I don't understand this objection. What do you mean by "importing a module under the wrong name"? As I see it, if relative imports are a bad idea for the std lib, they're a bad idea for everyone. (What am I missing?) So why do we have relative imports? Was it a mistake? I don't think so. It seems to me that you're giving some fairly obscure edge cases, if I've understood you correctly. If adding a package directory to sys.path breaks relative imports, isn't that an argument against adding the package directory to sys.path rather than relative imports?
Again, these are some fairly odd edge cases. I don't mean to imply that they never occur in practice, obviously they do, but perhaps you're overly sensitive to them. It seems strange to me that PEP 8 should recommend against using relative imports for the 49 packages that would benefit from it just to avoid problems with the 1 that does weird things with __import__ or manually mucks about with sys.modules. Have I missed something?
Even so, there's no reason for *the standard library* to use explicit relative imports, and that's what PEP 8 is supposed to cover, right? :)
If a third-party package uses relative imports, are you suggesting that those rel imports should be turned into absolute imports as a precondition of it being moved into the std lib? -- Steven
I have to say that a long time ago, when you couldn't tell a relative import from a top-level import, the ban on relative imports made more sense. -- --Guido van Rossum (python.org/~guido)
On 17 July 2013 11:55, Steven D'Aprano <steve@pearwood.info> wrote:
What Thomas is saying is that, given the current state of the import system, using absolute imports usually give you better behaviour (or at least better error messages) than explicit relative imports do when things go wrong. At present, there are still too many ways to get your import configuration into a bad state (most notably running a module inside a package by filepath rather than the -m switch), and if that happens, using absolute imports is likely to make it a bit easier to find your way out again. I actually agree with that rationale - while I prefer explicit relative imports myself, I'm also more experienced than most when it comes to identifying and resolving import configuration problems (see http://python-notes.curiousefficiency.org/en/latest/python_concepts/import_t...) So, for now, avoiding even explicit relative imports is still a good default position. People who feel qualified to reject PEP 8's advice on this topic should also know enough to deal with the potential consequences. I'll be proposing a rewording of this admonition to give a better rationale, but I won't be proposing that the recommendation be changed :) Once we get sys.path initialisation to a better place (amongst other things), then it will be time to revisit the recommendation itself. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
If sounds like the main problem with relative imports (even explicit) is caused by ignoring the package structure and running modules inside a package as a script, without using -m. Maybe we should update the PEP to make this abundantly clear? I note that an additional problem with informational PEPs like PEP 8 is that they aren't tied to a particular Python version, so even if we solved the above issue perfectly in Python 3.4 there'd still be tons of people applying PEP 8 to older code who would benefit from the warning against relative imports. So maybe we should also add that to the PEP. -- --Guido van Rossum (python.org/~guido)
On 16 Jul, 2013, at 14:02, Thomas Wouters <thomas@python.org> wrote:
That's only a problem for subpackages (that is, for example when distutils.commands ends up being importable as commands), because explicit relative imports (``from .other import attr``) don't work in toplevel modules or scripts because the leading dot means "the current package". I tend to use explicit relative imports in my code when that increases readability by reducing unnecessary duplication. That is, readability tends to suffer when you have a number of lines like with "from package.module1 import name". Ronald
On 7/16/2013 7:40 AM, Nick Coghlan wrote:
From my viewpoint, absolute imports always work, on all python versions, whereas people have reported various problems with relative imports on python-list. At least some of the complaints were valid, at least at the time, but I have not paid too much attention. Even if idlelib files used relative imports, 'idlelib' cannot change and I see no reason to ever change 'idle_test'. I would like to change camelCaseFileNames.py, but if ever done, that would affect both types of imports. -- Terry Jan Reedy
participants (7)
-
Barry Warsaw -
Guido van Rossum -
Nick Coghlan -
Ronald Oussoren -
Steven D'Aprano -
Terry Reedy -
Thomas Wouters