Hi all, Im tired of not being able to make relative imports freely. Now trying to develop a module which enable any project to use relative imports once it is loaded. Anybody interested in? Best,
On Fri, Nov 09, 2018 at 02:54:50PM -0800, danish bluecheese wrote:
Im tired of not being able to make relative imports freely.
Python has supported relative imports for a while now. https://docs.python.org/3/tutorial/modules.html#intra-package-references What do you mean? -- Steve
It supports, but whenever you get multiple folders there is no clean solution. Either there are some sys.path hacks or running things as modules in some cases. These are not pleasant at all. I think we can come up with something better. Interested in? On Fri, Nov 9, 2018 at 3:17 PM Steven D'Aprano <steve@pearwood.info> wrote:
└── src ├── __init__.py ├── main.py └── test ├── __init__.py └── test_main.py assume the structure above. To be able to use relative imports with such fundamental structure either i can go for sys.path hacks or could run as a module from one further level parent. I do not like this :D want to be able to use relational imports freely as soon as I provide correct relational path. Please let me know, if it is not clear on any aspect. Thank you. Regards. On Fri, Nov 9, 2018 at 3:39 PM Steven D'Aprano <steve@pearwood.info> wrote:
On Fri, Nov 09, 2018 at 03:51:46PM -0800, danish bluecheese wrote:
I don't understand. From the top level of the package, running inside either __init__ or main, you should be able to say: from . import test from .test import test_main
From the test subpackage, you should be able to say:
from .. import main to get the src/main module, or from . import test_main to get the test/test_main module from the test/__init__ module. (Disclaimer: I have not actually run the above code to check that it works, beyond testing that its not a SyntaxError.) What *precisely* is the problem you are trying to solve, and your proposed solution? -- Steve
you are right on the lines you mentioned. Those are all working if i run it as a module which i do every time. This is somewhat unpleasant to me, especially while developing something and trying to test it quickly. I just want to be able to use same relative imports and run single file with `python3 test_main.py` for example. Running files as modules every time is tiring. This is my problem. I could not come up with a concrete solution idea yet i am thinking on it. Open to suggestions. Thank you all for your help! On Fri, Nov 9, 2018 at 4:16 PM Steven D'Aprano <steve@pearwood.info> wrote:
This is somewhat unpleasant to me, especially while developing something and trying to test it quickly. I just want to be able to use same relative imports and run single file with `python3 test_main.py` for example. I had the same frustration when I first tried to use relative imports. Then I discovered setuptools’ develop mode (now pip editable install) It is the right way to run code in packages under development. -CHB Running files as modules every time is tiring. This is my problem. I could not come up with a concrete solution idea yet i am thinking on it. Open to suggestions. Thank you all for your help! On Fri, Nov 9, 2018 at 4:16 PM Steven D'Aprano <steve@pearwood.info> wrote:
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On Tue, Nov 13, 2018 at 4:46 PM Chris Barker - NOAA Federal via Python-ideas <python-ideas@python.org> wrote:
Then I discovered setuptools’ develop mode (now pip editable install)
It is the right way to run code in packages under development.
In multiple workplaces I found a folder with python utility scripts that users can just double-click. The need for installing causes problems with handling different versions on one machine, and the need for "__init__.py" files makes the folders less pretty. Sure - sometimes I need to install stuff anyway - but that's just one "install.py" double click away. I would like to propose allowing importing of strings that would support relative paths. For example in Danish's example: # use this in `test_main.py` import '../main.py' as main Maybe the syntax can be improved, but to me this need has been aching since I started using Python 12 years ago. I've used C, C++, and Javascript where the whole "how do I connect these two files that are a folder apart" problem doesn't require googling for documentation on packaging tools, magic filenames, constraints and gotchas. The solution is always obvious because it works just like it works in every system - with a file-relative path. File-relative imports is probably highest on my Python wish list. I've drafted but not sent out a python-ideas email about it multiple times. I've seen a lot of "sys.path" hacking that would've been solved by file-relative-paths. Cheers and thanks, Yuval Greenfield
On Tue, Nov 13, 2018 at 09:34:33PM -0800, Yuval Greenfield wrote:
How does that differ from existing syntax? from .. import main Off the top of my head, a few more questions that don't have obvious answers (at least not to me): What happens if main.py doesn't exist, but main.pyc does? What if you want to import from a sub-package, rather than a single-file module? What happens when Windows users use a backslash instead of a forward-slash? Does this syntax support arbitrary relative paths anywhere on the file system, or is it limited to only searching the current package? How does it interact with namespace packages? What happens if you call os.chdir() before calling this? Invariably people will want to write things like: path = '../spam.py' import path as spam (I know that's something I'd try.) What will happen there? If that is supported, invariably people will want to use pathlib.Path objects. Should that work?
Beware of "obvious" solutions, because so often they lead to not so obvious problems. Like Javascript's "relative import hell": Quote: // what we want import reducer from 'reducer'; // what we don't want import reducer from '../../../reducer'; https://medium.com/@sherryhsu/how-to-change-relative-paths-to-absolute-paths... And more here: https://goenning.net/2017/07/21/how-to-avoid-relative-path-hell-javascript-t... https://lostechies.com/derickbailey/2014/02/20/how-i-work-around-the-require... It seems to me that in languages which support this file-relative import feature, people spend a lot of time either trying to avoid using it, or building tools to allow them to avoid using it. I don't know if that makes it better or worse than Python's solution for relative imports :-) -- Steve
On Wed, Nov 14, 2018 at 5:15 PM Steven D'Aprano <steve@pearwood.info> wrote:
Agreed. Having spent a lot of time with JavaScript students, I actually am NOT a fan of directory-relative imports. They inevitably result in equivalent code looking different, and subtly different code looking identical. Consider: // main.js import User from './models/users'; // routers/users.js import User from '../models/users'; That example is probably okay, because if you ever get it wrong, you get an immediate error. But what about this? // routers/index.js import usersRouter from './users'; // models/stuff.js import User from './users'; The exact same import does completely different things based on which file it's in. That's dangerous, because it makes code subtly context sensitive. I would much rather work with package-relative pathing, where there is a known basis for *all* local imports, no matter what file the import is actually happening in. In Python, that's best done by naming the package again, so that's not quite ideal either, but it's better than having to pile in the exact right number of "../" to make the import work. ChrisA
On Fri, Nov 9, 2018 at 4:32 PM, danish bluecheese <danish.bluecheese@gmail.com> wrote:
Have you tried 'python3 -m test_main'? IIRC it should be effectively the same as 'python3 test_main.py' but with working relative imports. -n -- Nathaniel J. Smith -- https://vorpus.org
On Fri, Nov 09, 2018 at 02:54:50PM -0800, danish bluecheese wrote:
Im tired of not being able to make relative imports freely.
Python has supported relative imports for a while now. https://docs.python.org/3/tutorial/modules.html#intra-package-references What do you mean? -- Steve
It supports, but whenever you get multiple folders there is no clean solution. Either there are some sys.path hacks or running things as modules in some cases. These are not pleasant at all. I think we can come up with something better. Interested in? On Fri, Nov 9, 2018 at 3:17 PM Steven D'Aprano <steve@pearwood.info> wrote:
└── src ├── __init__.py ├── main.py └── test ├── __init__.py └── test_main.py assume the structure above. To be able to use relative imports with such fundamental structure either i can go for sys.path hacks or could run as a module from one further level parent. I do not like this :D want to be able to use relational imports freely as soon as I provide correct relational path. Please let me know, if it is not clear on any aspect. Thank you. Regards. On Fri, Nov 9, 2018 at 3:39 PM Steven D'Aprano <steve@pearwood.info> wrote:
On Fri, Nov 09, 2018 at 03:51:46PM -0800, danish bluecheese wrote:
I don't understand. From the top level of the package, running inside either __init__ or main, you should be able to say: from . import test from .test import test_main
From the test subpackage, you should be able to say:
from .. import main to get the src/main module, or from . import test_main to get the test/test_main module from the test/__init__ module. (Disclaimer: I have not actually run the above code to check that it works, beyond testing that its not a SyntaxError.) What *precisely* is the problem you are trying to solve, and your proposed solution? -- Steve
you are right on the lines you mentioned. Those are all working if i run it as a module which i do every time. This is somewhat unpleasant to me, especially while developing something and trying to test it quickly. I just want to be able to use same relative imports and run single file with `python3 test_main.py` for example. Running files as modules every time is tiring. This is my problem. I could not come up with a concrete solution idea yet i am thinking on it. Open to suggestions. Thank you all for your help! On Fri, Nov 9, 2018 at 4:16 PM Steven D'Aprano <steve@pearwood.info> wrote:
This is somewhat unpleasant to me, especially while developing something and trying to test it quickly. I just want to be able to use same relative imports and run single file with `python3 test_main.py` for example. I had the same frustration when I first tried to use relative imports. Then I discovered setuptools’ develop mode (now pip editable install) It is the right way to run code in packages under development. -CHB Running files as modules every time is tiring. This is my problem. I could not come up with a concrete solution idea yet i am thinking on it. Open to suggestions. Thank you all for your help! On Fri, Nov 9, 2018 at 4:16 PM Steven D'Aprano <steve@pearwood.info> wrote:
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On Tue, Nov 13, 2018 at 4:46 PM Chris Barker - NOAA Federal via Python-ideas <python-ideas@python.org> wrote:
Then I discovered setuptools’ develop mode (now pip editable install)
It is the right way to run code in packages under development.
In multiple workplaces I found a folder with python utility scripts that users can just double-click. The need for installing causes problems with handling different versions on one machine, and the need for "__init__.py" files makes the folders less pretty. Sure - sometimes I need to install stuff anyway - but that's just one "install.py" double click away. I would like to propose allowing importing of strings that would support relative paths. For example in Danish's example: # use this in `test_main.py` import '../main.py' as main Maybe the syntax can be improved, but to me this need has been aching since I started using Python 12 years ago. I've used C, C++, and Javascript where the whole "how do I connect these two files that are a folder apart" problem doesn't require googling for documentation on packaging tools, magic filenames, constraints and gotchas. The solution is always obvious because it works just like it works in every system - with a file-relative path. File-relative imports is probably highest on my Python wish list. I've drafted but not sent out a python-ideas email about it multiple times. I've seen a lot of "sys.path" hacking that would've been solved by file-relative-paths. Cheers and thanks, Yuval Greenfield
On Tue, Nov 13, 2018 at 09:34:33PM -0800, Yuval Greenfield wrote:
How does that differ from existing syntax? from .. import main Off the top of my head, a few more questions that don't have obvious answers (at least not to me): What happens if main.py doesn't exist, but main.pyc does? What if you want to import from a sub-package, rather than a single-file module? What happens when Windows users use a backslash instead of a forward-slash? Does this syntax support arbitrary relative paths anywhere on the file system, or is it limited to only searching the current package? How does it interact with namespace packages? What happens if you call os.chdir() before calling this? Invariably people will want to write things like: path = '../spam.py' import path as spam (I know that's something I'd try.) What will happen there? If that is supported, invariably people will want to use pathlib.Path objects. Should that work?
Beware of "obvious" solutions, because so often they lead to not so obvious problems. Like Javascript's "relative import hell": Quote: // what we want import reducer from 'reducer'; // what we don't want import reducer from '../../../reducer'; https://medium.com/@sherryhsu/how-to-change-relative-paths-to-absolute-paths... And more here: https://goenning.net/2017/07/21/how-to-avoid-relative-path-hell-javascript-t... https://lostechies.com/derickbailey/2014/02/20/how-i-work-around-the-require... It seems to me that in languages which support this file-relative import feature, people spend a lot of time either trying to avoid using it, or building tools to allow them to avoid using it. I don't know if that makes it better or worse than Python's solution for relative imports :-) -- Steve
On Wed, Nov 14, 2018 at 5:15 PM Steven D'Aprano <steve@pearwood.info> wrote:
Agreed. Having spent a lot of time with JavaScript students, I actually am NOT a fan of directory-relative imports. They inevitably result in equivalent code looking different, and subtly different code looking identical. Consider: // main.js import User from './models/users'; // routers/users.js import User from '../models/users'; That example is probably okay, because if you ever get it wrong, you get an immediate error. But what about this? // routers/index.js import usersRouter from './users'; // models/stuff.js import User from './users'; The exact same import does completely different things based on which file it's in. That's dangerous, because it makes code subtly context sensitive. I would much rather work with package-relative pathing, where there is a known basis for *all* local imports, no matter what file the import is actually happening in. In Python, that's best done by naming the package again, so that's not quite ideal either, but it's better than having to pile in the exact right number of "../" to make the import work. ChrisA
On Fri, Nov 9, 2018 at 4:32 PM, danish bluecheese <danish.bluecheese@gmail.com> wrote:
Have you tried 'python3 -m test_main'? IIRC it should be effectively the same as 'python3 test_main.py' but with working relative imports. -n -- Nathaniel J. Smith -- https://vorpus.org
participants (6)
-
Chris Angelico -
Chris Barker - NOAA Federal -
danish bluecheese -
Nathaniel Smith -
Steven D'Aprano -
Yuval Greenfield