pathlib.Path should handle Pythonpath or package root
Hi there, I created a program which uses plugins (import them). I started to test it, and found that I need two types of paths: one for file system and another one which is package relative. So I thing this is a good idea, to enhance pathlib to handle package roots. (I know about sys.path, environment variables, importlib etc, but I think it should be in pathlib.) BR, George
On 18 July 2017 at 22:08, George Fischhof <george@fischhof.hu> wrote:
Hi there,
I created a program which uses plugins (import them). I started to test it, and found that I need two types of paths: one for file system and another one which is package relative.
So I thing this is a good idea, to enhance pathlib to handle package roots.
Is there a specific behaviour you're looking for that isn't already addressed by "pathlib.Path(module.__file__).parent"? Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
2017-07-18 15:56 GMT+02:00 Nick Coghlan <ncoghlan@gmail.com>:
On 18 July 2017 at 22:08, George Fischhof <george@fischhof.hu> wrote:
Hi there,
I created a program which uses plugins (import them). I started to test it, and found that I need two types of paths: one for file system and another one which is package relative.
So I thing this is a good idea, to enhance pathlib to handle package roots.
Is there a specific behaviour you're looking for that isn't already addressed by "pathlib.Path(module.__file__).parent"?
Cheers, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Hi Nick, I think yes ;-) I would like to use (or I think it would be good to use) something like pathlib.Path(package_root) so I could use importlib.import_module(pathlib.Path(package_root) / plugins / plugin_name) and normal file system operations (for example) with open(pathlib.Path(package_root) / plugins / plugin_name) as my_file: do_something_with_file Import statement can be used to go downward and path can be used up and down in the directory hierarchy. If someone wants to use both of them, the only common point (branch) is the package root. From the package root one can use the same path expressions for import and for other file system operations. BR, George
On 19 July 2017 at 06:40, George Fischhof <george@fischhof.hu> wrote:
I think yes ;-) I would like to use (or I think it would be good to use) something like pathlib.Path(package_root) so I could use
importlib.import_module(pathlib.Path(package_root) / plugins / plugin_name)
No, as that's fundamentally incompatible with how Python's import system works - the filesystem is *a* way of representing package namespacing, but far from the only way. Managing the import state also has nothing whatsoever to do with pathlib. That said, the idea of better encapsulating the import state so we can more readily have constrained "import engines" *is* a reasonable one, it just runs into significant practical problems related to the handling of transitive imports in both Python modules and (especially) extension modules. The last serious attempt at pursuing something like that is documented in PEP 406, "Improved Encapsulation of Import State": https://www.python.org/dev/peps/pep-0406/ Unfortunately, the main outcome of Greg Slodkowicz's GSoC work on the idea was to conclude that that particular approach wasn't viable due to the fact that import system plugins at that time were pretty much required to directly manipulate global state, which ran directly counter to the goal of encapsulation. However, we also haven't had anyone seriously revisit the idea since the updated import plugin API was defined in PEP 451 - that deliberately moved a lot of the global state management out of the plugins and into the import system, so it should be more amenable to an "import engine" style approach to state encapsulation. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Sorry it was misunderstandable. I do not want to touch the import. I just would like to access the package root (of my current script). Practically in a Path object if the package resides in the file system. Parent is not enough, because the place of actual running script can vary. BR, George 2017-07-19 3:51 GMT+02:00 Nick Coghlan <ncoghlan@gmail.com>:
I think yes ;-) I would like to use (or I think it would be good to use) something like pathlib.Path(package_root) so I could use
importlib.import_module(pathlib.Path(package_root) / plugins /
On 19 July 2017 at 06:40, George Fischhof <george@fischhof.hu> wrote: plugin_name)
No, as that's fundamentally incompatible with how Python's import system works - the filesystem is *a* way of representing package namespacing, but far from the only way. Managing the import state also has nothing whatsoever to do with pathlib.
That said, the idea of better encapsulating the import state so we can more readily have constrained "import engines" *is* a reasonable one, it just runs into significant practical problems related to the handling of transitive imports in both Python modules and (especially) extension modules.
The last serious attempt at pursuing something like that is documented in PEP 406, "Improved Encapsulation of Import State": https://www.python.org/dev/peps/pep-0406/
Unfortunately, the main outcome of Greg Slodkowicz's GSoC work on the idea was to conclude that that particular approach wasn't viable due to the fact that import system plugins at that time were pretty much required to directly manipulate global state, which ran directly counter to the goal of encapsulation.
However, we also haven't had anyone seriously revisit the idea since the updated import plugin API was defined in PEP 451 - that deliberately moved a lot of the global state management out of the plugins and into the import system, so it should be more amenable to an "import engine" style approach to state encapsulation.
Cheers, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 19 July 2017 at 22:01, George Fischhof <george@fischhof.hu> wrote:
Sorry it was misunderstandable. I do not want to touch the import.
I just would like to access the package root (of my current script). Practically in a Path object if the package resides in the file system. Parent is not enough, because the place of actual running script can vary.
If you just want to do relative imports, then that's the purpose of the explicit relative import syntax ("from . import sibling" etc). You can also do dynamic relative imports by passing "__package__" as the second argument to importlib.import_module. If you're looking to read *data* files relative to the current module, then the API you're likely after is `pkgutil.get_data`: https://docs.python.org/3/library/pkgutil.html#pkgutil.get_data And if you're looking to access files relative to *__main__*, then the metadata atribute you want is __main__.__file__. So it's currently still really unclear what you actually mean by "package root", and what related metadata you see as currently being missing at runtime. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
participants (2)
-
George Fischhof -
Nick Coghlan