I had announced this to import-sig already; now python-dev.
I have now written an implementation of PEP 382, and fixed some details of the PEP in the process. The implementation is available at
http://hg.python.org/features/pep-382-2/
With this PEP, a Python package P can consist of either a P/__init__.py directory and module, or multiple P.pyp directories, or both. Using a directory suffix resulted from the following requirements/observations: - people apparently prefer an approach where a directory has to be declared as a Python package (several people commented this after my PyConDE talk, expressing dislike of how Java packages are "unflagged" directories) - people also commented that any declaration should indicate that this is about Python, hence the choice of .pyp as the directory suffix. - in choosing between a file marker inside of the directory (e.g. zope-interfaces.pyp) and a directory suffix, the directory suffix wins for simplicity reasons. A file marker would have to have a name which wouldn't matter except that it needs to be unique - which is a confusing requirement that people likely would fail to meet.
In the new form, the PEP was much easier to implement than in the first version (plus I understand import.c better now).
This implementation now features .pyp directories, zipimporter support, documentation and test cases.
As the next step, I'd like to advance this to ask for pronouncement.
Regards, Martin
On Sun, Nov 6, 2011 at 6:10 PM, "Martin v. Löwis" martin@v.loewis.de wrote:
I had announced this to import-sig already; now python-dev.
I have now written an implementation of PEP 382, and fixed some details of the PEP in the process. The implementation is available at
http://hg.python.org/features/pep-382-2/
With this PEP, a Python package P can consist of either a P/__init__.py directory and module, or multiple P.pyp directories, or both. Using a directory suffix resulted from the following requirements/observations:
- people apparently prefer an approach where a directory has to be
declared as a Python package (several people commented this after my PyConDE talk, expressing dislike of how Java packages are "unflagged" directories)
- people also commented that any declaration should indicate that this
is about Python, hence the choice of .pyp as the directory suffix.
- in choosing between a file marker inside of the directory (e.g.
zope-interfaces.pyp) and a directory suffix, the directory suffix wins for simplicity reasons. A file marker would have to have a name which wouldn't matter except that it needs to be unique - which is a confusing requirement that people likely would fail to meet.
I finally got around to doing the search Barry and I promised for the previously raised objections to the directory suffix approach.
They were in the "Rejected Alternatives" section of PJE's proposed redraft of PEP 382 (before he decided to create PEP 402 as a competing proposal):
============= * Another approach considered during revisions to this PEP was to simply rename package directories to add a suffix like ``.ns`` or ``-ns``, to indicate their namespaced nature. This would effect a small performance improvement for the initial import of a namespace package, avoid the need to create empty ``*.ns`` files, and even make it clearer that the directory involved is a namespace portion.
The downsides, however, are also plentiful. If a package starts its life as a normal package, it must be renamed when it becomes a namespace, with the implied consequences for revision control tools.
Further, there is an immense body of existing code (including the distutils and many other packaging tools) that expect a package directory's name to be the same as the package name. And porting existing Python 2.x namespace packages to Python 3 would require widespread directory renaming as well.
In short, this approach would require a vastly larger number of changes to both the standard library and third-party code, for a tiny potential performance improvement and a small increase in clarity. It was therefore rejected on "practicality vs. purity" grounds. =============
I think this was based on the assumption that *existing* namespace package approaches would break under the new scheme. Since that is not the case, I suspect those previous objections were overstated (and all packaging related code manages to cope well enough with modules where the file name doesn't match the package name)
Cheers, Nick.
On Sun, Nov 6, 2011 at 7:29 AM, Nick Coghlan ncoghlan@gmail.com wrote:
I think this was based on the assumption that *existing* namespace package approaches would break under the new scheme. Since that is not the case, I suspect those previous objections were overstated (and all packaging related code manages to cope well enough with modules where the file name doesn't match the package name)
I was actually referring to all the code that does things like split package names on '.' and then use os.path.join, or that makes assumptions which are the moral equivalent of that. PEP 402's version of namespace packages should break less of that sort of code than adding a directory name extension.
Am 06.11.2011 16:38, schrieb PJ Eby:
On Sun, Nov 6, 2011 at 7:29 AM, Nick Coghlan <ncoghlan@gmail.com mailto:ncoghlan@gmail.com> wrote:
I think this was based on the assumption that *existing* namespace package approaches would break under the new scheme. Since that is not the case, I suspect those previous objections were overstated (and all packaging related code manages to cope well enough with modules where the file name doesn't match the package name)I was actually referring to all the code that does things like split package names on '.' and then use os.path.join, or that makes assumptions which are the moral equivalent of that. PEP 402's version of namespace packages should break less of that sort of code than adding a directory name extension.
I think tools emulating the import mechanism will break no matter what change is made: the whole point of changing it is that it does something new that didn't work before.
I think adjusting the tools will be straight-forward: they already need to recognize that an imported name could come either from a file (with various extensions), or a directory with special properties. So extending this should be "easy". Also, the number of tools that emulate the Python import algorithm is rather small.
Tools that merely inspect __path__ after importing a package will continue to work just fine even under PEP 382.
Regards, Martin
I think this was based on the assumption that *existing* namespace package approaches would break under the new scheme. Since that is not the case, I suspect those previous objections were overstated (and all packaging related code manages to cope well enough with modules where the file name doesn't match the package name)
I just drafted a message rebutting Phillip's objection, when I then found that you disagree as well :-)
One elaboration:
The downsides, however, are also plentiful. If a package starts its life as a normal package, it must be renamed when it becomes a namespace, with the implied consequences for revision control tools.
If a package starts out as a regular (P/__init__.py) package (possibly with __init__.py being empty), then making it a namespace package actually requires no change at all - that single __init__.py could continue to exist.
Developers may decide to still delete __init__.py and rename the package to .pyp (with VCS consequences), but that would be their choice deciding whether purity beats practicality in their case (where I think most developers actually would rename the directory once they can drop support for pre-3.3 releases).
Regards, Martin
-
"Martin v. Löwis" -
Nick Coghlan -
PJ Eby