[Import-SIG] New PEP draft: "Simplified Package Layout and Partitioning"

[P.J. Eby]

At 08:07 AM 7/19/2011 +1000, Nick Coghlan wrote:
>On Tue, Jul 19, 2011 at 2:17 AM, Barry Warsaw <barry at python.org> wrote:
> > 2. The __file__ issue.  My gut tells me that pure virtual modules 
> should have
> >   None as their __file__.  It seems wrong to use anything else, and your
> >   "accidentally work" observation is not calming. ;)
> >
> >   The inability to use __file__ to find data files is somewhat troubling
> >   though.  Let's say we want to find the foo/test/data subdir above, and
> >   `foo` is pure-virtual, while `test` is an __init__.py-less package.
> >
> >   I'm fine not being able to use foo.__file__, but I will probably want to
> >   use `os.path.join(foo.test.__file__, 'data')`.  Will that 
> work?  What would
> >   foo.test's __file__ be?  The `foo/test` directory perhaps?  Of 
> course there
> >   could be multiple `foo/test` directories, so this is probably why your
> >   suggesting to search foo.test.__path__ instead.
> >
> >   I'd actually be okay with that, *if* pkg_resources will be updated to
> >   handle this case.  In general, we've been recommending people use
> >   pkg_resources anyway (wasn't there a push to move part of this 
> package into
> >   the stdlib?).
>
>pkgutil.get_data() needs to be updated to handle this case, so
>retrieving the contents of a specific file in the directory above
>could be written as either of the following:
>
>pkgutil.get_data(foo, 'test/data/file.dat')
>pkgutil.get_data(foo.test, 'data/file.dat')

Really, these should be done relative to either a module or a 
self-contained package, unless we want to modify these things to 
search the __path__ -- and I'm not entirely sure that we do.


>The question of PEP 302 and listing *available* data files (and other
>directory-style or lazy data access I/O operations) remains open
>(independent of the changes in this PEP). Note that os.path.join based
>approaches already break as soon as you put the package and data files
>in a zipfile.
>
>In reality, I believe people should be using the appropriate packaging
>APIs so that source files and data files may be deployed to distinct
>locations.

Indeed.