[Import-SIG] New PEP draft: "Simplified Package Layout and Partitioning"

[Nick Coghlan]

On Tue, Jul 19, 2011 at 2:17 AM, Barry Warsaw <barry at python.org> wrote:
> 2. The __file__ issue.  My gut tells me that pure virtual modules should have
>   None as their __file__.  It seems wrong to use anything else, and your
>   "accidentally work" observation is not calming. ;)
>
>   The inability to use __file__ to find data files is somewhat troubling
>   though.  Let's say we want to find the foo/test/data subdir above, and
>   `foo` is pure-virtual, while `test` is an __init__.py-less package.
>
>   I'm fine not being able to use foo.__file__, but I will probably want to
>   use `os.path.join(foo.test.__file__, 'data')`.  Will that work?  What would
>   foo.test's __file__ be?  The `foo/test` directory perhaps?  Of course there
>   could be multiple `foo/test` directories, so this is probably why your
>   suggesting to search foo.test.__path__ instead.
>
>   I'd actually be okay with that, *if* pkg_resources will be updated to
>   handle this case.  In general, we've been recommending people use
>   pkg_resources anyway (wasn't there a push to move part of this package into
>   the stdlib?).

pkgutil.get_data() needs to be updated to handle this case, so
retrieving the contents of a specific file in the directory above
could be written as either of the following:

pkgutil.get_data(foo, 'test/data/file.dat')
pkgutil.get_data(foo.test, 'data/file.dat')

The question of PEP 302 and listing *available* data files (and other
directory-style or lazy data access I/O operations) remains open
(independent of the changes in this PEP). Note that os.path.join based
approaches already break as soon as you put the package and data files
in a zipfile.

In reality, I believe people should be using the appropriate packaging
APIs so that source files and data files may be deployed to distinct
locations.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia