On 1 February 2014 17:26, Brett Cannon <brett@python.org> wrote:
PEP 302 tried to unify this with get_data() and set_data() on loaders, but prior to Python 3.3 you just didn't have any guarantee that __loader__ would even be set, let alone have a loader with those methods. Paul can tell me if my hunch is off, but I assume the dream was that people would do `__loader__.get_data('asset.txt')` instead of `os.path.join(os.path.dirname(__file__), 'asset.txt')` to read something bundled with your package. But as Brian has pointed out, people just have habits at this point of assuming unpacked files and working off of __file__.
That's right. Unfortunately, (a) people have habits, as you say, (b) the loader API was too limited - for good reason, we couldn't add any more without breaking more than we gained at the time, and (c) things like ssl that need files as Donald points out are always a problem. Setuptools (and distlib more recently) tried to address this with unpack-on-demand tricks, but honestly non-filesystem loaders are still such an uncommon use case that most people don't bother. It's just a very, very slow process of change.
I know Daniel has said he wanted some concept of a listdir() on loaders so that they could basically act as a virtual file system for packages, but it would also require locking down what relative vs. absolute paths meant in get_data/set_data (i.e. is it relative to the loader in the package or the package itself? My pref is the latter for the case of reused loaders) and really pushing people to use the loader APIs for reading intra-package "files".
That is probably the most annoying omission from PEP 302. IIRC, we did want to add it, but couldn't. But I can't recall the reason now. Maybe it could be remedied - it really is a pain. Paul