[Distutils] "Python Package Management Sucks"

[Phillip J. Eby]

At 03:14 PM 10/1/2008 -0700, Toshio Kuratomi wrote:
>resources, as I said needs to be defined.  You're saying here that a
>resource is something internal to the library.  A "file" is something
>that a user can read, copy, or modify.

I should probably clarify that I mean "unmediated by the 
program"...  which is why I disagree regarding message 
catalogs.  They're not user-modifiable and there's nothing you can 
usefully do with them outside the program that uses them.  Of course, 
a default message catalog for someone to use to *create* translations 
from might be another story...


>In a typical TurboGears app, there's the following things to be found
>inside of the app's directory in site-packages:
>
>config/{app.cfg,__init__.py,log.cfg} - These could go in /etc/ as their
>configuration.  However, I've tried to stress to upstream that only
>things that configure the TurboGears framework for use with their app
>should go in these files (default templating language, identity
>controller).  When those things are true, I can see this as being an
>"internal resource".  If upstream can't get their act together, it's config.

Agreed.


>locale/{message catalogs for various languages} --  These are binary
>files that contain strings that the user may see when a message is
>given.  These, I think are data.

I lean the other way, since they're not editable.  Keep in mind that 
some platforms have no "locale" directories as such, and thus trying 
to support multiple locations thereof is a burden for cross-platform 
app developers, vs. simply treating them as internal resources.  This 
is especially true for plugin-oriented architectures that want to 
distribute localizations as plugins, with one plugin being able to 
supply localization for another.


>templates/*html -- These are templates that the application fills in
>with variables intermixed with short bits of code.  These are on the
>border between code and data.  The user sees them in a modified form.
>The app sometimes executes pieces of them before the user sees them.
>Some template languages create python byte code from the templates,
>others load them and write into them every time.  None of them can be
>executed on their own.  All of them have to be loaded by a call to parse
>them from a piece of python code in another file.  None of them are
>directly called or invoked.  My leaning is that these are data.

If you follow this logic (create bytecode from it, can't run w/out 
interp or compiler), then any .py file is *also* data; i.e., this 
Does Not Follow.


>static/{javascript,css,images} -- These are things that are definitely
>never executed.  They are served by the webserver verbatim when their
>URL is called.  These are certainly data. (Note: I don't believe these
>are referenced using the resources API, just via URL.)

Uh...  that would depend entirely on the library or application.  But 
if they're static and the user's got no business tinkering with them, 
then it's a resource.



>So... do you agree on which of these are data and which are resources?
>Do you have an idea on how we can prevent application and framework
>writers from misusing the resources API to load things that are data?

Apparently not.  The alternative I would suggest is that under the 
new standard, an install tool should be allowed to relocate any 
non-Python files, and all access has to go through a resource 
API.  The install tool would then have to be responsible for putting 
some kind of forwarding information in the package directory to tell 
the resource API where it squirrelled the file(s) off to.  Then we 
can avoid all this angels-on-a-pin argument and the distros can Have 
It Their Way[tm].

I'd have preferred to avoid that complexity, but if the two of us 
can't agree then there's no way on earth to get a community consensus.

Btw, pkg_resources' concept of "metadata" would also need to be 
relocatable, since e.g. the "EggTranslations" package uses that 
metadata to store localizations of image resources and message 
catalogs.  (Other uses of the metadata files also inlcude scripts, 
dependencies, version info, etc.)

Hopefully, those folks who want relocation ability will chip in with 
code, docs, testing, etc. for the feature at some point.