how to organize a module that requires a data file

[Steven Bethard]

Terry Hancock wrote:
> On Thu, 17 Nov 2005 12:18:51 -0700
> Steven Bethard <steven.bethard at gmail.com> wrote:
> 
>>My problem is with the text file.  Where should I keep it?
>>
>>I can only think of a few obvious places where I could
>>find the text  file at import time -- in the same
>>directory as the module (e.g.  lib/site-packages), in the
>>user's home directory, or in a directory  indicated by an
>>environment variable.  
> 
> 
> Why don't you search those places in order for it?
> 
> Check ~/.mymod/myfile, then /etc/mymod/myfile, then
> /lib/site-packages/mymod/myfile or whatever. It won't take
> long, just do the existence checks on import of the module.
> If you don't find it after checking those places, *then*
> raise an exception.
> 
> You don't say what this data file is or whether it is
> subject to change or customization. If it is, then there is
> a real justification for this approach, because an
> individual user might want to shadow the system install with
> his own version of the data.

The file is a lookup table of word stems distributed by the University 
of Pennsylvania.  It doesn't really make sense for users to customize 
it, because it's not a configuration file, but it is possible that UPenn 
would distribute a new version at some point.  That's what I meant when 
I said "it's not a per-user configuration - it's a data file shared by 
all users".  So there should be exactly one copy of the file, so I 
shouldn't have to deal with shadowing.

Of course, even with only one copy of the file, that doesn't mean that I 
couldn't search a few places.  Maybe I could by default put it in 
lib/site-packages, but allow an option to setup.py to put it somewhere 
else for anyone who was worried about putting 10MB into 
lib/site-packages.  Those folks would then have to use an environment 
variable, say $MORPH_FLAT, to identify the directory they .  At module 
import I would just check both locations...

I'll have to think about this some more...

STeVe