[Python-ideas] Add appdirs module to stdlib

Andrew Barnert abarnert at yahoo.com
Tue Sep 1 23:47:22 CEST 2015

Responses below, but first, another issue:

Things like app data and prefs aren't a single directory. XDG has a notion of a search path rather than a single directory; Windows has a notion of separate search domains; OS X makes things as fun as possible by having both. For writing a new file, you're usually fine just writing to the first path in the default domain (as long as it exists or you can create it), but for reading files you're supposed to look in /etc or All Users or whatever if it's not found there. Most cross-platform wrappers I've used in the past didn't deal with this automatically, and a lot of them didn't even make it easy to do manually.

On Sep 1, 2015, at 14:19, Nathaniel Smith <njs at vorpus.org> wrote:
> On Sep 1, 2015 02:45, "Andrew Barnert via Python-ideas"
> <python-ideas at python.org> wrote:
>>> On Sep 1, 2015, at 01:00, Philipp A. <flying-sheep at web.de> wrote:
>>> When defining a place for config files, cache files, and so on, people usually hack around in a OS-dependent, misinformed, and therefore wrong way.
>>> Thanks to the tempfile API we at least don’t see people hardcoding /tmp/ too much.
>>> There is a beautiful little module that does things right and is easy to use: appdirs
>> Is appdirs compatible with the OS X recommendations (as required by the App Store). Apple only gives you cache and app data directories; prefs are supposed to use NSDefaults API or emulate the file names and formats properly, and you have to be sensitive to the sandbox.)
> No, AFAICT it doesn't get this right -- it just hard-codes the OS X
> directories.

The biggest problem with most of the cross-platform libraries I've seen is that they assume there is a prefs directory, and on OS X, that really isn't true. If your app explicitly opens the exact same file that NSDefaults would have opened, you're breaking the rules. (Since the mandatory sandbox went into effect, this usually doesn't get you rejected from the App Store anymore, but before that it did.) And taking a quick look at appdirs, it has a user_config_dir that seems to mean exactly that. So, how can a stdlib library handle that?

Meanwhile, it looks like appdirs expects you to give it an app name and company name to construct the paths. What happens if you give it names that don't match the ones in your bundle? You're opening files that belong to another app. Which is again violating Apple's rules.

One more thing: I don't know if it's guaranteed that the right way of doing things on OS X (whether via Cocoa or CoreFoundation) won't spawn a background thread for you. After all, the APIs can talk to the sandbox service and sometimes even the iCloud service. Is that a problem for something in the stdlib?

> It also didn't quite implement the XDG spec correctly
> (there's some fallback behavior you're supposed to do if the magic
> envvars don't make sense that it skips -- very unusual that this will
> matter). And windows I'm not sure about -- the logic in appdirs looked
> reasonable to me when I was reviewing this a few months ago, but there
> seem to be a bunch of semi-contradictory standards and so it's hard to
> know what's even "correct" in the tricky cases.
> All of this is probably as much an argument *for* providing the
> functionality as a standard thing as it is against,
> but any PEP here
> probably needs to be thorough about citing the research to show that
> it's actually getting the various platform standards correct.
> What makes it particularly difficult is that if you "fix a bug" in a
> library like appdirs, so that it starts suddenly returning different
> results on some computer somewhere, then what it looks like to the end
> user is that their data/settings/whatever have suddenly evaporated and
> whatever disk space was being used for caches never gets cleaned up
> and so forth. Generally when applications change how they compute
> these directories, they also include tricky migration logic to check
> both the old and new names, move stuff over if needed, but I'm not
> sure how a low-level library like this can support that usefully...

And that's especially true in the case of Apple's standards, which also include specific rules about how you're supposed to do such a migration, and doing so requires stuff that can't be done from entirely inside the code.

More information about the Python-ideas mailing list