Re: [Python-ideas] Add appdirs module to stdlib
Responses below, but first, another issue: Things like app data and prefs aren't a single directory. XDG has a notion of a search path rather than a single directory; Windows has a notion of separate search domains; OS X makes things as fun as possible by having both. For writing a new file, you're usually fine just writing to the first path in the default domain (as long as it exists or you can create it), but for reading files you're supposed to look in /etc or All Users or whatever if it's not found there. Most cross-platform wrappers I've used in the past didn't deal with this automatically, and a lot of them didn't even make it easy to do manually. On Sep 1, 2015, at 14:19, Nathaniel Smith <njs@vorpus.org> wrote:
The biggest problem with most of the cross-platform libraries I've seen is that they assume there is a prefs directory, and on OS X, that really isn't true. If your app explicitly opens the exact same file that NSDefaults would have opened, you're breaking the rules. (Since the mandatory sandbox went into effect, this usually doesn't get you rejected from the App Store anymore, but before that it did.) And taking a quick look at appdirs, it has a user_config_dir that seems to mean exactly that. So, how can a stdlib library handle that? Meanwhile, it looks like appdirs expects you to give it an app name and company name to construct the paths. What happens if you give it names that don't match the ones in your bundle? You're opening files that belong to another app. Which is again violating Apple's rules. One more thing: I don't know if it's guaranteed that the right way of doing things on OS X (whether via Cocoa or CoreFoundation) won't spawn a background thread for you. After all, the APIs can talk to the sandbox service and sometimes even the iCloud service. Is that a problem for something in the stdlib?
And that's especially true in the case of Apple's standards, which also include specific rules about how you're supposed to do such a migration, and doing so requires stuff that can't be done from entirely inside the code.
On 1 September 2015 at 22:47, Andrew Barnert via Python-ideas <python-ideas@python.org> wrote:
Things like app data and prefs aren't a single directory. XDG has a notion of a search path rather than a single directory; Windows has a notion of separate search domains; OS X makes things as fun as possible by having both. For writing a new file, you're usually fine just writing to the first path in the default domain (as long as it exists or you can create it), but for reading files you're supposed to look in /etc or All Users or whatever if it's not found there. Most cross-platform wrappers I've used in the past didn't deal with this automatically, and a lot of them didn't even make it easy to do manually.
This is a fair point. But it's also worth noting that the current state of affairs for many apps is to just bung stuff in ~/whatever. While appdirs may not get things totally right, at least it improves things. And if it (or something similar) were in the stdlib, it would at least provide a level of uniformity. So, in my view: 1. We should have something that provides the functionality of appdirs in the stdlib. 2. It probably needs a PEP to get the corner cases right. 3. The behaviour of appdirs is a good baseline default - even if it isn't 100% compliant with platform standards it'll be better than what someone unfamiliar with the platform will invent. 4. We shouldn't abandon the idea just because a perfect solution is unattainable. There are complex cases to consider (search paths, for example, and even worse how search paths interact with the app writing config data rather than just reading it, or migration when a scheme changes). The PEP should at least mention these cases, but it's not unreasonable to simply declare them out of scope of the module (most applications don't need anything this complex). Paul
On Wed, Sep 2, 2015 at 9:05 AM, Paul Moore <p.f.moore@gmail.com> wrote:
+1
Might be worth starting with something simple: ask for one directory (the default or most obvious place), or ask for a full list of plausible directories to try. Then a config manager could be built on top of that which would handle write location selection, migration, etc, and that would be a separate proposal that makes use of the appdata module for the cross-platform stuff. ChrisA
On 2 September 2015 at 11:05, Paul Moore <p.f.moore@gmail.com> wrote:
In about 5 years time. Maybe, The adoption curve for something that works on all Pythons is able to be much much higher than that for something which is only in the stdlib 6 months (or more) from now. Unless we do a rolling backport of it. And if we're going to do that... why? Why not just provide a documentation link to the thing and say 'pip install this' and/or 'setuptools install_require this'. -Rob -- Robert Collins <rbtcollins@hp.com> Distinguished Technologist HP Converged Cloud
On 2 September 2015 at 14:38, Robert Collins <robertc@robertcollins.net> wrote:
My perspective on that has been shifting in recent years, to the point where I view this kind of standard library modules primarily as a tool to helps folks learn how the computing ecosystem works in practice. Consider PEP 3144, for example, and the differences between ipaddress, and its inspiration, ipaddr. The standard library one is significantly stricter about correctly using networking terminology, so you can actually study the ipaddress module as a way of learning how IP addressing works. The original ipaddr, by contrast, is only easy to use if you already know all the terms, and can say "Oh, OK, they're using that term somewhat loosely, but I can see what they mean". I think this is a case where a similar approach would make sense - like ipaddr before it, appdirs represents an actual cross-version production module, put together by a company (in this case ActiveState rather than Google) for their own use, but made available to the wider community Python through PyPI. As such, we know it's feature coverage is likely to be good, but the API design is likely to be optimised for experienced developers that already understand the concepts, and just want a library to handle the specific technical details. A standard library API would shift the emphasis slightly, and take into account the perspective of the *beginning* programmer, who may have only first learned about the command line and files and directories in the course of learning Python, and is now venturing into the realm of designing full desktop (and mobile?) applications. Regards, Nick. P.S. The statistics module is another example of being able to use the Python standard as a teaching tool to help learn something else: for many production use cases, you'd reach for something more sophisticated (like the NumPy stack), but if what you're aiming to do is to learn (or teach) basic statistical concepts, it covers the essentials. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 2 September 2015 at 06:57, Nick Coghlan <ncoghlan@gmail.com> wrote:
Agreed. In this case, the focus should be on providing "correct" cross-platform defaults, and assisting (and encouraging) users to understand the choices and constraints applicable to other platforms, which may not be relevant on theirs. This may feel like a nuisance for single-platform developers (a Unix-only program doesn't need to care about local or roaming preferences, because they don't have the concept of a domain account), so it's important to get the defaults right so that people who don't need to care, don't have to. But the options should be accessible, so that people can learn to make the right choices (for example, pip puts preferences in the roaming profile, but the cache in the local profile, because you don't want to bloat the roaming profile with a cache, but you do want the user's preferences to be available on all the machines they use). Paul
OK, original poster here. thanks for the positive reception! so there are some issues which i’ll address 1. *appdirs doesn’t get everything right* → in order not to have inconsistencies, we could abandon the “appdirs as a fallback” approach and create our own API which returns more correct results. this also frees us to make other changes where we see fit, e.g. see next point 2. *there is a search path instead of a single directory sometimes* → appdirs provides a multipath keyword argument which returns a (colon-delimited) “list” of paths. we should provide sth. similar, only with python lists. maybe also convenience functions for getting a list of all files matching some subpath similar to [d / subpath for d in site_data_dir(appname, appauthor, all=True) if (d / subpath).exists()] 3. *some platforms don’t have some specific subset of the functionality (e.g. no config dir on OSX)* → provide a warning in the stdlib docs about that and refer to another settings API. unfortunately i can’t find a good NSDefaults library in python right now. i think the API should still return some directory that works for storing settings on OSX in case people want to use platform-independent config files. 4. *it’s hard to tell newbies where their files are* → unfortunately that’s how things are. the existing standards are confusing, but should be honored nonetheless. we can provide an api like python -m stddirs [--config|--data|...] <companyname> <appname> which prints a selection of standard directories. did i miss anything? and yeah: i also think that something that’s a little opinionated in case of ambiguities is vastly better than everyone hacking their own impromptu platform-dependent alternative. ~/.appname stopped being right on linux long ago and never was right on other platforms, which we should teach people. best, phil Nick Coghlan <ncoghlan@gmail.com> schrieb am Mi., 2. Sep. 2015 um 07:57 Uhr:
On 02.09.2015 10:10, Philipp A. wrote:
~/.appname stopped being right on linux long ago and never was right on other platforms, which we should teach people.
Looking at the my home dir on Linux, there doesn't seem to be one standard, but rather a whole set of them and the good old ~/.appname is still a popular one (e.g. pip and ansible from Python land still use it; as do many other non-Python applications such as ncftp, emacs, svn, git, gpg, etc.). ~/.config/ does get some use, but mostly for GUI applications, not so much for command line ones. ~/.local/lib/ only appears to be used by Python :-) ~/.local/share/ is mostly used by desktops to register application shortcuts ~/.cache/ is being used by just a handful of tools, pip being one of them. appdirs seems to rely on the XDG Base Directory Specification for a lot of things on Linux (http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html). That's probably fine for desktop GUI apps (the standard was apparently built for this use case), but doesn't apply at all for command line tools or applications like daemons or services which don't interact with the desktop, e.g. you typically won't find global config files for command line tools under /etc/xdg/, but instead under /etc/. For Windows, the CSIDL_* values have also been replaced with new ones under FOLDERID_* (the APIs have also evolved): Values: https://msdn.microsoft.com/en-us/library/windows/desktop/bb762494%28v=vs.85%... https://msdn.microsoft.com/en-us/library/windows/desktop/dd378457%28v=vs.85%... APIs: https://msdn.microsoft.com/en-us/library/windows/desktop/bb762181%28v=vs.85%... https://msdn.microsoft.com/en-us/library/windows/desktop/bb762188%28v=vs.85%... BTW: I wonder why the Windows functions in appdirs don't use the environment for much easier access to e.g. APPDATA and USERPROFILE. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Sep 02 2015)
2015-08-27: Released eGenix mx Base 3.2.9 ... http://egenix.com/go83 ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
I just commented on the ticket that references this discussion: http://bugs.python.org/issue7175 In essence, Python already has an installation scheme which is defined in sysconfig.py (see the top of the file) and has had this ever since distutils got added to the stdlib. It just lacks explicit entries for "config" and "cache" files, so adding those would be more in line with coming up with yet another standard, e.g. for posix_user: 'posix_user': { 'stdlib': '{userbase}/lib/python{py_version_short}', 'platstdlib': '{userbase}/lib/python{py_version_short}', 'purelib': '{userbase}/lib/python{py_version_short}/site-packages', 'platlib': '{userbase}/lib/python{py_version_short}/site-packages', 'include': '{userbase}/include/python{py_version_short}', 'scripts': '{userbase}/bin', 'config': '{userbase}/etc', 'cache': '{userbase}/var', 'data': '{userbase}', }, ({userbase} is set by looking at PYTHONUSERBASE and defaults to ~/.local/) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Sep 02 2015)
2015-08-27: Released eGenix mx Base 3.2.9 ... http://egenix.com/go83 ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
"BTW: I wonder why the Windows functions in appdirs don't use the environment for much easier access to e.g. APPDATA and USERPROFILE." The environment can become corrupted more easily and it's really difficult to diagnose that from a bug report that says "my config is corrupt". I assume appdirs using ctypes now, but I'd be happy to add the call into the os module to avoid that. Cheers, Steve Top-posted from my Windows Phone -----Original Message----- From: "M.-A. Lemburg" <mal@egenix.com> Sent: 9/2/2015 2:31 To: "Philipp A." <flying-sheep@web.de>; "Nick Coghlan" <ncoghlan@gmail.com>; "Robert Collins" <robertc@robertcollins.net> Cc: "Nathaniel Smith" <njs@vorpus.org>; "python-ideas@python.org" <python-ideas@python.org> Subject: Re: [Python-ideas] Add appdirs module to stdlib On 02.09.2015 10:10, Philipp A. wrote:
~/.appname stopped being right on linux long ago and never was right on other platforms, which we should teach people.
Looking at the my home dir on Linux, there doesn't seem to be one standard, but rather a whole set of them and the good old ~/.appname is still a popular one (e.g. pip and ansible from Python land still use it; as do many other non-Python applications such as ncftp, emacs, svn, git, gpg, etc.). ~/.config/ does get some use, but mostly for GUI applications, not so much for command line ones. ~/.local/lib/ only appears to be used by Python :-) ~/.local/share/ is mostly used by desktops to register application shortcuts ~/.cache/ is being used by just a handful of tools, pip being one of them. appdirs seems to rely on the XDG Base Directory Specification for a lot of things on Linux (http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html). That's probably fine for desktop GUI apps (the standard was apparently built for this use case), but doesn't apply at all for command line tools or applications like daemons or services which don't interact with the desktop, e.g. you typically won't find global config files for command line tools under /etc/xdg/, but instead under /etc/. For Windows, the CSIDL_* values have also been replaced with new ones under FOLDERID_* (the APIs have also evolved): Values: https://msdn.microsoft.com/en-us/library/windows/desktop/bb762494%28v=vs.85%... https://msdn.microsoft.com/en-us/library/windows/desktop/dd378457%28v=vs.85%... APIs: https://msdn.microsoft.com/en-us/library/windows/desktop/bb762181%28v=vs.85%... https://msdn.microsoft.com/en-us/library/windows/desktop/bb762188%28v=vs.85%... BTW: I wonder why the Windows functions in appdirs don't use the environment for much easier access to e.g. APPDATA and USERPROFILE. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Sep 02 2015)
2015-08-27: Released eGenix mx Base 3.2.9 ... http://egenix.com/go83 ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
hi marc-andre, you seem some misconceptions and a *very* different experience from mine: M.-A. Lemburg mal@egenix.com <http://mailto:mal@egenix.com> schrieb am Mi., 2. Sep. 2015 um 11:30 Uhr: Looking at the my home dir on Linux, there doesn't seem to be
one standard, but rather a whole set of them
can you please link to the document from a standards authority describing those other stadards? 😉 and the good old ~/.appname is still a popular one (e.g. pip and ansible
as hinted at by my tongue-in-cheek comment from above: that’s not a standard but an old convention. git uses ${XDG_CONFIG_DIR-$HOME/.config}/git/config (try it!), as does pip. the other ones are old as dirt so this is excusable. regarding newly developed programs i’ll only approve it for some rare exceptions like shells, where ~/.${SHELL}rc really is the only expected place for a config file. (and not that me approving it means something) ~/.config/ does get some use, but mostly for GUI applications,
not so much for command line ones.
where do you get this figure from? the xdg standard doesn’t say anything about only a kind of application being targeted by the standard, and as my fontconfig example shows, even libraries follow it. ~/.local/lib/ only appears to be used by Python :-)
yeah, on my system it doesn’t even exist! but that has a reson which you can read in the standard: only ~/.local/share is pointed at by the default for a standard dir. ~/.local/lib is probably an invention by whoever included that path in the default $PYTHONPATH ~/.local/share/ is mostly used by desktops to register application shortcuts
that’s a big understatement, there’s all kinds of stuff in there. i have 56 dirs and files in there. ~/.cache/ is being used by just a handful of tools, pip being one of them.
hah! various parts of KDE, matplotlib, gstreamer, chromium, fontconfig, atom, shall i go on? --
Marc-Andre Lemburg eGenix.com
i hope i could convince you that this isn’t “some contender among many”, but really *the* standard for some directories on linux. best, phil
<random832@fastmail.us> schrieb am Mi., 2. Sep. 2015 um 15:36 Uhr:
no idea why, but once we start defining our own API, this will be the most important cange from appdirs. maybe they use semicolons on windows, but i really don’t see why we shouldn’t just use python lists. best, philipp
~/.appname stopped being right on linux long ago and never was right on other platforms, which we should teach people.
Ah, yes. I count 17 of those on my Windows machine (!) right now, including .idlerc, .ipython, .matplotlib, .ipylint.d etc., so we've got a ways to go. :)
On 2 September 2015 at 05:38, Robert Collins <robertc@robertcollins.net> wrote:
Most of the programs I write are for Python 3.4 at the moment, and will be for Python 3.5 as soon as it comes out. They won't be distributed outside of my group at work, if indeed anyone but me uses them. They won't care about older versions of Python. I don't even care about cross-platform. But I do want to set up a cache directory, or save some settings, without thinking *too* hard about where to put them. I don't want to have an external dependency because I'm forever running these things from whatever virtualenv I currently have active (sloppy work habits, I know, but that's the point - not everything is a nicely structured development project). For programs that need to support older versions of Python, a backport should be trivial - I can't see that this would need any particularly "modern" Python features. Paul
On Wed, Sep 02, 2015 at 04:38:05PM +1200, Robert Collins wrote:
I can think of two reasons, one minor, one major: 1. Many people behind corporate or school firewalls cannot just "pip install this". By "firewall" I'm talking more figuratively than literally. Of course there may be an actual firewall blocking access to PyPI, but that's typically very easy to bypass (download the library at home and bring it in on a USB stick). Less easy to bypass is corporate/ school policy which prohibits the installation of unapproved software, often being a firing or expulsion offense. Getting approval may be difficult, slow or downright impossible. However, what makes this a minor reason is that software written under those conditions probably won't be distributed outside of the organisation itself, so who cares whether it complies with the standard locations for application data? 2. More importantly is the stdlib itself. As someone has pointed out, the stdlib already drops config files in completely inappropriate locations on Windows, e.g. $HOME/.idlelib. It would be good if the stdlib itself could consistently use the standard locations for files. -- Steve
On 1 September 2015 at 22:47, Andrew Barnert via Python-ideas <python-ideas@python.org> wrote:
Things like app data and prefs aren't a single directory. XDG has a notion of a search path rather than a single directory; Windows has a notion of separate search domains; OS X makes things as fun as possible by having both. For writing a new file, you're usually fine just writing to the first path in the default domain (as long as it exists or you can create it), but for reading files you're supposed to look in /etc or All Users or whatever if it's not found there. Most cross-platform wrappers I've used in the past didn't deal with this automatically, and a lot of them didn't even make it easy to do manually.
This is a fair point. But it's also worth noting that the current state of affairs for many apps is to just bung stuff in ~/whatever. While appdirs may not get things totally right, at least it improves things. And if it (or something similar) were in the stdlib, it would at least provide a level of uniformity. So, in my view: 1. We should have something that provides the functionality of appdirs in the stdlib. 2. It probably needs a PEP to get the corner cases right. 3. The behaviour of appdirs is a good baseline default - even if it isn't 100% compliant with platform standards it'll be better than what someone unfamiliar with the platform will invent. 4. We shouldn't abandon the idea just because a perfect solution is unattainable. There are complex cases to consider (search paths, for example, and even worse how search paths interact with the app writing config data rather than just reading it, or migration when a scheme changes). The PEP should at least mention these cases, but it's not unreasonable to simply declare them out of scope of the module (most applications don't need anything this complex). Paul
On Wed, Sep 2, 2015 at 9:05 AM, Paul Moore <p.f.moore@gmail.com> wrote:
+1
Might be worth starting with something simple: ask for one directory (the default or most obvious place), or ask for a full list of plausible directories to try. Then a config manager could be built on top of that which would handle write location selection, migration, etc, and that would be a separate proposal that makes use of the appdata module for the cross-platform stuff. ChrisA
On 2 September 2015 at 11:05, Paul Moore <p.f.moore@gmail.com> wrote:
In about 5 years time. Maybe, The adoption curve for something that works on all Pythons is able to be much much higher than that for something which is only in the stdlib 6 months (or more) from now. Unless we do a rolling backport of it. And if we're going to do that... why? Why not just provide a documentation link to the thing and say 'pip install this' and/or 'setuptools install_require this'. -Rob -- Robert Collins <rbtcollins@hp.com> Distinguished Technologist HP Converged Cloud
On 2 September 2015 at 14:38, Robert Collins <robertc@robertcollins.net> wrote:
My perspective on that has been shifting in recent years, to the point where I view this kind of standard library modules primarily as a tool to helps folks learn how the computing ecosystem works in practice. Consider PEP 3144, for example, and the differences between ipaddress, and its inspiration, ipaddr. The standard library one is significantly stricter about correctly using networking terminology, so you can actually study the ipaddress module as a way of learning how IP addressing works. The original ipaddr, by contrast, is only easy to use if you already know all the terms, and can say "Oh, OK, they're using that term somewhat loosely, but I can see what they mean". I think this is a case where a similar approach would make sense - like ipaddr before it, appdirs represents an actual cross-version production module, put together by a company (in this case ActiveState rather than Google) for their own use, but made available to the wider community Python through PyPI. As such, we know it's feature coverage is likely to be good, but the API design is likely to be optimised for experienced developers that already understand the concepts, and just want a library to handle the specific technical details. A standard library API would shift the emphasis slightly, and take into account the perspective of the *beginning* programmer, who may have only first learned about the command line and files and directories in the course of learning Python, and is now venturing into the realm of designing full desktop (and mobile?) applications. Regards, Nick. P.S. The statistics module is another example of being able to use the Python standard as a teaching tool to help learn something else: for many production use cases, you'd reach for something more sophisticated (like the NumPy stack), but if what you're aiming to do is to learn (or teach) basic statistical concepts, it covers the essentials. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 2 September 2015 at 06:57, Nick Coghlan <ncoghlan@gmail.com> wrote:
Agreed. In this case, the focus should be on providing "correct" cross-platform defaults, and assisting (and encouraging) users to understand the choices and constraints applicable to other platforms, which may not be relevant on theirs. This may feel like a nuisance for single-platform developers (a Unix-only program doesn't need to care about local or roaming preferences, because they don't have the concept of a domain account), so it's important to get the defaults right so that people who don't need to care, don't have to. But the options should be accessible, so that people can learn to make the right choices (for example, pip puts preferences in the roaming profile, but the cache in the local profile, because you don't want to bloat the roaming profile with a cache, but you do want the user's preferences to be available on all the machines they use). Paul
OK, original poster here. thanks for the positive reception! so there are some issues which i’ll address 1. *appdirs doesn’t get everything right* → in order not to have inconsistencies, we could abandon the “appdirs as a fallback” approach and create our own API which returns more correct results. this also frees us to make other changes where we see fit, e.g. see next point 2. *there is a search path instead of a single directory sometimes* → appdirs provides a multipath keyword argument which returns a (colon-delimited) “list” of paths. we should provide sth. similar, only with python lists. maybe also convenience functions for getting a list of all files matching some subpath similar to [d / subpath for d in site_data_dir(appname, appauthor, all=True) if (d / subpath).exists()] 3. *some platforms don’t have some specific subset of the functionality (e.g. no config dir on OSX)* → provide a warning in the stdlib docs about that and refer to another settings API. unfortunately i can’t find a good NSDefaults library in python right now. i think the API should still return some directory that works for storing settings on OSX in case people want to use platform-independent config files. 4. *it’s hard to tell newbies where their files are* → unfortunately that’s how things are. the existing standards are confusing, but should be honored nonetheless. we can provide an api like python -m stddirs [--config|--data|...] <companyname> <appname> which prints a selection of standard directories. did i miss anything? and yeah: i also think that something that’s a little opinionated in case of ambiguities is vastly better than everyone hacking their own impromptu platform-dependent alternative. ~/.appname stopped being right on linux long ago and never was right on other platforms, which we should teach people. best, phil Nick Coghlan <ncoghlan@gmail.com> schrieb am Mi., 2. Sep. 2015 um 07:57 Uhr:
On 02.09.2015 10:10, Philipp A. wrote:
~/.appname stopped being right on linux long ago and never was right on other platforms, which we should teach people.
Looking at the my home dir on Linux, there doesn't seem to be one standard, but rather a whole set of them and the good old ~/.appname is still a popular one (e.g. pip and ansible from Python land still use it; as do many other non-Python applications such as ncftp, emacs, svn, git, gpg, etc.). ~/.config/ does get some use, but mostly for GUI applications, not so much for command line ones. ~/.local/lib/ only appears to be used by Python :-) ~/.local/share/ is mostly used by desktops to register application shortcuts ~/.cache/ is being used by just a handful of tools, pip being one of them. appdirs seems to rely on the XDG Base Directory Specification for a lot of things on Linux (http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html). That's probably fine for desktop GUI apps (the standard was apparently built for this use case), but doesn't apply at all for command line tools or applications like daemons or services which don't interact with the desktop, e.g. you typically won't find global config files for command line tools under /etc/xdg/, but instead under /etc/. For Windows, the CSIDL_* values have also been replaced with new ones under FOLDERID_* (the APIs have also evolved): Values: https://msdn.microsoft.com/en-us/library/windows/desktop/bb762494%28v=vs.85%... https://msdn.microsoft.com/en-us/library/windows/desktop/dd378457%28v=vs.85%... APIs: https://msdn.microsoft.com/en-us/library/windows/desktop/bb762181%28v=vs.85%... https://msdn.microsoft.com/en-us/library/windows/desktop/bb762188%28v=vs.85%... BTW: I wonder why the Windows functions in appdirs don't use the environment for much easier access to e.g. APPDATA and USERPROFILE. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Sep 02 2015)
2015-08-27: Released eGenix mx Base 3.2.9 ... http://egenix.com/go83 ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
I just commented on the ticket that references this discussion: http://bugs.python.org/issue7175 In essence, Python already has an installation scheme which is defined in sysconfig.py (see the top of the file) and has had this ever since distutils got added to the stdlib. It just lacks explicit entries for "config" and "cache" files, so adding those would be more in line with coming up with yet another standard, e.g. for posix_user: 'posix_user': { 'stdlib': '{userbase}/lib/python{py_version_short}', 'platstdlib': '{userbase}/lib/python{py_version_short}', 'purelib': '{userbase}/lib/python{py_version_short}/site-packages', 'platlib': '{userbase}/lib/python{py_version_short}/site-packages', 'include': '{userbase}/include/python{py_version_short}', 'scripts': '{userbase}/bin', 'config': '{userbase}/etc', 'cache': '{userbase}/var', 'data': '{userbase}', }, ({userbase} is set by looking at PYTHONUSERBASE and defaults to ~/.local/) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Sep 02 2015)
2015-08-27: Released eGenix mx Base 3.2.9 ... http://egenix.com/go83 ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
"BTW: I wonder why the Windows functions in appdirs don't use the environment for much easier access to e.g. APPDATA and USERPROFILE." The environment can become corrupted more easily and it's really difficult to diagnose that from a bug report that says "my config is corrupt". I assume appdirs using ctypes now, but I'd be happy to add the call into the os module to avoid that. Cheers, Steve Top-posted from my Windows Phone -----Original Message----- From: "M.-A. Lemburg" <mal@egenix.com> Sent: 9/2/2015 2:31 To: "Philipp A." <flying-sheep@web.de>; "Nick Coghlan" <ncoghlan@gmail.com>; "Robert Collins" <robertc@robertcollins.net> Cc: "Nathaniel Smith" <njs@vorpus.org>; "python-ideas@python.org" <python-ideas@python.org> Subject: Re: [Python-ideas] Add appdirs module to stdlib On 02.09.2015 10:10, Philipp A. wrote:
~/.appname stopped being right on linux long ago and never was right on other platforms, which we should teach people.
Looking at the my home dir on Linux, there doesn't seem to be one standard, but rather a whole set of them and the good old ~/.appname is still a popular one (e.g. pip and ansible from Python land still use it; as do many other non-Python applications such as ncftp, emacs, svn, git, gpg, etc.). ~/.config/ does get some use, but mostly for GUI applications, not so much for command line ones. ~/.local/lib/ only appears to be used by Python :-) ~/.local/share/ is mostly used by desktops to register application shortcuts ~/.cache/ is being used by just a handful of tools, pip being one of them. appdirs seems to rely on the XDG Base Directory Specification for a lot of things on Linux (http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html). That's probably fine for desktop GUI apps (the standard was apparently built for this use case), but doesn't apply at all for command line tools or applications like daemons or services which don't interact with the desktop, e.g. you typically won't find global config files for command line tools under /etc/xdg/, but instead under /etc/. For Windows, the CSIDL_* values have also been replaced with new ones under FOLDERID_* (the APIs have also evolved): Values: https://msdn.microsoft.com/en-us/library/windows/desktop/bb762494%28v=vs.85%... https://msdn.microsoft.com/en-us/library/windows/desktop/dd378457%28v=vs.85%... APIs: https://msdn.microsoft.com/en-us/library/windows/desktop/bb762181%28v=vs.85%... https://msdn.microsoft.com/en-us/library/windows/desktop/bb762188%28v=vs.85%... BTW: I wonder why the Windows functions in appdirs don't use the environment for much easier access to e.g. APPDATA and USERPROFILE. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Sep 02 2015)
2015-08-27: Released eGenix mx Base 3.2.9 ... http://egenix.com/go83 ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
hi marc-andre, you seem some misconceptions and a *very* different experience from mine: M.-A. Lemburg mal@egenix.com <http://mailto:mal@egenix.com> schrieb am Mi., 2. Sep. 2015 um 11:30 Uhr: Looking at the my home dir on Linux, there doesn't seem to be
one standard, but rather a whole set of them
can you please link to the document from a standards authority describing those other stadards? 😉 and the good old ~/.appname is still a popular one (e.g. pip and ansible
as hinted at by my tongue-in-cheek comment from above: that’s not a standard but an old convention. git uses ${XDG_CONFIG_DIR-$HOME/.config}/git/config (try it!), as does pip. the other ones are old as dirt so this is excusable. regarding newly developed programs i’ll only approve it for some rare exceptions like shells, where ~/.${SHELL}rc really is the only expected place for a config file. (and not that me approving it means something) ~/.config/ does get some use, but mostly for GUI applications,
not so much for command line ones.
where do you get this figure from? the xdg standard doesn’t say anything about only a kind of application being targeted by the standard, and as my fontconfig example shows, even libraries follow it. ~/.local/lib/ only appears to be used by Python :-)
yeah, on my system it doesn’t even exist! but that has a reson which you can read in the standard: only ~/.local/share is pointed at by the default for a standard dir. ~/.local/lib is probably an invention by whoever included that path in the default $PYTHONPATH ~/.local/share/ is mostly used by desktops to register application shortcuts
that’s a big understatement, there’s all kinds of stuff in there. i have 56 dirs and files in there. ~/.cache/ is being used by just a handful of tools, pip being one of them.
hah! various parts of KDE, matplotlib, gstreamer, chromium, fontconfig, atom, shall i go on? --
Marc-Andre Lemburg eGenix.com
i hope i could convince you that this isn’t “some contender among many”, but really *the* standard for some directories on linux. best, phil
<random832@fastmail.us> schrieb am Mi., 2. Sep. 2015 um 15:36 Uhr:
no idea why, but once we start defining our own API, this will be the most important cange from appdirs. maybe they use semicolons on windows, but i really don’t see why we shouldn’t just use python lists. best, philipp
~/.appname stopped being right on linux long ago and never was right on other platforms, which we should teach people.
Ah, yes. I count 17 of those on my Windows machine (!) right now, including .idlerc, .ipython, .matplotlib, .ipylint.d etc., so we've got a ways to go. :)
On 2 September 2015 at 05:38, Robert Collins <robertc@robertcollins.net> wrote:
Most of the programs I write are for Python 3.4 at the moment, and will be for Python 3.5 as soon as it comes out. They won't be distributed outside of my group at work, if indeed anyone but me uses them. They won't care about older versions of Python. I don't even care about cross-platform. But I do want to set up a cache directory, or save some settings, without thinking *too* hard about where to put them. I don't want to have an external dependency because I'm forever running these things from whatever virtualenv I currently have active (sloppy work habits, I know, but that's the point - not everything is a nicely structured development project). For programs that need to support older versions of Python, a backport should be trivial - I can't see that this would need any particularly "modern" Python features. Paul
On Wed, Sep 02, 2015 at 04:38:05PM +1200, Robert Collins wrote:
I can think of two reasons, one minor, one major: 1. Many people behind corporate or school firewalls cannot just "pip install this". By "firewall" I'm talking more figuratively than literally. Of course there may be an actual firewall blocking access to PyPI, but that's typically very easy to bypass (download the library at home and bring it in on a USB stick). Less easy to bypass is corporate/ school policy which prohibits the installation of unapproved software, often being a firing or expulsion offense. Getting approval may be difficult, slow or downright impossible. However, what makes this a minor reason is that software written under those conditions probably won't be distributed outside of the organisation itself, so who cares whether it complies with the standard locations for application data? 2. More importantly is the stdlib itself. As someone has pointed out, the stdlib already drops config files in completely inappropriate locations on Windows, e.g. $HOME/.idlelib. It would be good if the stdlib itself could consistently use the standard locations for files. -- Steve
participants (12)
-
Andrew Barnert -
Chris Angelico -
Donald Stufft -
Eric Fahlgren -
M.-A. Lemburg -
Nick Coghlan -
Paul Moore -
Philipp A. -
random832＠fastmail.us -
Robert Collins -
Steve Dower -
Steven D'Aprano