"setuptools has divided the Python community"
On Thu, Mar 26, 2009 at 2:37 PM, Barry Warsaw <barry@python.org> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Mar 26, 2009, at 2:31 PM, Guido van Rossum wrote:
One thing that /would/ be helpful though is the ability to list all the resources under a specific package path. This is (I think) one use case that pkg_resource fails to support and it's the one place that I've had to drop down to file system introspection.
Really ... ? What are all these for ? {{{
[x for x in dir(pkg_resources) if all(y in x for y in ['dir', 'resource_'])] ['resource_isdir', 'resource_listdir']
}}} -- Regards, Olemis. Blog ES: http://simelo-es.blogspot.com/ Blog EN: http://simelo-en.blogspot.com/ Featured article: Comandos : Pipe Viewer ... ¿Qué está pasando por esta tubería?
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 26, 2009, at 2:43 PM, Olemis Lang wrote:
One thing that /would/ be helpful though is the ability to list all the resources under a specific package path. This is (I think) one use case that pkg_resource fails to support and it's the one place that I've had to drop down to file system introspection.
Really ... ? What are all these for ?
E.g. find all the doctests under foo.bar.docs Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBScvc9XEjvBPtnXfVAQJT8AP/QhT6Pm4xEL+Bas96cD9waGOTcy1NIYoj kiWGGbp3VGlXrEaRqDx0f0J51IIQ+VZJMu2M2tMKGMXX4ePrLoJsEavGduN34jat KvBN43ZbIlgOFcojSmz9pSjGSUw9T7URHUavnS+psz0SipYceWbCghpML/ORtZml i+bVUwPvRFg= =9k5E -----END PGP SIGNATURE-----
On Thu, Mar 26, 2009 at 2:52 PM, Barry Warsaw <barry@python.org> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Mar 26, 2009, at 2:43 PM, Olemis Lang wrote:
One thing that /would/ be helpful though is the ability to list all the resources under a specific package path. This is (I think) one use case that pkg_resource fails to support and it's the one place that I've had to drop down to file system introspection.
Really ... ? What are all these for ?
E.g. find all the doctests under foo.bar.docs
Could you explain this a little more pls ? You mean ... using doctest finders in a module (even inside a zip file ;) that's already loaded ? Sure ... else I dont understand ... -- Regards, Olemis. Blog ES: http://simelo-es.blogspot.com/ Blog EN: http://simelo-en.blogspot.com/ Featured article: Comandos : Pipe Viewer ... ¿Qué está pasando por esta tubería?
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 26, 2009, at 3:07 PM, Olemis Lang wrote:
On Thu, Mar 26, 2009 at 2:52 PM, Barry Warsaw <barry@python.org> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Mar 26, 2009, at 2:43 PM, Olemis Lang wrote:
One thing that /would/ be helpful though is the ability to list all the resources under a specific package path. This is (I think) one use case that pkg_resource fails to support and it's the one place that I've had to drop down to file system introspection.
Really ... ? What are all these for ?
E.g. find all the doctests under foo.bar.docs
Could you explain this a little more pls ? You mean ... using doctest finders in a module (even inside a zip file ;) that's already loaded ?
Sure ... else I dont understand ...
Here's a concrete example. I have a bunch of .sql files inside the mailman.database package. Their file names encode a sort order. I want to load each one in order and apply them to my database. I'd like to be able to find all these .sql files (which are intermixed with .py files in the same package) without having to use os.listdir() because that forces the package to live on the file system. Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBScv4SnEjvBPtnXfVAQIVXgP/eBYSmfYHkebECPVDmCpewZVC9ydi4WdI agKW41CKPh6PwFFSq35u41KjXEIBIvAcVM9wbYy3zUieTdY+RVe2ceQM/fGILzny bsd2uLuNHMh0af2fen8JaY1DIbKfKrO3Ea4puvr7Uw1EGjN5IShA0WSgWpO5Ggs7 OKkhmXlPe0A= =sLWb -----END PGP SIGNATURE-----
On Thu, Mar 26, 2009 at 4:48 PM, Barry Warsaw <barry@python.org> wrote:
On Mar 26, 2009, at 3:07 PM, Olemis Lang wrote:
On Thu, Mar 26, 2009 at 2:52 PM, Barry Warsaw <barry@python.org> wrote:
On Mar 26, 2009, at 2:43 PM, Olemis Lang wrote:
> > One thing that /would/ be helpful though is the ability to list all > the > resources under a specific package path. This is (I think) one use > case > that pkg_resource fails to support and it's the one place that I've > had > to > drop down to file system introspection. >
Really ... ? What are all these for ?
E.g. find all the doctests under foo.bar.docs
Could you explain this a little more pls ? You mean ... using doctest finders in a module (even inside a zip file ;) that's already loaded ?
Sure ... else I dont understand ...
Here's a concrete example. I have a bunch of .sql files inside the mailman.database package. Their file names encode a sort order. I want to load each one in order and apply them to my database. I'd like to be able to find all these .sql files (which are intermixed with .py files in the same package) without having to use os.listdir() because that forces the package to live on the file system.
Ok ... I will assume the following scenario : - You have many .py together ... so you distribute them in a mailman.database package (not module ... but it should be almost the same thing ...) and provide setup.py for this ... therefore you should have something like this in the later file {{{ #!python try: from setuptools import setup except ImportError: from distutils.core import setup # Many incredibly useful statements ... setup( name='mailman.database', package_dir = { 'mailman.database' : './db', }, packages= ['mailman.database'], package_data={ 'mailman.database': ['sql/*', 'messages/es/LC_MESSAGES/*'] }, **many_incredibly_useful_args ) }}} - Provided you have done this ... I will assume that u need to execute such scripts at run-time, and not at install-time ... else setup.py is more than enough ... ;) {{{ #!python from pkg_resources import * for fnm in sorted(resource_listdir('mailman.database', 'sql'), \ my_own_cmp ): # Only if needed ... ;) exec_script(resource_string('mailman.database', 'sql/' + fnm)) }}} ... perhaps your specific example is a little bit more complex than that ... but the procedure should be similar to the one ci-dessus ... :) ResourceManager instances handle all the details involved disregarding whether the pkg is in a zip file, an egg file or FS ... ;) CMIIW anyway ... ;) -- Regards, Olemis. Blog ES: http://simelo-es.blogspot.com/ Blog EN: http://simelo-en.blogspot.com/ Featured article: Comandos : Pipe Viewer ... ¿Qué está pasando por esta tubería?
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 27, 2009, at 7:51 AM, Olemis Lang wrote:
from pkg_resources import *
for fnm in sorted(resource_listdir('mailman.database', 'sql'), \ my_own_cmp ): # Only if needed ... ;)
Thanks, it was pkg_resource.resource_listdir() that I was missing. Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSc1B6HEjvBPtnXfVAQI39wP/VZ4m7x8hd6HzI4VFetFwe/zgLxIJO5Wt adpQEv/hou+JVgRxSaa0mNAzZSVzDw8pDa2aZ0RXeM/fEaSnhX2IUh+28gNvhyMZ m+57LF5GZMxJsV2fYNj6ELFKeC/JXCQYB43Fxs7AShzlP/1LRnrN9LntQzFUp6hV /GddDd/vOhw= =FN5H -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 26, 2009, at 2:43 PM, Olemis Lang wrote:
{{{
[x for x in dir(pkg_resources) if all(y in x for y in ['dir', 'resource_'])] ['resource_isdir', 'resource_listdir']
BTW, under a better name, I would support putting pkg_resources in the stdlib. Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBScvdH3EjvBPtnXfVAQKDvQP7B8/POejfswQ8A+Xl4+XJQiO8xHpyPUbC wpja12gXhamSvBxgoKnFCeTzMgdEoaeK838CR4yvn9YBM7hzJWW0seODG7JpmxGp w9RXDaf1BuFuxdtTZ8qsoNzRtFQ0LOEh5HtnHYVcpwDHg9FMOKu4cNX19BpvrerU xwx60X5lsfg= =mKi4 -----END PGP SIGNATURE-----
On Thu, Mar 26, 2009 at 2:53 PM, Barry Warsaw <barry@python.org> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Mar 26, 2009, at 2:43 PM, Olemis Lang wrote:
{{{
[x for x in dir(pkg_resources) if all(y in x for y in ['dir', 'resource_'])]
['resource_isdir', 'resource_listdir']
BTW, under a better name, I would support putting pkg_resources in the stdlib.
... or a subset of it ? or integrating its features with PEP 302 ? ... Sounds interesting to me ... /me thinking aloud ... perhaps unifying these features with loaders in standard PEP 302 is the way ... :-§ ... dont see the point of having an isolated module for doing what loaders should do ... including that there are plans to do it already ... thereby duplicating functionality in loaders and ResourceManager, and so on ... And'u ? What d'u think ? -- Regards, Olemis. Blog ES: http://simelo-es.blogspot.com/ Blog EN: http://simelo-en.blogspot.com/ Featured article: Comandos : Pipe Viewer ... ¿Qué está pasando por esta tubería?
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 26, 2009, at 3:14 PM, Olemis Lang wrote:
On Thu, Mar 26, 2009 at 2:53 PM, Barry Warsaw <barry@python.org> wrote:
BTW, under a better name, I would support putting pkg_resources in the stdlib.
... or a subset of it ? or integrating its features with PEP 302 ? ... Sounds interesting to me ...
/me thinking aloud
... perhaps unifying these features with loaders in standard PEP 302 is the way ... :-§ ... dont see the point of having an isolated module for doing what loaders should do ... including that there are plans to do it already ... thereby duplicating functionality in loaders and ResourceManager, and so on ...
And'u ? What d'u think ?
Let me clarify my position: I just want the functionality (preferably in the stdlib); I don't really care how it's spelled (except please not pkg_resource.whatever() :). Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBScv4o3EjvBPtnXfVAQLDQQQAsW6so3MMUgGQHndEAkESZSIFNlyTPPtr ZfAtRcS3y1v+f6qrab7hxY+GTNKhAJnnWEtKbDwzG5NcFMttCKAJQyFr7HL1tvAA 9dbbskN3qjFtjTQ5tz/DJb87pWtFfuisWGp6DWxaUKCVOYWohXxpE/KyGj8KXjOK d+YAPJeLtDM= =Lg0j -----END PGP SIGNATURE-----
2009/3/26 Barry Warsaw <barry@python.org>:
Let me clarify my position: I just want the functionality (preferably in the stdlib); I don't really care how it's spelled (except please not pkg_resource.whatever() :).
Agreed. My one major reservation is that conceptually, the whole pkg_resource infrastructure seems stunningly complex (at this point PJE pops up to point out that it's not, really, and I'm confusing pkg_resource with setuptools, or eggs, or some such - let's just say it makes my head hurt, and defer judgement on whether that's a comment about pkg_resource or my head :-)) What I'd really like is essentially some form of "virtual filesystem" access to stuff addressed relative to a Python package name, rather than to a filesystem path. How much filesystem functionality is required is somewhat up for debate, but the basic listdir, get (text/binary) contents, open (text/binary) stream would be a good start (as Guido pointed out). I know of one or two packages that might benefit from write access (clearly not available for all loaders - zipfiles and URLs couldn't do it, but the filesystem and something like a database could...) But that's just details - the principle still applies. Paul.
On Thu, Mar 26, 2009 at 6:27 PM, Paul Moore <p.f.moore@gmail.com> wrote:
2009/3/26 Barry Warsaw <barry@python.org>:
Let me clarify my position: I just want the functionality (preferably in the stdlib); I don't really care how it's spelled (except please not pkg_resource.whatever() :).
Agreed.
+1 -- Regards, Olemis. Blog ES: http://simelo-es.blogspot.com/ Blog EN: http://simelo-en.blogspot.com/ Featured article: Comandos : Pipe Viewer ... ¿Qué está pasando por esta tubería?
2009/3/26 Barry Warsaw <barry@python.org>:
BTW, under a better name, I would support putting pkg_resources in the stdlib.
Last time I looked it was an incredibly complicated piece of code that would have to be refactored considerably before it would be maintainable by the core developers. I never did manage to get a good understanding of the code, but I expect that a lot of the complexity exists so that it works for all Python versions. The stdlib version shouldn't need this -- it should only care about providing a stable API that works with the current version. -- --Guido van Rossum (home page: http://www.python.org/~guido/)
participants (4)
-
Barry Warsaw -
Guido van Rossum -
Olemis Lang -
Paul Moore