Re: [Python-Dev] pkgutil, pkg_resource and Python 3.0 name space packages
At 12:03 PM 1/6/2008 -0700, Steven Bethard wrote:
Maybe the situation is different here, but having someone installing a different version of sqlite behind my back makes me nervous.
Er, someone who? Behind whose back? I'm quite confused by what it is that's making you nervous. Do you worry about people bundling newer versions of say, the optparse module or wsgiref with their applications? If so, why? Or if not, what's different?
On Jan 6, 2008 1:07 PM, Phillip J. Eby <pje@telecommunity.com> wrote:
At 12:03 PM 1/6/2008 -0700, Steven Bethard wrote:
Maybe the situation is different here, but having someone installing a different version of sqlite behind my back makes me nervous.
Er, someone who? Behind whose back? I'm quite confused by what it is that's making you nervous.
Do you worry about people bundling newer versions of say, the optparse module or wsgiref with their applications? If so, why? Or if not, what's different?
I worry about exactly the pyxml problem. Someone installs pyxml on my system, pyxml replaces xml.parsers.expat with a different version of pyexpat than the rest of Python, and then programs like mod_python crash because the two components were compiled against different versions of a common library. Here's the link again that I posted earlier: http://www.dscpl.com.au/wiki/ModPython/Articles/ExpatCausingApacheCrash Note that this all happens "behind my back" because I didn't know that pyxml would be replacing pyexpat in such a way that would cause this crash. In fact, I didn't even know that pyxml was installing pyexpat. Steve -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy
On Jan 6, 2008 2:01 PM, Steven Bethard <steven.bethard@gmail.com> wrote:
On Jan 6, 2008 1:07 PM, Phillip J. Eby <pje@telecommunity.com> wrote:
At 12:03 PM 1/6/2008 -0700, Steven Bethard wrote:
Maybe the situation is different here, but having someone installing a different version of sqlite behind my back makes me nervous.
Er, someone who? Behind whose back? I'm quite confused by what it is that's making you nervous.
Do you worry about people bundling newer versions of say, the optparse module or wsgiref with their applications? If so, why? Or if not, what's different?
I worry about exactly the pyxml problem. Someone installs pyxml on my system, pyxml replaces xml.parsers.expat with a different version of pyexpat than the rest of Python, and then programs like mod_python crash because the two components were compiled against different versions of a common library. Here's the link again that I posted earlier:
http://www.dscpl.com.au/wiki/ModPython/Articles/ExpatCausingApacheCrash
Note that this all happens "behind my back" because I didn't know that pyxml would be replacing pyexpat in such a way that would cause this crash. In fact, I didn't even know that pyxml was installing pyexpat.
This is my worry as well (and why I was going to phrase the situation in a more "should this be considered" tone than the way Christian stated it. =) . But then again none of this is impossible even if we do leave out extending the namespace. Tweaking a package's __path__ value after the fact is not exactly hard. So even if we don't do this we are not preventing anyone from extending the namespace, just discouraging. My question becomes whether we want to allow something like this even if we explicitly state people should not use this mechanism to override pre-existing modules. Do we want people tossing stuff into the 'databases' package, or should the packages in the stdlib be considered sacred? I am leaning towards not, but that's because I personally like knowing that if I import something from a stdlib namespace it is from the stdlib. I don't want some bug report from a naive user of cx_Oracle ending up in the stdlib because the import came from the 'databases' package. And I would guess that if you don't consider this a stdlib thing but for any third-party package that other third-party packages would not love other packages mucking with their namespace. -Brett
On 06/01/2008, Brett Cannon <brett@python.org> wrote:
My question becomes whether we want to allow something like this even if we explicitly state people should not use this mechanism to override pre-existing modules. Do we want people tossing stuff into the 'databases' package, or should the packages in the stdlib be considered sacred? I am leaning towards not, but that's because I personally like knowing that if I import something from a stdlib namespace it is from the stdlib. I don't want some bug report from a naive user of cx_Oracle ending up in the stdlib because the import came from the 'databases' package. And I would guess that if you don't consider this a stdlib thing but for any third-party package that other third-party packages would not love other packages mucking with their namespace.
I see the point, but I'm against reserving generic package names like "databases" for the stdlib, unless 3rd parties can add modules in there. Another example might be "gui.tkinter" - why shouldn't "gui.wx" be allowed? If we want a "guaranteed-stdlib" package form, we should probably have a top-level package, "std" or whatever. That notion has, I believe, been shot down before (no time to look up references now). Paul.
On Sun, Jan 06, 2008 at 11:12:43PM +0000, Paul Moore wrote:
If we want a "guaranteed-stdlib" package form, we should probably have a top-level package, "std" or whatever.
py.
That notion has, I believe, been shot down before (no time to look up references now).
Mr Van Rossum has spoken against it many times. Now I think - if we don't want a separate Python's top-level namespace may be we should think about a separate top-level non-Python's (3rd parties') namespace? With it we could have database.sqlite (Python's sqlite) and user.database.sqlite (a newer version); and by doing import database.sqlite you know exactly what version you are importing. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.
On Jan 6, 2008 3:28 PM, Oleg Broytmann <phd@phd.pp.ru> wrote:
Now I think - if we don't want a separate Python's top-level namespace may be we should think about a separate top-level non-Python's (3rd parties') namespace? With it we could have database.sqlite (Python's sqlite) and user.database.sqlite (a newer version); and by doing import database.sqlite you know exactly what version you are importing.
Bleh. I'm +1 on allowing third-party additions to the same 'database' namespace that the stdlib puts packages in. People will just have to get used to the package, and not the namespace, determining who to complain to. In theory, it might make sense to allow libraries to "close" some namespaces to deal with Brett's worry, but I think the consenting adults rule says not to bother. -- Namasté, Jeffrey Yasskin
On Jan 6, 2008 3:35 PM, Jeffrey Yasskin <jyasskin@gmail.com> wrote:
On Jan 6, 2008 3:28 PM, Oleg Broytmann <phd@phd.pp.ru> wrote:
Now I think - if we don't want a separate Python's top-level namespace may be we should think about a separate top-level non-Python's (3rd parties') namespace? With it we could have database.sqlite (Python's sqlite) and user.database.sqlite (a newer version); and by doing import database.sqlite you know exactly what version you are importing.
Bleh.
I'm +1 on allowing third-party additions to the same 'database' namespace that the stdlib puts packages in. People will just have to get used to the package, and not the namespace, determining who to complain to.
In theory, it might make sense to allow libraries to "close" some namespaces to deal with Brett's worry, but I think the consenting adults rule says not to bother.
There seems to be a misunderstanding. This is *not* going to happen for standard library package names. I'm fine with inventing mechanisms to allow 3rd party packages to beo cobbled together from multiple contributions (it would seem to make sense for e.g. Twisted or Zope). But I am absolutely squarely against 3rd party installs adding to (or worse, overriding) standard library package names. This is (to me) simple common sense, the reason being "knowing who to blame". We get enough bug reports in the Python tracker about 3rd party software as it is. (In general I'm not keen on attempts to create a single unified ontology for library modules. It's like expecting all doctors in town to live on the same street. But my feelings on keeping a clean demarcation between standard and 3rd party are much stronger.) -- --Guido van Rossum (home page: http://www.python.org/~guido/)
On 07/01/2008, Guido van Rossum <guido@python.org> wrote:
There seems to be a misunderstanding. This is *not* going to happen for standard library package names. I'm fine with inventing mechanisms to allow 3rd party packages to beo cobbled together from multiple contributions (it would seem to make sense for e.g. Twisted or Zope). But I am absolutely squarely against 3rd party installs adding to (or worse, overriding) standard library package names. This is (to me) simple common sense, the reason being "knowing who to blame". We get enough bug reports in the Python tracker about 3rd party software as it is.
OK, that seems pretty clear. In which case, I'm -1 on having a "databases" package in the stdlib (this discussion should probably move back to the stdlib reorg list now). Keeping things like sqlite at the top level seems fine to me. I don't have a general rule I'm applying here - I think each suggestion should be considered on its own merits. Paul.
On Jan 6, 2008, at 6:28 PM, Oleg Broytmann wrote:
On Sun, Jan 06, 2008 at 11:12:43PM +0000, Paul Moore wrote:
If we want a "guaranteed-stdlib" package form, we should probably have a top-level package, "std" or whatever.
py.
That notion has, I believe, been shot down before (no time to look up references now).
Mr Van Rossum has spoken against it many times.
Now I think - if we don't want a separate Python's top-level namespace may be we should think about a separate top-level non-Python's (3rd parties') namespace?
I think some things make sense to move some things into a common namespace: hashlib, email, xml et al... I also think there are probably other candidates for similar grouping and cleanup, but I don't have a well thought out set. (db, url??, zip??); and I think it is happening, albeit slowly.
With it we could have database.sqlite (Python's sqlite) and user.database.sqlite (a newer version); and by doing import database.sqlite you know exactly what version you are importing.
At first blush I am +1 for a third party or non stdlib namespace; user is already a module though. Other ideas: sitepkg, addon, extra, ext But then what of packages that are already namespaced? It would be tedious to; from sitepkg.zope.app.interface import SomeThing So while I like the idea, I think I am against wedging everything into namespaces just for the sake of it; at the end of the day I think I am -1 on *requiring* it. Additionally, I have only been reading this list for a week or so and feel a little like I may be injecting a novice and poorly formed opinion into a discussion that has been ongoing for years. Please pardon my ignorance if this is the case. ~ro
Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/reed% 40reedobrien.com
participants (8)
-
Brett Cannon -
Guido van Rossum -
Jeffrey Yasskin -
Oleg Broytmann -
Paul Moore -
Phillip J. Eby -
Reed O'Brien -
Steven Bethard