Splitting distribute in several distributions
Hey, I've started to work on splitting Distribute into several distributions for 0.7 Here's the organization I was thinking about: - Distribution Name = "Distribute" - Content: setuptools package renamed to distribute (minus some elements). root modules in setuptools will be moved to the distribute.core package so the root namespace stays empty. - Distribution Name = "DistributeResources" - Content: the part of pkg_resources.py that handles installed distribution querying + renamed to "distribute.resources" - Distribution Name = "DistributeEntryPoints" - Content: the part of pkg_resources.py that handles entry points + renamed to "distribute.entry_points" - Distribution Name = "DistributeInstaller" - Content: the part of pkg_resources.py that handles version comparison + package_index, that gets files at pypi + easy_install.py + easy_install command extracted from setuptools package + renamed into a "distribute.installer" package I am not sure yet how the distribute namespace will be handled, because (maybe ala PEP 382) I am not sure either if "setuptools.sandbox" should get its own distribution. I'd keep it in Distribute Any thoughts ? Cheers Tarek -- Tarek Ziadé | http://ziade.org
I am glad that you have started thinking about the module split. 1) Why are the distributions named in CamelCase? Why not 'distribute.resources' instead of 'DistributeResources' (like, for instance, zope.interface instead of ZopeInterface). 2) In your proposal below, version comparison is part of distribute.installer; this means distribute.resources (and other modules needing version comparison) will be depending on distribute.installer, correct? Is this intended, or should version comparison go to something like distribute.core? 3) PyPM's backend uses a) pkg_resources' version comparison, b) package_index's download logic (not API-friendly). I'd be interested to see distribute.installer provide this download logic (finding URLs, tarballs and fetching them) as an API. I believe pip and zc.buildout too relies on this download logic. 4) How about distribute.pypi that interfaces with pypi? -srid On Tue, 11 Aug 2009 15:30:47 -0700, Tarek Ziadé <ziade.tarek@gmail.com> wrote:
Hey,
I've started to work on splitting Distribute into several distributions for 0.7
Here's the organization I was thinking about:
- Distribution Name = "Distribute" - Content: setuptools package renamed to distribute (minus some elements). root modules in setuptools will be moved to the distribute.core package so the root namespace stays empty.
- Distribution Name = "DistributeResources" - Content: the part of pkg_resources.py that handles installed distribution querying + renamed to "distribute.resources"
- Distribution Name = "DistributeEntryPoints" - Content: the part of pkg_resources.py that handles entry points + renamed to "distribute.entry_points"
- Distribution Name = "DistributeInstaller" - Content: the part of pkg_resources.py that handles version comparison + package_index, that gets files at pypi + easy_install.py + easy_install command extracted from setuptools package + renamed into a "distribute.installer" package
I am not sure yet how the distribute namespace will be handled, because (maybe ala PEP 382)
I am not sure either if "setuptools.sandbox" should get its own distribution. I'd keep it in Distribute
Any thoughts ?
Cheers Tarek
2009/8/12 Sridhar Ratnakumar <SridharR@activestate.com>:
I am glad that you have started thinking about the module split.
1) Why are the distributions named in CamelCase? Why not 'distribute.resources' instead of 'DistributeResources' (like, for instance, zope.interface instead of ZopeInterface).
Because the distribution name is not the packages names. See for instance the Paste project : PasteScript, Paste, PasteDeploy.
2) In your proposal below, version comparison is part of distribute.installer; this means distribute.resources (and other modules needing version comparison) will be depending on distribute.installer, correct? Is this intended, or should version comparison go to something like distribute.core?
When would distribute.resources need version comparisons if its sole role is to provide query APIs for installed packages ? (pre-PEP 376, then PEP 376-compliant)
3) PyPM's backend uses a) pkg_resources' version comparison, b) package_index's download logic (not API-friendly). I'd be interested to see distribute.installer provide this download logic (finding URLs, tarballs and fetching them) as an API. I believe pip and zc.buildout too relies on this download logic.
pakage_index would be included in distribute.installer, and we could probably add simpler APIs to use it to drive a download
4) How about distribute.pypi that interfaces with pypi?
So, like a subpart of package_index ? this is a nice idea, so tools like yolk or z3c.pypimirror could browse pypi index with it, It could also have a full xmlrpc client implementation for PyPI xml-rpc APIs,
-srid
On Tue, 11 Aug 2009 15:30:47 -0700, Tarek Ziadé <ziade.tarek@gmail.com> wrote:
Hey,
I've started to work on splitting Distribute into several distributions for 0.7
Here's the organization I was thinking about:
- Distribution Name = "Distribute" - Content: setuptools package renamed to distribute (minus some elements). root modules in setuptools will be moved to the distribute.core package so the root namespace stays empty.
- Distribution Name = "DistributeResources" - Content: the part of pkg_resources.py that handles installed distribution querying + renamed to "distribute.resources"
- Distribution Name = "DistributeEntryPoints" - Content: the part of pkg_resources.py that handles entry points + renamed to "distribute.entry_points"
- Distribution Name = "DistributeInstaller" - Content: the part of pkg_resources.py that handles version comparison + package_index, that gets files at pypi + easy_install.py + easy_install command extracted from setuptools package + renamed into a "distribute.installer" package
I am not sure yet how the distribute namespace will be handled, because (maybe ala PEP 382)
I am not sure either if "setuptools.sandbox" should get its own distribution. I'd keep it in Distribute
Any thoughts ?
Cheers Tarek
-- Tarek Ziadé | http://ziade.org
On Tue, 11 Aug 2009 17:44:01 -0700, Tarek Ziadé <ziade.tarek@gmail.com> wrote:
2009/8/12 Sridhar Ratnakumar <SridharR@activestate.com>:
I am glad that you have started thinking about the module split.
1) Why are the distributions named in CamelCase? Why not 'distribute.resources' instead of 'DistributeResources' (like, for instance, zope.interface instead of ZopeInterface). Because the distribution name is not the packages names. See for instance the Paste project : PasteScript, Paste, PasteDeploy.
Ok. So there are apparently two conventions for naming distributions. However, zope.*, zc.*, z3c.* fall under more wide-namespaces (zope is a large namespace with many sub-packages).
2) In your proposal below, version comparison is part of distribute.installer; this means distribute.resources (and other modules needing version comparison) will be depending on distribute.installer, correct? Is this intended, or should version comparison go to something like distribute.core? When would distribute.resources need version comparisons if its sole role is to provide query APIs for installed packages ? (pre-PEP 376, then PEP 376-compliant)
When multiple versions of a same package are installed, then perhaps distribute.resources might have to return them in sorted order .. but, well, I am not aware about the querying API yet .. so cannot say for sure. Further, if you implement distribute.pypi .. then that too would require version comparison, would it not? In that case, distribute.pypi would depend upon distribute.installer .. and vice versa, I suppose, leading to circular dependencies. Eg: distribute.pypi.Distribution.find('PasteScript').get_releases() may have to return the releases in sorted order (by version number). There must be a reason why PJE kept parts of setuptools in pkg_resources.py. What is it? We may have to consider it before splitting pkg_resources.py itself.
3) PyPM's backend uses a) pkg_resources' version comparison, b) package_index's download logic (not API-friendly). I'd be interested to see distribute.installer provide this download logic (finding URLs, tarballs and fetching them) as an API. I believe pip and zc.buildout too relies on this download logic. pakage_index would be included in distribute.installer, and we could probably add simpler APIs to use it to drive a download
Note that package_index is not API-friendly. A few violations I can think of: 1) it prints warning messages to stderr without programmer control, 2) throws random network exceptions that should be handled in a custom exception (see the issues I reported in distribute tracker) 3) no ability to control URL types (eg: I want to fetch only tarballs/zipfiles, not svn/hg/etc..) and so on. This is roughly the code I currently use in PyPM (that uses setuptools): (...) class PackageIndex2(PackageIndex): def _download_svn(self, url, filename): raise SVNDisabled, 'svn not allowed at the moment' (...) def download_latest_sdist(req, directory): (...) # TODO: we need timeout on network operations (not just socket timeouts) try: sdist = pi.fetch_distribution(req, directory, source=True) except (DistutilsError, httplib.BadStatusLine, socket.timeout), e: # catch BadStatusLine: http://bitbucket.org/tarek/distribute/issue/18/ raise PackageError, e if sdist == None: raise SdistMissing, 'cannot find a sdist for %s' % req if not exists(sdist.location): raise SdistLocationMissing, 'downloaded sdist <%s> missing' % sdist.location return sdist -srid
On Wed, Aug 12, 2009 at 3:00 AM, Sridhar Ratnakumar<SridharR@activestate.com> wrote:
On Tue, 11 Aug 2009 17:44:01 -0700, Tarek Ziadé <ziade.tarek@gmail.com> wrote:
2009/8/12 Sridhar Ratnakumar <SridharR@activestate.com>:
I am glad that you have started thinking about the module split.
1) Why are the distributions named in CamelCase? Why not 'distribute.resources' instead of 'DistributeResources' (like, for instance, zope.interface instead of ZopeInterface).
Because the distribution name is not the packages names. See for instance the Paste project : PasteScript, Paste, PasteDeploy.
Ok. So there are apparently two conventions for naming distributions. However, zope.*, zc.*, z3c.* fall under more wide-namespaces (zope is a large namespace with many sub-packages).
Well, when Zope and Plone got splitted in multiple distributions, the convention was : distribution name == package name because every distribution has one single package. The Zope community eventually followed, for the worse and the better (sometimes it's better to distribute several packages in one distribution, or to have modules rathers than packages)
2) In your proposal below, version comparison is part of distribute.installer; this means distribute.resources (and other modules needing version comparison) will be depending on distribute.installer, correct? Is this intended, or should version comparison go to something like distribute.core?
When would distribute.resources need version comparisons if its sole role is to provide query APIs for installed packages ? (pre-PEP 376, then PEP 376-compliant)
When multiple versions of a same package are installed, then perhaps distribute.resources might have to return them in sorted order .. but, well, I am not aware about the querying API yet .. so cannot say for sure.
Further, if you implement distribute.pypi .. then that too would require version comparison, would it not? In that case, distribute.pypi would depend upon distribute.installer .. and vice versa, I suppose, leading to circular dependencies. Eg:
I am not sure because the only use case I have in mind is comparing versions when you install, update or check installed distributions. What I propose is to leave the version stuff in installer, and as soon as we realize it's needed elsewhere (that will happen if it's the case before 0.7 is out), we split it on its own.
distribute.pypi.Distribution.find('PasteScript').get_releases() may have to return the releases in sorted order (by version number).
There must be a reason why PJE kept parts of setuptools in pkg_resources.py. What is it? We may have to consider it before splitting pkg_resources.py itself.
Sure. I would tend to think it has grown too quickly.
3) PyPM's backend uses a) pkg_resources' version comparison, b) package_index's download logic (not API-friendly). I'd be interested to see distribute.installer provide this download logic (finding URLs, tarballs and fetching them) as an API. I believe pip and zc.buildout too relies on this download logic.
pakage_index would be included in distribute.installer, and we could probably add simpler APIs to use it to drive a download
Note that package_index is not API-friendly. A few violations I can think of: 1) it prints warning messages to stderr without programmer control, 2) throws random network exceptions that should be handled in a custom exception (see the issues I reported in distribute tracker) 3) no ability to control URL types (eg: I want to fetch only tarballs/zipfiles, not svn/hg/etc..) and so on.
Yes there are a lot of stuff to improve there, Cheers Tarek -- Tarek Ziadé | http://ziade.org
On Aug 12, 2009, at 4:35 AM, Tarek Ziadé wrote:
I am not sure because the only use case I have in mind is comparing versions when you install, update or check installed distributions. What I propose is to leave the version stuff in installer, and as soon as we realize it's needed elsewhere (that will happen if it's the case before 0.7 is out), we split it on its own.
One feature I would love to see make standard is the ability to import a package and check its __version__. Version information must be available from Python, and it seems like __version__ is the standard place to put it. Too many times I've had to figure out the version of foo.bar that I'm using. When I can do
import foo.bar foo.bar.__version__ 3.2.1
I am a much happier camper. Right now this is largely left to the library developer, but I think distribute has some influence here. People want to describe version numbers in only one place and if that's in the metadata, they'll tend not to also include it in the package namespace. In that case, providing for a recommended standard way of doing it is essential. -Barry
On Wed, Aug 12, 2009 at 2:18 PM, Barry Warsaw<barry@python.org> wrote:
On Aug 12, 2009, at 4:35 AM, Tarek Ziadé wrote:
One feature I would love to see make standard is the ability to import a package and check its __version__. Version information must be available from Python, and it seems like __version__ is the standard place to put it. Too many times I've had to figure out the version of foo.bar that I'm using. When I can do
import foo.bar foo.bar.__version__ 3.2.1
I am a much happier camper.
The easiest way I could find right now seems to be:
import pkg_resources dist = pkg_resources.get_distribution('foo') dist.version 3.2.1
Which isn't too bad and could be simplified into an even simpler API call. If the package author doesn't set a __version__ attribute in his own code, I'd be hesitant to magically make it available from the accompanying metadata. If it's part of the metadata, an explicit API that makes it clear that you are querying the package metadata seems cleaner to me. Hanno
On Wed, Aug 12, 2009 at 2:31 PM, Hanno Schlichting<hanno@hannosch.eu> wrote:
On Wed, Aug 12, 2009 at 2:18 PM, Barry Warsaw<barry@python.org> wrote:
On Aug 12, 2009, at 4:35 AM, Tarek Ziadé wrote:
One feature I would love to see make standard is the ability to import a package and check its __version__. Version information must be available from Python, and it seems like __version__ is the standard place to put it. Too many times I've had to figure out the version of foo.bar that I'm using. When I can do
import foo.bar foo.bar.__version__ 3.2.1
I am a much happier camper.
The easiest way I could find right now seems to be:
import pkg_resources dist = pkg_resources.get_distribution('foo') dist.version 3.2.1
Which isn't too bad and could be simplified into an even simpler API call.
If the package author doesn't set a __version__ attribute in his own code, I'd be hesitant to magically make it available from the accompanying metadata. If it's part of the metadata, an explicit API that makes it clear that you are querying the package metadata seems cleaner to me.
yes, and with PEP 376 (which I need to work back in asap), it will be similar and we will have that explicit, clean, get_metadata api. Notice that I think it's better to use pkg_resources (and PEP 376) tehcnique because it grabs the version number in the PKG-INFO file which is static, and doesn't forces you to import a module in the execution context just to get the version. You never know what gets executed when you import a module Cheers Tarek
Hanno _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
-- Tarek Ziadé | http://ziade.org
On Aug 12, 2009, at 5:58 PM, Tarek Ziadé wrote:
yes, and with PEP 376 (which I need to work back in asap), it will be similar and we will have that explicit, clean, get_metadata api.
Notice that I think it's better to use pkg_resources (and PEP 376) tehcnique because it grabs the version number in the PKG-INFO file which is static, and doesn't forces you to import a module in the execution context just to get the version. You never know what gets executed when you import a module
Right, but sometimes all you know is the execution context. Do you see any reason why a library author couldn't write this in their __init__.py? import pkg_resources __version__ = pkg_resources.get_distribution('foo.bar').version ? -Barry
On Thu, Aug 13, 2009 at 12:01 AM, Barry Warsaw<barry@python.org> wrote:
On Aug 12, 2009, at 5:58 PM, Tarek Ziadé wrote:
yes, and with PEP 376 (which I need to work back in asap), it will be similar and we will have that explicit, clean, get_metadata api.
Notice that I think it's better to use pkg_resources (and PEP 376) tehcnique because it grabs the version number in the PKG-INFO file which is static, and doesn't forces you to import a module in the execution context just to get the version. You never know what gets executed when you import a module
Right, but sometimes all you know is the execution context.
Do you see any reason why a library author couldn't write this in their __init__.py?
import pkg_resources __version__ = pkg_resources.get_distribution('foo.bar').version
If we make the assumption the module is called only when it's installed in Python, not at all. But then, the real version number has to be somewhere, eg setup.py, which is less nice that having it in __version__ ... :) I think things will look better the day we remove setup.py in favor of a static metadata file. -- Tarek Ziadé | http://ziade.org
On Thu, 13 Aug 2009 00:13:59 +0200, Tarek Ziadé <ziade.tarek@gmail.com> wrote:
I think things will look better the day we remove setup.py in favor of a static metadata file.
It's good to see that you have your progressive thinking hat on Tarek but given the history that surrounds setup.py, I suggest that removing "setup.py" is "unrealistic". You're far better off to add the functionality so that setup.py can pick up it's metadata from an external file at runtime. For example, add the ability for setup.py to interact with a new file called, say "setup.xml". To use the metadata, we'd need a function to load the metadata. Which shouldn't be so hard to do. Perphaps, self.loadmetadata() Avoid the temptation to use a heavyweight xml processor. Surely doing this cannot be that hard....? even I would volunteer to help out with this. Tell me how? David
On Aug 12, 2009, at 7:09 PM, David Lyon wrote:
For example, add the ability for setup.py to interact with a new file called, say "setup.xml".
Oh for Guido's sake, can we please not use XML? I'd much prefer a human-friendly format ;) -Barry
On Wed, 12 Aug 2009 19:21:45 -0400, Barry Warsaw <barry@python.org> wrote:
Oh for Guido's sake, can we please not use XML? I'd much prefer a human-friendly format ;)
xml is a human friendly format providing that it stays 'simple'. The advantage of xml in this instance is that the purpose of the file is immediately obvious to newcomers ie "setup.py + setup.xml". Whereas other extensions are less clear. Choosing another extension and rolling your own format just adds another level of complexity for other people to deal with. Having said that, we already have a .EGG_INFO file with the metadata already in it (a funny twist of fate...) so having a "setup.EGG_INFO" or "setup.INFO" file would seem to me to be the shortest route to accomplishing what we have here... Still... I prefer setup.xml over setup.INFO.. but I could live with either... David
On Thursday 13 August 2009 01:34:21 David Lyon wrote:
On Wed, 12 Aug 2009 19:21:45 -0400, Barry Warsaw <barry@python.org> wrote:
Oh for Guido's sake, can we please not use XML? I'd much prefer a human-friendly format ;)
xml is a human friendly format providing that it stays 'simple'.
The advantage of xml in this instance is that the purpose of the file is immediately obvious to newcomers ie "setup.py + setup.xml". Whereas other extensions are less clear.
what's wrong with setup.json ? that's a tad friendlier than setup.xml. cheers, sebastien. -- ######################################### # Dr. Sebastien Binet # Laboratoire de l'Accelerateur Lineaire # Universite Paris-Sud XI # Batiment 200 # 91898 Orsay #########################################
On Thu, 13 Aug 2009 09:19:05 +0200, Sebastien Binet <seb.binet@gmail.com> wrote:
what's wrong with setup.json ? that's a tad friendlier than setup.xml.
Probably nothing.. So check with Tarek.... He's in charge with distutils.. not me. The very one person that has the authority to actually *fix* the problem is complaining about the problem. If he can't do anything about it - there's certainly no hope for anybody else. It just seems that if it isn't a plone/zope problem, then it won't be looked at or even considered. I would have thought that improving setup.py, as Tarek himself pointed out.. would have been something that he could do. What's happened here, is that all the political baggage of of setuptools (management style) has also been forked...... it would seem... No changes for 9 months... hmmmm.... setup.py has stayed the same for way... way... longer than that... David
Barry Warsaw wrote:
On Aug 12, 2009, at 7:09 PM, David Lyon wrote:
For example, add the ability for setup.py to interact with a new file called, say "setup.xml".
Oh for Guido's sake, can we please not use XML? I'd much prefer a human-friendly format ;)
+10, but I'm +sys.maxint on not using json. We want to use javascript to configure python packages? Wait, what?! It the data doesn't need to be nested, I'd say use ConfigParser format... If it does, then reluctantly xml... Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk
On Sat, Aug 15, 2009 at 11:11 AM, Chris Withers<chris@simplistix.co.uk> wrote:
Barry Warsaw wrote:
On Aug 12, 2009, at 7:09 PM, David Lyon wrote:
For example, add the ability for setup.py to interact with a new file called, say "setup.xml".
Oh for Guido's sake, can we please not use XML? I'd much prefer a human-friendly format ;)
+10, but I'm +sys.maxint on not using json. We want to use javascript to configure python packages? Wait, what?!
It the data doesn't need to be nested, I'd say use ConfigParser format...
+1, even if the information is nested. Note that there are variants of config parser that support nesting and there are techniques for representing nested data in normal config-parser files. Jim -- Jim Fulton
Jim Fulton wrote:
Note that there are variants of config parser that support nesting and there are techniques for representing nested data in normal config-parser files.
...but I've not seen one that doesn't suck ;-) (for example the one at http://docs.python.org/library/logging.html#configuring-logging) I'd be happy to be proved wrong of course... Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk
On Sat, 15 Aug 2009 16:11:52 +0100, Chris Withers <chris@simplistix.co.uk> wrote:
Barry Warsaw wrote:
On Aug 12, 2009, at 7:09 PM, David Lyon wrote:
For example, add the ability for setup.py to interact with a new file called, say "setup.xml".
Not disagreeing - just discussing.. ConfigParser is good.. no debate.. it's more modern than the .egg_info metadata format, and easier than xml.
It the data doesn't need to be nested, I'd say use ConfigParser format... If it does, then reluctantly xml...
We must resist the option to nest (against unnecessary complexity). ConfigParser does lists easily so you can store a list of files extremely easily. It also does free form text (descriptions). One advantage of ConfigParser is that it's more built in to python than xml and just a breeze to edit the data. If somebody needs scripts to adjust version numbers, that's just so easy also. For me, that only leaves the choice of a filename.. - setup.info - setup.conf - others..? Having a setup.py and a setup.info to me seems simple and straightforward. It's easier to understand than having all the metadata 'hardcoded' in a program (as "cute" as that is). I'll try to build some samples and post them on the distutils page during the week. It may only be an hour or two of work to flesh out some improvements. David
David Lyon wrote:
We must resist the option to nest (against unnecessary complexity). ConfigParser does lists easily so you can store a list of files extremely easily.
Cool, what's the format for lists?
It also does free form text (descriptions).
And for this?
For me, that only leaves the choice of a filename..
- setup.info
- setup.conf
- others..?
...is entirely up to the package developer. We can suggest whatever, but since it'll be called from setup.py, we can't do anything other than suggest a sensible default that will work with the tools that will hopefully show up that only parse the static file...
Having a setup.py and a setup.info to me seems simple and straightforward. It's easier to understand than having all the metadata 'hardcoded' in a program (as "cute" as that is).
For some packages (see lxml for example) it's not cute but a necessity to figure out how to build their extensions properly... cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk
On Monday, 17 August, 2009, at 05:09PM, "Chris Withers" <chris@simplistix.co.uk> wrote:
For me, that only leaves the choice of a filename..
- setup.info
- setup.conf
- others..?
...is entirely up to the package developer. We can suggest whatever, but since it'll be called from setup.py, we can't do anything other than suggest a sensible default that will work with the tools that will hopefully show up that only parse the static file...
We already have 'setup.cfg', what's wrong with that? If another filename is needed: please pick one where the suffix is unique, that would make it easier to associate the file with specific programs (double-click to open, spotlight/quicklook helpers on OSX, ...) Ronald
On Mon, 17 Aug 2009 17:23:14 +0200, Ronald Oussoren <ronaldoussoren@mac.com> wrote:
We already have 'setup.cfg', what's wrong with that?
Only that the metadata file needs to be 'transmitted' along-with/inside the package. As such it isn't a config file as such. My apologies for being pedantic, but setup.cfg has other information that you want to keep on your machine.
If another filename is needed: please pick one where the suffix is unique, that would make it easier to associate the file with specific programs (double-click to open, spotlight/quicklook helpers on OSX, ...)
Exactly. David
On Mon, Aug 17, 2009 at 5:23 PM, Ronald Oussoren<ronaldoussoren@mac.com> wrote:
On Monday, 17 August, 2009, at 05:09PM, "Chris Withers" <chris@simplistix.co.uk> wrote:
For me, that only leaves the choice of a filename..
- setup.info
- setup.conf
- others..?
...is entirely up to the package developer. We can suggest whatever, but since it'll be called from setup.py, we can't do anything other than suggest a sensible default that will work with the tools that will hopefully show up that only parse the static file...
We already have 'setup.cfg', what's wrong with that? If another filename is needed: please pick one where the suffix is unique, that would make it easier to associate the file with specific programs (double-click to open, spotlight/quicklook helpers on OSX, ...)
Yes please, have a look at my mail on this (next thread) I don't see why we would have another file
On Mon, 17 Aug 2009 16:09:31 +0100, Chris Withers <chris@simplistix.co.uk> wrote:
We must resist the option to nest (against unnecessary complexity). ConfigParser does lists easily so you can store a list of files extremely easily.
Cool, what's the format for lists?
If there isn't a built in format for lists, you can simulate it by numbering items. Eg: [InstallableFiles] file1 = abc.txt file2 = def.txt ..etc..
It also does free form text (descriptions).
And for this?
From my observations it seems to tabulate the first column. Eg:
[PackageMetadata] description = This package converts popular tv soap operas into something that is readily viewable on your python powered mobile phone.
For me, that only leaves the choice of a filename..
- setup.info
- setup.conf
- others..?
...is entirely up to the package developer. We can suggest whatever, but since it'll be called from setup.py, we can't do anything other than suggest a sensible default that will work with the tools that will hopefully show up that only parse the static file...
My idea was to have the distutils "setup()" read the data from the config file. Defaulting to a particular file with a particular name. Since we have a convention for setup.py, adding the convention for a particular 'setup.info'/'setup.xml'/'package.setup' doesn't seem to me to be a difficult ask. As long as it has 'setup' included somewhere in the filename.
Having a setup.py and a setup.info to me seems simple and straightforward. It's easier to understand than having all the metadata 'hardcoded' in a program (as "cute" as that is).
For some packages (see lxml for example) it's not cute but a necessity to figure out how to build their extensions properly...
ok - I'll check it out. David
Tarek Ziadé wrote:
I think things will look better the day we remove setup.py in favor of a static metadata file.
I concur this is unlikely to ever happen and agree that the best thing to do is writer a helper that lets people get access to their metadata from a declarative file, eg: from distribute import setup ... setup(metadata='setup.ini',**other_options) cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk
Hanno Schlichting wrote:
The easiest way I could find right now seems to be:
import pkg_resources dist = pkg_resources.get_distribution('foo') dist.version 3.2.1
Which isn't too bad and could be simplified into an even simpler API call.
If the package author doesn't set a __version__ attribute in his own code, I'd be hesitant to magically make it available from the accompanying metadata. If it's part of the metadata, an explicit API that makes it clear that you are querying the package metadata seems cleaner to me.
+1 to all of this... Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk
Sridhar Ratnakumar schrieb:
Note that package_index is not API-friendly. A few violations I can think of: 1) it prints warning messages to stderr without programmer control, 2) throws random network exceptions that should be handled in a custom exception (see the issues I reported in distribute tracker) 3) no ability to control URL types (eg: I want to fetch only tarballs/zipfiles, not svn/hg/etc..) and so on.
+1 Also missing: 4) No easy way to change priorities for filetypes (eggs are always preferred) 5) No easy way to fetch other filetypes (eg. .msi or installer-exe, esp. important on Windows) 6) No easy way to change the URL-Scanner (if sourceforge changes URLs again) When distribute evolves, I may contribute some code / patches for this. -- Schönen Gruß - Regards Hartmut Goebel Dipl.-Informatiker (univ.), CISSP, CSSLP Goebel Consult Spezialist für IT-Sicherheit in komplexen Umgebungen http://www.goebel-consult.de Monatliche Kolumne: <http://www.all-about-security.de/kolumnen/cissp-gefluester/> Goebel Consult mit Mitglied bei <http://www.7-it.de>
Hartmut Goebel kirjoitti:
Sridhar Ratnakumar schrieb:
Note that package_index is not API-friendly. A few violations I can think of: 1) it prints warning messages to stderr without programmer control, 2) throws random network exceptions that should be handled in a custom exception (see the issues I reported in distribute tracker) 3) no ability to control URL types (eg: I want to fetch only tarballs/zipfiles, not svn/hg/etc..) and so on.
+1
Also missing: 4) No easy way to change priorities for filetypes (eggs are always preferred) 5) No easy way to fetch other filetypes (eg. .msi or installer-exe, esp. important on Windows) 6) No easy way to change the URL-Scanner (if sourceforge changes URLs again)
When distribute evolves, I may contribute some code / patches for this.
Make sure you add an issue for this in the bug tracker, so it won't be forgotten by the time the code gets an overhaul.
------------------------------------------------------------------------
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Alex Grönholm schrieb:
Make sure you add an issue for this in the bug tracker, so it won't be forgotten by the time the code gets an overhaul.
Done. -- Schönen Gruß - Regards Hartmut Goebel Dipl.-Informatiker (univ.), CISSP, CSSLP Goebel Consult Spezialist für IT-Sicherheit in komplexen Umgebungen http://www.goebel-consult.de Monatliche Kolumne: <http://www.all-about-security.de/kolumnen/cissp-gefluester/> Goebel Consult mit Mitglied bei <http://www.7-it.de>
Tarek Ziadé wrote:
So, like a subpart of package_index ? this is a nice idea, so tools like yolk or z3c.pypimirror could browse pypi index with it,
It could also have a full xmlrpc client implementation for PyPI xml-rpc APIs,
I like this idea too, but why call it distribute.anything? It sounds like it's basically a python client library for pypi, so why not just have both the distribution and package name be pypi? cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk
2009/8/12 Sridhar Ratnakumar <SridharR@activestate.com>:
I am glad that you have started thinking about the module split.
1) Why are the distributions named in CamelCase? Why not 'distribute.resources' instead of 'DistributeResources' (like, for instance, zope.interface instead of ZopeInterface).
+1 for that. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64
Lennart Regebro kirjoitti:
2009/8/12 Sridhar Ratnakumar <SridharR@activestate.com>:
I am glad that you have started thinking about the module split.
1) Why are the distributions named in CamelCase? Why not 'distribute.resources' instead of 'DistributeResources' (like, for instance, zope.interface instead of ZopeInterface).
+1 for that.
Looking at PyPI, there seems to be no clear standard on this, though I would say 60% of distribution names are in lower case. Is there a PEP that makes any recommendations either way?
2009/8/12 Alex Grönholm <alex.gronholm@nextday.fi>:
Lennart Regebro kirjoitti:
2009/8/12 Sridhar Ratnakumar <SridharR@activestate.com>:
I am glad that you have started thinking about the module split.
1) Why are the distributions named in CamelCase? Why not 'distribute.resources' instead of 'DistributeResources' (like, for instance, zope.interface instead of ZopeInterface).
+1 for that.
Looking at PyPI, there seems to be no clear standard on this, though I would say 60% of distribution names are in lower case. Is there a PEP that makes any recommendations either way?
I don't think so but when you pip-install or easy-install a package, PyPI provides an alias for a lower case version of the name, meaning that these two lines are equal: easy_install FooBar == easy_install foobar The problem is, IIRC, the XML-RPC Apis don't provide this case insensivity. Last, why would we have a dot (.) in the distribution name ? Suppose that 'distribute.resources' contains more than the 'distribute.resources' package. Let's say : - distribute.resources (package) - distribute.util (package) - run_me.py (script) How would you call in that case the distribution ? the "one package=one distribution" rule sounds very Java-ish to me. In the meantime, one may consider that the first namespace is the brand or the company rather than the packaged namespace it contains. for the stats at PyPI, when you say 60%, do you count all zope and Plone distributions (whatever the namespace is) as a single count ? Because beside them, I don't think lower, namespaced distribution names are really frequent. I would tend to think that people use the name of the project rather than the namespaces. Cheers Tarek -- Tarek Ziadé | http://ziade.org
On Aug 12, 2009, at 1:51 AM, Alex Grönholm wrote:
Looking at PyPI, there seems to be no clear standard on this, though I would say 60% of distribution names are in lower case. Is there a PEP that makes any recommendations either way?
I don't think so, but clearly PEP 8 recommends package names be lowercase and it just seems easier to name the distribution the same way. Actually, I'm not even sure what "distribution name" means otherwise <wink>. -Barry
Barry Warsaw kirjoitti:
On Aug 12, 2009, at 1:51 AM, Alex Grönholm wrote:
Looking at PyPI, there seems to be no clear standard on this, though I would say 60% of distribution names are in lower case. Is there a PEP that makes any recommendations either way?
I don't think so, but clearly PEP 8 recommends package names be lowercase and it just seems easier to name the distribution the same way. Actually, I'm not even sure what "distribution name" means otherwise <wink>.
I don't see any connection between conventions for package names and conventions for distribution names. As for what "distribution name" means, the distutils documentation explains the difference between distributions and packages: http://docs.python.org/distutils/introduction.html#distutils-specific-termin... Note that all referenced distributions there have camelCase names.
-Barry
Tarek Ziadé wrote:
elements). root modules in setuptools will be moved to the distribute.core package so the root namespace stays empty.
Is this purely to avoid the problem solved by Martin von Lewis's PEP to do with namespace packages?
- Distribution Name = "DistributeResources" - Content: the part of pkg_resources.py that handles installed distribution querying + renamed to "distribute.resources"
- Distribution Name = "DistributeEntryPoints" - Content: the part of pkg_resources.py that handles entry points + renamed to "distribute.entry_points"
- Distribution Name = "DistributeInstaller" - Content: the part of pkg_resources.py that handles version comparison + package_index, that gets files at pypi + easy_install.py + easy_install command extracted from setuptools package + renamed into a "distribute.installer" package
These all raise the question for me, again: What's the difference between a distribution and a package? Which is the one with a setup.py and what is the other one? If a distribution has a setup.py, then I'm not just having the distribution names be the same as the module names and be: distribution_resources distribution_entrypoints (why is this not part of distribution_resources?) distribution_installer If it's package that has the setup.py, then just replace 'distribution' with 'package' in the above. If you're splitting these because they deserve to be independent packages, then why do they need to be tied together under the 'distribute' name? Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk
Chris Withers <chris@simplistix.co.uk> writes:
These all raise the question for me, again: What's the difference between a distribution and a package? Which is the one with a setup.py and what is the other one?
<URL:http://docs.python.org/distutils/introduction.html#general-python-terminolog...> <URL:http://docs.python.org/distutils/introduction.html#distutils-specific-termin...>
If a distribution has a setup.py, then I'm not just having the distribution names be the same as the module names
This policy fails when the distribution includes multiple modules, or multiple packages. -- \ “Simplicity is prerequisite for reliability.” —Edsger W. | `\ Dijkstra | _o__) | Ben Finney
participants (13)
-
Alex Grönholm -
Barry Warsaw -
Ben Finney -
Chris Withers -
David Lyon -
Hanno Schlichting -
Hartmut Goebel -
Jim Fulton -
Lennart Regebro -
Ronald Oussoren -
Sebastien Binet -
Sridhar Ratnakumar -
Tarek Ziadé