Mailman 3 August 2005 - Distutils-SIG

Metadata.
by Vincenzo Di Massa 14 Aug '05

14 Aug '05

Hi, Thank you Phillip for your help and support! I would like to access a package metadata before it is installed: I use the following class ####################CLASS class Easy_deb: def __init__(self,req_string): self.requirement = pkg_resources.Requirement.parse(req_string) self.index = PackageIndex() self.found = None self.dest = None self.filename = None self.source_dir = None def download_source(self): self.index.find_packages(self.requirement) self.found=self.index.obtain(self.requirement) if self.found: self.dest='deb-pkg-'+self.found.key+self.found.version if os.path.isdir(self.dest): shutil.rmtree(self.dest) os.makedirs(self.dest) self.filename=self.index.download(self.found.location, self.dest) def unpack(self): unpack_archive(self.filename, self.dest) setups = glob.glob(os.path.join(self.dest, '*', 'setup.py')) self.source_dir = os.path.dirname(setups[0]) def deb_src(self): debian_dir=os.path.join(self.source_dir,'debian') shutil.copytree('template',debian_dir) #file=os.path.join(debian_dir,"control") #print file sed(os.path.join(debian_dir,"control"),"%%PKGNAME%%",self.requirement.key) <------ SED LINE os.chdir(self.source_dir) #inst_dir=os.path.join('./debian','tmp') #main(['-d']+[inst_dir,'.']) def create(self): self.download_source() self.unpack() self.deb_src() ####################END OF CLASS to download modules and create a deb package. I need to retrieve other metadata like description, author and license. I nned it to put it into lines like SED LINE. Thank you again Vincenzo ___________________________________ Yahoo! Mail: gratis 1GB per i messaggi e allegati da 10MB http://mail.yahoo.it

2 1

pkg_resources API reference now available
by Phillip J. Eby 14 Aug '05

14 Aug '05

I've just checked in the completion of a pkg_resources API reference. In case you'd like to see the HTML version, or if SourceForge hasn't updated anonymous CVS yet, you can see it at: http://peak.telecommunity.com/DevCenter/PkgResources Keep in mind that only the API reference is completed, and it turned out to be a much denser, far more interconnected document than I imagined it would be. My plan is to write a short architecture overview and a tutorial/FAQ-style "developer's guide" to go with this, so that most people won't have to slog through the API reference just to learn how to use the basic features. But, I wanted to start by writing the reference section so that I'd know what stuff I could then leave out of those other guides. :) However, if your goal is to extend pkg_resources or setuptools in some way, create new distribution formats, support custom importers, or create package management tools, you will probably benefit greatly from taking the time to read the full API reference. (Especially since the API reference is less text to read than the pkg_resources source code!)

1 0

Dependencies
by Vincenzo Di Massa 13 Aug '05

13 Aug '05

Hi, Are dependencies taken into account inside PyPI records? I mean both build dependencies and run dependencies. If yes where can I find more info? How do setuptools manage it, how can I access this info? Ciao Vincenzo ___________________________________ Yahoo! Mail: gratis 1GB per i messaggi e allegati da 10MB http://mail.yahoo.it

2 3

setuptools presentation
by Ian Bicking 12 Aug '05

12 Aug '05

I made a presentation on setuptools & company last night at the Chicago Python user group. For the interested, the slides: http://ianbicking.org/docs/setuptools-presentation/ source: http://ianbicking.org/docs/setuptools-presentation/setuptools-slides.txt Hit the /O in the corner to get all the HTML in one page. The slides kind of dwindle at the end, because I got tired of collecting stuff. But eh, they're just slides, not documentation. I think it went well. There were about 15 people there. -- Ian Bicking / ianb(a)colorstudy.com / http://blog.ianbicking.org

4 7

setuptools: package management and explicit version numbers
by Ian Bicking 12 Aug '05

12 Aug '05

Has there been any progress on some of the package management tools people were talking about? I.e., list and delete packages? (At the moment no other features come to mind, but I'm definitely seeing a purpose for these functions) Someone said they were going to give it a go, but I can't remember who. Another option that I think would be useful is to modify requirements in a package to make version numbers explicit. So I might have something in development and it requires a bunch of packages. It just requires the Most Recent version of those packages; I haven't tested anything but that, so I don't know if older versions would work, and I can't know if newer versions will work. When I get ready to release, I want to be conservative. Which means that I know that the exact versions I am using work well, but nothing else; I don't know that past or future versions of the packages I depend on will work. So I'd like to change my requirements so that they specify the exact versions I'm using at the time of release. As I think about it, it sounds pretty easy really. I just find all the requirements, see what version is installed, and rewrite requires.txt with those versions. At first I thought it should be a package management function, but should this be part of setuptools? A new command, or maybe an option to egg_info? It seems like it fits into egg_info well. This means requirements should be specified only in requires.txt, not in setup(). There's some other features I can think of specifically using subversion together with setuptools, but maybe I'll mull on those a bit longer. Nothing fancy, but codifying a specific pattern, something like: easy_install.py -e -b dev package_name cd dev/package_name # I would like "in-development" packages to have some specific version # but I don't know what that version should be...? sudo python setup.py develop -m # time passes, development is done, release is ready... python setup.py svntag 0.5 # changes PKG-INFO with new version, copies trunk to tags/0.5, does an # svn switch huh... maybe I can add such a command with an entry point? -- Ian Bicking / ianb(a)colorstudy.com / http://blog.ianbicking.org

3 17

easy_install: custom package indexes
by Ian Bicking 10 Aug '05

10 Aug '05

I have a custom package index that lists some Subversion repositories at http://pythonpaste.org/package_index.html ; I'm just looking for a little feedback on best practices. For instance, here is a link: http://svn.colorstudy.com/trunk/SQLObject#egg=SQLObject (note that there's a small bug in setuptools that'll keep these from working at the moment, but putting that aside...) Should that look like #egg=SQLObject-DEV ? Or something else? I think that tag will get ignored later in the process; should I modify setup.py (or setup.cfg) in some way to install the subversion checkout with an appropriate version number? If that package_index.html is added with --find-links (or added to ~/.pydistutils.cfg or elsewhere) then the subversion version will always be installed when no version is given in the package specifier ("easy_install.py SQLObject"). I don't know if that's right. I could change the PyPI entry and put the subversion repository there, and then I think the most up-to-date stable version is more likely to be installed (unless they do "easy_install.py SQLObject==DEV"?). But I don't control the PyPI entries for some of these projects. -- Ian Bicking / ianb(a)colorstudy.com / http://blog.ianbicking.org

2 3

easy_install: --tag-svn-revision
by Ian Bicking 10 Aug '05

10 Aug '05

Should easy_install have a --tag-svn-revision option like setuptools/setup.py has? I'm feeling a desire for it now, while installing someone else's package that's only available in svn. -- Ian Bicking / ianb(a)colorstudy.com / http://blog.ianbicking.org

2 5

setuptools updated in CVS
by Phillip J. Eby 10 Aug '05

10 Aug '05

FYI, I've just completed another round of changes to setuptools in CVS, including performance enhancements and new extensibility features (with documentation). (And the 'sdist' command now incorporates the workaround patch for 'delete="true"' in .svn/entries files.) setup() can now take a 'setup_requires' argument that specifies requirements that must be present before setup() runs; this allows you to depend on projects like buildutils, py2exe, py2app, etc. for setup arguments, commands, etc.. distutils extensions can now also define entry points to define new setup() arguments and new EGG-INFO metadata files, as well as setup commands. With this release, it should be possible to take a distutils extension like "buildutils" and implement all of its distutils hooks just by defining "entry points" in its setup script, to declare setup commands, setup() args, and metadata types. Entry points can declare extras that they depend on, too, so this means that commands can depend on external packages, and setuptools will attempt to download and install those packages in the setup directory if the package isn't already available on sys.path. This is probably the last round of non-bugfix changes before releasing 0.6a1, apart from further documentation enhancements. Before 0.6a1 is released, I'd like to add more tips on non-root installations with EasyInstall, and create an updated API reference for pkg_resources. Anyway, I haven't done a lot of regression testing yet, mostly just new features testing, so these changes may still be a bit rough around the edges. I'd appreciate any feedback. Thanks.

3 6

bdist_egg and source files
by Kevin Dangoor 09 Aug '05

09 Aug '05

Something that just struck me as I kept running bdist_egg --exclude-source-files is that it seems like "bdist" would imply exclusion of source files. Perhaps there could be an sdist_egg that creates an egg that includes all of the source... Kevin

2 2

Re: [Distutils] Problems with Python eggs/setuptools
by Phillip J. Eby 08 Aug '05

08 Aug '05

At 06:28 PM 8/8/2005 +0200, Walter Dörwald wrote: >Phillip J. Eby wrote: > >>At 07:29 PM 8/5/2005 +0200, Walter Dörwald wrote: >> >>>Hello Philip! >>> >>>I've started to play around with your easy_install script, since >>>managing our packages gets more complicated with each version. >>> >>>After I've installed setuptools via ez_setup.py I've tried downloading >>>the ll-xist package via "python -m easy_install ll-xist" but got a stack >>>trace instead (see the attached stacktrace.txt). The problem seems to be >>>the umlaut in my name. As PyPI requires UTF-8 encoded strings now, I've put a >>> author=u"Walter Dörwald".encode("utf-8") >>>in my setup.py when I registered the package (but a plain >>> author=u"Walter Dörwald" >>>in the setup.py in the package). >>> >>>To work around this problem it's possible to set the system default >>>encoding to Latin-1. I don't know if this is a problem with setuptools >>>or distutils, but doing a simple "python setup.py install" works. >>Note that this problem is a distutils problem writing PKG-INFO files; >>it's not anything specific to setuptools. I would recommend you use the >>.encode('utf-8') workaround until there is an official policy for the >>distutils to deal with encodings in PKG-INFO. > >As PyPI espects UTF-8 now, I'd say it makes sense to use UTF-8 in package >info files too. That's probably true, but this probably needs at least some distutils-sig discussion and maybe a PEP update. >>>The other problem seems more severe: ll-xist is installed as the >>>packages ll.xist, but the package init file ll/__init__.py is *not* >>>provided by this package, but by the ll-core package instead. If I >>>understood >>>http://peak.telecommunity.com/DevCenter/setuptools#namespace-packages >>>correctly distributing subpackages via setuptools only works if the >>>package init file is empty, which it isn't in this case. >>Technically, you can do it, it's just not guaranteed that the non-empty >>one will be executed, unless it's first on sys.path. > > >>> Is there any workaround for this? >>You can distribute an identical __init__.py with each distribution that >>shares that parent package. > >I tried to avoid that, because the Windows installer then asks the user, >whether files should be overwritten or not when installing the second >package, and for package managers like RPM, Debian .deb or Gentoo emerge >it's also problematic, if one file (even if it is empty) is provided by >several packages. Maybe the best solution, then, is to arrange things such that these installers install the packages as eggs, since this would then avoid any overwrite issues. This is pretty much how eggs are supposed to work with packaging systems in the long run. >>and make sure that all the distributions that don't contain the _my_init >>module have declared a dependency to the one that does. > >OK, this would work, but the clearer alternative would be to simply move >everything from ll to ll.core and have ll as an empty package module >again. Nevertheless this would require that each package includes the >empty __init__.py again, with the problems mentioned above. Hm. Tricky. I'll have to give this some thought, because setuptools *has* to have the __init__, since separately distributed packages are in separate directories or zipfiles, but it never occurred to me that you could hack the distutils to do partial distribution in this way. So, I never came up with a strategy for it. Ideally, it sounds like I should have a way to ensure that when setuptools builds an egg, it should create empty __init__.py files if they don't exist. That way, you could leave out the __init__.py for other builds, but it would ensure the package exists for the egg build. The tricky part is that I can't do it in the build directory, because then it might sneak into other distributions. But it's probably doable. >>If you want to try it, change this line in the _handle_ns routine of >>pkg_resources: >> module = sys.modules.get(packageName) or loader.load_module(packageName) >>to read instead: >> module = loader.load_module(packageName) >>and this will cause it to load *all* of the __init__ modules. However, >>the order of execution is still not guaranteed, and I'm not certain that >>it might not cause an __init__ to be reloaded if its grandparent >>directory is included twice on sys.path. Anyway, let me know if that >>change works for you, and I will think some more on the "load all >>__init__ modules" strategy. Even if I do implement it, I will want to >>mainly advise people to use empty __init__ modules for any new namespace >>packages they create. > >If I've understoodd this correcty, it doesn't solve my problem, as I only >have *one* __init__.py. And as setuptools puts different subpackages in >different directories, that get added via .pth files, I'm not sure, if >there can be a simple solution. Well, if I create the __init__ in the zipfile, and pkg_resources executes all the __init__ files, then it should work even with your current layout, as long as there's a dependency declared from the package without the __init__ to the package with it.

1 0