Mailman 3 September 2009 - Distutils-SIG

Static metadata using setup.cfg
by Tarek Ziadé 18 Sep '09

18 Sep '09

Hello Following the discussion on the format of the static metadata file, it's evident for me that it has to be a ConfigParser file. Now for the name of the file, I wouldn't want yet another file since we already have setup.cfg and since it can be extended w/o any problem or bacward compatibility issue. (That's from Tres and Matthias idea at Pycon) I am also realizing that we can use it to describe "static metadata" almost with the existing distutils code. We can use the [global] section of the setup.cfg file to describe them. If we want to remove all metadata expressed as setup() arguments, we can just move them to the setup.cfg. example of setup.cfg file: """ [global] name: foo version: 1.0 author: tarek long_description: some long description here url: http://example.com """ What's left in setup.py : """ from distutils.core import setup setup() """ Right now the behavior of the code is: Distutils will take the setup.cfg options and apply them to the Distribution class, overriding any argument passed to setup(), then they will be in turn overriden by the command line options if any. This behavior seems fine. Now there's a very small change to make in distutils to make this work, wich consists of applying these values to the DistutilsMetadata object (the metadata attribute in the dist instance) I've changed this in my working trunk to give a try, and it works fine. the long_description field is expressed as a multi-line field following the config parser convention so no problem for it (see my example) Although there's another change we need to apply and decide : being able to express a list of values, for fields like "keywords" or "classifiers". It can be a "," separated list of values, or a "\n" separated one. I am proposing the current scheme (applying it in this precise order): 1/ if the value contains '\n' signs, it's a list and values are splitted using \n, then each element is left stripped (not sure yet if we want to right strip them too) 2/ if the value contains ',' signs, it's a list and values are splitted using ',' (not sure yet if we want to strip them) 3/ else, it's a value This allows us to use "," for small lists like keywords we know they will never contain any "," sign, and several lines when the values might contain "," signs. So for example: """ [global] name: foo version: 1.0 author: tarek long_description: some long description here url: http://example.com keywords: one, two, three classifiers = Classifier1 Classifier2 Classifier3 """ Any thoughts ? Cheers Tarek -- Tarek Ziadé | http://ziade.org

15 66

non-ascii characters in long description
by Reinout van Rees 18 Sep '09

18 Sep '09

I have non-ascii characters in my long description (as I just fixed a bug in my package that had to do with non-ascii characters). I did all the right things like opening my changelog and readme with "codecs.open(..., encoding='utf-8')" and so. But I ran into the following setuptools problem: When calling "python setup.py --long-description", setuptools effectively does a "print long_description", which works with a utf8 string, but fails with a unicode string. When uploading to pypi, setuptools calls unicode() on my long description, which fails with a utf8 string and works with a unicode string. So if I do long_description=my_unicode_string, --long-description fails and if I do long_description=my_unicode_string.encode('utf-8') the pypi upload fails. In the end I used the dirty UltraMagicString() hack from http://stackoverflow.com/questions/1162338/whats-the-right-way-to-use-unico… Any suggestions? Reinout -- Reinout van Rees - reinout(a)vanrees.org - http://reinout.vanrees.org Software developer at http://www.thehealthagency.com "Military engineers build missiles. Civil engineers build targets"

4 9

Problem installing setuptools under virtual python environment
by dan 18 Sep '09

18 Sep '09

I am on a Cent OS shared system (i.e. I don't have root) with python 2.4 and am having trouble setting up setuptools in a virtual environment. I start by setting up the virtual environment: $/usr/bin/python2.4 virtual-python.py --prefix=~/apps/python Creating /fslhome/ddw28/apps/python/lib/python2.4 Creating /fslhome/ddw28/apps/python/lib/python2.4/site-packages Creating /fslhome/ddw28/apps/python/include/python2.4 Creating /fslhome/ddw28/apps/python/bin Copying /usr/bin/python2.4 to /fslhome/ddw28/apps/python/bin You're now ready to download ez_setup.py, and run /fslhome/ddw28/apps/python/bin/python ez_setup.py then I add "~/apps/python/bin" to my PATH and run the setup script with the following errors $ sh setuptools-0.6c9-py2.4.egg error: can't create or remove files in install directory The following error occurred while trying to add or remove files in the installation directory: [Errno 13] Permission denied: '/usr/lib/python2.4/site-packages/test-easy-install-5267.write-test' The installation directory you specified (via --install-dir, --prefix, or the distutils default setting) was: /usr/lib/python2.4/site-packages/ ... I get the right answer when I run which: $ which python ~/apps/python/bin/python I also created a symlink ~/apps/python/bin/python2.4 -> ~/apps/python/bin/python and get the same errors (this fixed a similar problem I had before on a different machine). Can anybody think of what might be going wrong? Thanks, dan

2 1

Re: [Distutils] setup.cfg new format proposal
by P.J. Eby 17 Sep '09

17 Sep '09

At 12:41 PM 9/17/2009 +0200, Tarek Ziadé wrote: >Also, if I understand clearly the idea, I find it rather cryptic to >add conditions to each dependency >like what Sridhar has shown. That's actually not how it would work; you simply put section headings inside the extras_require field, rather than having multiple sections in setup.cfg. Then, the "static metadata" is just the existing PKG-INFO format. Setuptools already supports section headings in extras_require, it just doesn't (yet) automatically install the contents of those sections based on platform/python version; you'd have to explicitly request "easy_install somepackage[platform.win32,pyver-2.6]" at the moment. But adding automatic defaulting of those flags would be pretty trivial, once their format was officially defined. See http://peak.telecommunity.com/DevCenter/setuptools#declaring-extras-optiona… for more details.

3 2

uses for setup.cfg and extracting data from it
by Chris Withers 17 Sep '09

17 Sep '09

Hi All, Do people generally source control their package's setup.cfg? http://docs.python.org/distutils/configfile.html sort of implies it should be editable by the person installing the package, but I've never personally used a package where that's the case... Assuming the distutils docs are out of date and this file is really "owned" by the package maintainer, how do I extract information from it in setup.py (and elsewhere for that matter!) I'm looking for somewhere consistent to store the following for all my packages: - mailing lists url - issue tracker url - change log url - documentation url ...such that I can generate a sensible long_description for use on PyPI but also such that I can include the information in the Sphinx docs... cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk

3 5

Greylisting?
by Wolodja Wentland 15 Sep '09

15 Sep '09

Hi all, i've just seen the following in a mail header: --- snip --- X-Greylist: delayed 2135 seconds by postgrey-1.31 at albatross; Tue, 15 Sep 2009 18:14:38 CEST --- snip --- Why do my mail got delayed for more than 35 min? Is there anything I, as a subscriber to this list, can do about it? with kind regards Wolodja Wentland

2 1

Re: [Distutils] setup.cfg new format proposal
by P.J. Eby 14 Sep '09

14 Sep '09

At 10:13 AM 9/14/2009 -0400, Glyph Lefkowitz wrote: >Setuptools must be able to do it internally, but I can't find an API >in the documentation. The only time setuptools pays any attention to PKG-INFO metadata is to get a package's version when the filename isn't enough to determine it - something that usually happens only on an in-development copy of a package. All of the metadata that it uses comes from setuptools-defined files, and is provided via the API of Distribution objects: http://peak.telecommunity.com/DevCenter/PkgResources#distribution-objects

1 0

Compile C code and install to script dir?
by OB 14 Sep '09

14 Sep '09

Hello! I am trying to get distutils to compile a single C script and install the binary to the same script directory as the entry point scripts that are generated for the python part of the module. I would like to use all the includes, CFLAGS, etc that are used when compiling extension modules, but this C script is not an extension. Is there a way to do this through setup, or do I need to use distutils.ccompiler.new_compiler() and try to hunt down CFLAGS, default libs, etc? I am using setuptools 0.6c9 and distutils with Python 2.5.4. Any help would be greatly appreciated! Thanks, Orion

2 2

[buidout] ignoring site-packages
by Martijn Faassen 14 Sep '09

14 Sep '09

Hi there, A long-standing issue with using buildout is that even though it creates a restricted environment, it still uses the site-packages of the system python and there's no way to get rid of it. This is why for instance the Grok project tells people to create their buildouts within a "virtualenv --no-site-packages" virtual env. Because if we don't, a lot of people (especially all Mac OS X users that don't create their own python) will have a failing buildout due to conflicting libraries. The following code snippet could be added to the buildout-generated scripts, after the 'sys.path[:]' assignment: import site packages = set(sys.path) site_packages = set() site.addsitepackages(site_packages) packages = packages.difference(site_packages) sys.path = list(packages) This cleans all site packages out of sys.path before any further code is executed. How to make sure we run without site packages? We could of course add an option to buildout when it's run: $ bin/buildout --no-site-packages Unfortunately this would mean all buildout users would need to remember this. It therefore seems sensible to me to also add one to buildout.cfg: [buildout] no-site-packages = true What do people think? See also this bug: https://bugs.launchpad.net/zc.buildout/+bug/429383 Regards, Martijn

2 2

Re: [Distutils] setup.cfg new format proposal
by P.J. Eby 12 Sep '09

12 Sep '09

At 02:50 AM 9/12/2009 +0200, Tarek Ziadé wrote: >it means that we can even provide an XML-RPC service at PyPI so people >can query the metadata for their platform with zero download and zero >third-party >code execution. Ah, now that does sound rather useful, as it would allow installation and similar tools to resolve dependencies without first needing to build or download binaries. >I can see a use case of the configure.py file you are describing: >prepare the compilation some C Extensions by looking for some libs etc, > >But that's something that can already be done with setup.py when you build >or install the distribution. > >So what would be the metadata field that would require to look for a >library location ? I had in mind that if you had such a configure script, it could also generate a manifest of files to be installed, e.g. translation files, docs, data, scripts, etc., for the use of a simplified installer system. (Ala the BUILDS proposal of about a year ago.)

2 1