Mailman 3 February 2013 - Distutils-SIG

The Wheel specification and compatibility tags on Windows
by Vinay Sajip 28 Feb '13

28 Feb '13

I have a question about the compatibility tags used by Wheel (PEP 425) and their completeness on Windows. IIUC, it may not be enough to consider the platform and the Python ABI (determined by the Python version). In addition, on Windows, we may have to consider the version of compiler used to build a distribution, as it may not be compatible with a particular Python (even if platform and ABI match) if the Python was built with a different version of Visual Studio from that used to build the extension. I believe this is to do with the version of msvcrXX.dll, which is tied to the compiler version: both the Python and the extension need to link to the same Microsoft C run-time DLL. If I'm wrong about this, can someone please explain what my mistake is? If I'm right, do we perhaps need to amend the platform tag to include the compiler used? Currently, it's just distutils.util.get_platform().replace('-', '_'). Regards, Vinay Sajip

6 13

Re: [Distutils] [Catalog-sig] Deprecate External Links
by Daniel Holth 28 Feb '13

28 Feb '13

On Thu, Feb 28, 2013 at 1:23 PM, PJ Eby <pje(a)telecommunity.com> wrote: > Indeed. I'm hoping that the new tools will make the old ones (e.g. > setuptools) entirely irrelevant, which is why I'm hammering so hard in > the PEP discussions on some use cases that eggs do well that wheels > don't. I don't want people to have to keep using setuptools for those > use cases. (e.g. simple plugin deployment ala Trac) If the new tools > handle all of the use cases, then setuptools can die a natural death > sometime in the next decade or so, so I don't have to be responsible > for it when I turn old and senile. (It's already turned me grey as it > is.) ;-) Continued from catalog-sig... On my obsolete-setuptools checklist: - include just a small part of the pkg_resources API especially the "resources" API into a new standalone pkg_resources "pip install pkg_resources" distribution, backed by a different API. The parts only installers need would be excluded. - figure out what to do about the runtime-linker features I was always rather unhappy about .egg plugins in trac and am used to using lots of virtualenvs instead, but I think it is right to continue offering that kind of feature. Wheel is billed as an installation format but it is intentionally very similar to egg with revisions to what were in my opinion the most annoying parts, namely "wait for the packager to catch up when a new Python comes out" and "the EGG-INFO directory has the same name in all eggs". There's nothing stopping anyone from writing an "append .pyc files to this archive" script to overcome what's probably the first "I don't want to use this as a plugin" complaint. "Don't go crazy with paths in .data/" would be a second piece of advice for the budding plugin author.

1 0

PEP 426: proposed metadata caching convention
by Nick Coghlan 28 Feb '13

28 Feb '13

(This probably belongs in a successor to PEP 376, but I'll leave it under the PEP 426 umbrella for now) One of the points raised regarding PEP 426's integrated metadata format is the potential for runtime issues with pkg_resources as it reads and processes the metadata during startup, particularly if it needs to process any environment markers. While I acknowledge the suggestions I have received that we should really be moving away from the current filesystem based distributed installation information to a real database that properly handle import hooks, I'm looking for something simpler that will make it easier for setuptools and distribute to consume the new metadata format (and thus hopefully make them more amenable to generating it as well) Assuming we add an Entry-Points field as I have proposed in another message, I'd like to propose that installers generate three additional cache files as part of the installation process: <dist-info-dir>/__cache__/version.txt <dist-info-dir>/__cache__/requires-dist.txt <dist-info-dir>/__cache__/entry-points.txt version.txt would just be the version of the installed distribution (no need to parse the main metadata file just to read the version field) requires-dist.txt would be similar to the pkg_resources requires.txt format, but use PEP 426 version specifiers. It would: - only contain runtime requirements where the environment markers match the current system - be split into sections based on the "extras" definition needed to get the environment marker to pass entry-points.txt would be the same format as the pkg_resources entry_points.txt Cheers, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

7 21

Re: [Distutils] PEP 426: proposed metadata caching convention
by Glyph 27 Feb '13

27 Feb '13

On Feb 27, 2013, at 11:04 AM, Daniel Holth <dholth(a)gmail.com> wrote: > What does it have to do in the hook? > This: <https://twistedmatrix.com/documents/current/core/howto/plugin.html#auto3> While this is theoretically optional - Twisted will behave mostly correctly without it - it noticeably improves the start-up performance of Twisted-based command-line tools, like 'twistd' and 'trial'. -glyph

1 0

Re: [Distutils] [Python-Dev] [Python-checkins] peps: PEP 426: replace implied 'version starts with' with new ~= operator
by Daniel Holth 27 Feb '13

27 Feb '13

On Sat, Feb 23, 2013 at 10:23 AM, Nick Coghlan <ncoghlan(a)gmail.com> wrote: > On Sun, Feb 24, 2013 at 12:57 AM, Vinay Sajip <vinay_sajip(a)yahoo.co.uk> > wrote: > > Nick Coghlan <ncoghlan <at> gmail.com> writes: > > > >> Daniel is a fan of this syntax, but I think it is inferior to the > >> implied approach, so don't expect it to survive to any accepted > >> version of the PEP :) > > > > Another thing against ~= is that it isn't valid Python syntax. It's not > a deal- > > breaker, but it does mean that you can't e.g. use the ast module in the > > implementation. This might be a factor if the mini-language ever grows > (as it > > recently did, adding parentheses). > > Daniel persuaded me that the *semantics* of Ruby's ~> pessimistic > version comparison operator are highly desirable. I liked them so > much, I'm now proposing them as the default behaviour of version > specifiers. Thus, the "~=" operator goes away, and you can use "==" to > explicitly request the previously proposed default behaviour, or just > append an extra ".0" if you're more pessimistic about a dependency's > backwards compatibility policies than the default interpretation. > > This and other aspects will be brought up on distutils-sig at some > point not too far in the future :) > > Cheers, > Nick. > Join us on distutils-sig! It's like python-dev, except with more distutils! Good changes, ~> is useful, unlike "starts with". I don't think we should include starts-with at all. There simply must be a way to spell "equals" that means "equals". It will be used when that so-called security release broke my application that integrates said library in a way that doesn't even expose the flaw. Plone depends on hundreds of libraries that probably only use >= or no version at all in their setup.py. Then the buildout.cfg ships with the == versions of all the libraries that the Plone release team actually tested. == is not the common plague that you fear. Requirements specifiers have never been valid Python syntax; note the unquoted version strings. Environment markers are different and do not compare distribution versions. As a aside pkg_resources probably needs an operator [that might be removed when generating PKG-INFO] for use in setup.py install_requires. Setuptools parses distname [OPERATOR version] not distname [OPERATOR? version] Daniel

7 24

Re: [Distutils] PEP 426: proposed change to extension fields + entry points
by Vinay Sajip 25 Feb '13

25 Feb '13

Nick Coghlan <ncoghlan <at> gmail.com> writes: > The other area where I think such an "embedded JSON" approach could > work is coming up with a clean format for an Entry-Points field. > Specifically, I am thinking of proposing the setuptools.setup > inspired: > > Entry-Points: { > "console_scripts": { > "foo": "my_package.some_module:main_func", > "bar": "other_module:some_func" > }, > "gui_scripts": { > "baz": "my_package_gui.start_func" > } > } > What is the reason for this half-way-house, rather than going straight to JSON? I suppose one could argue backward compatibility, but since we are moving quite a distance from a plain old key-value here, we may as well embrace JSON properly. Perhaps we should embrace the dict rather than any specific file format, as I did with dictConfig in logging. Note that distlib already supports entry points, though these are called "exports" because they may or may not be callable code. (The name "exports" was suggested by PJE and, IMO, is better/more general than "entry points"). For example, for Babel-0.9.6: "exports": { "babel.extractors": [ "ignore = babel.messages.extract:extract_nothing", "python = babel.messages.extract:extract_python", "javascript = babel.messages.extract:extract_javascript" ], "distutils.commands": [ "compile_catalog = babel.messages.frontend:compile_catalog", "extract_messages = babel.messages.frontend:extract_messages", "init_catalog = babel.messages.frontend:init_catalog", "update_catalog = babel.messages.frontend:update_catalog" ], "babel.checkers": [ "num_plurals = babel.messages.checkers:num_plurals", "python_format = babel.messages.checkers:python_format" ], "distutils.setup_keywords": [ "message_extractors = babel.messages.frontend:check_message_extractors" ], "scripts": { "console": [ "pybabel = babel.messages.frontend:main" ] } } This is currently used, for example, in distlib's script generation logic (for example, to generate the pybabel script for Babel). Regards, Vinay Sajip

2 6

Do women work on Python packaging?
by Daniel Holth 25 Feb '13

25 Feb '13

With all the talk about pyladies etc. I wondered: are you female and working on Python packaging? Poll at http://strawpoll.me/7874

1 0

pip/easy_install mercurial in virtualenv on Python 3.3
by Christian Tismer 25 Feb '13

25 Feb '13

With Python 3.3: in a virtualenv/venv, when trying $ pip install mercurial or, after trying several options, $ easy_install mercurial I get the complaint error: Setup script exited with setup.py with python3 needs --c2to3 (experimental) But when I try to do this accordingly, I get in either case again: $ easy_install mercurial --c2to3 error: Setup script exited with setup.py with python3 needs --c2to3 (experimental) Any clue where to put this option? cheers -- chris -- Christian Tismer :^) <mailto:tismer@stackless.com> Software Consulting : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 : *Starship* http://starship.python.net/ 14482 Potsdam : PGP key -> http://pgp.uni-mainz.de phone +49 173 24 18 776 fax +49 (30) 700143-0023 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/

2 9

PEP 426: proposed change to extension fields + entry points
by Nick Coghlan 25 Feb '13

25 Feb '13

I've decided I'm really not happy with the current approach to extension fields in PEP 426. It's ugly, clunky, inflexible and is hard to cleanly convert to more structured metadata. Here's the current example from the PEP: Extension: Chili Chili/Type: Poblano Chili/Heat: Mild Here's what I'm thinking of changing it to: Extension: Chili Chili/json: { "type": "Poblano", "heat": "Mild" } So rather than allowing *arbitrary* additional headers as it does now, an Extension declaration would allow only one new header: <extension-name>/json That would then allow arbitrary embedded JSON (so long as you still follow the "no blank lines, because that 's the end of the headers" rule) When converting the entire metadata definition to JSON for subsequent processing, the extensions fields would then cleanly convert from multiple extension entries to something like: "extensions": { "Chili": { "type": "Poblano", "heat": "Mild" } } The other area where I think such an "embedded JSON" approach could work is coming up with a clean format for an Entry-Points field. Specifically, I am thinking of proposing the setuptools.setup inspired: Entry-Points: { "console_scripts": { "foo": "my_package.some_module:main_func", "bar": "other_module:some_func" }, "gui_scripts": { "baz": "my_package_gui.start_func" } } Cheers, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

2 6

Buildout 2.0.0 released
by Jim Fulton 24 Feb '13

24 Feb '13

I'm very happy, finally, to have released buildout 2.0.0. Much has changed from buildout 1: http://pypi.python.org/pypi/zc.buildout/2.0.0#id3 Installation instructions: http://pypi.python.org/pypi/zc.buildout/2.0.0#installation Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton Jerky is better than bacon! http://zo.pe/Kqm

11 24