Distutils-SIG September 2012

distutils-sig@python.org

36 participants
48 discussions

weekend project for someone...
by Daniel Holth 20 Sep '12

20 Sep '12

Got a project for someone... Take holger krekel's wonderful apipkg and setuptools' versatile pkg_resources.py, put them together, and define a "nice" API that exposes only the parts that are used externally. Including both the "installed/requirements" database and the actual resources "files on __path__" bits. Bonus points if you download every sdist from pypi and grep for "import pkg_resources" to see how it is really used. Daniel H https://crate.io/packages/apipkg/ http://nullege.com/codes/search?cq=pkg_resources

1 0

[issue145] .egg files not regenerated when .so file changes
by Chris Nehren 19 Sep '12

19 Sep '12

New submission from Chris Nehren: I'm trying to package Clearsilver for OmniOS (illumos derivative). I built it incorrectly the first time round (causing a symbol to not be found in the python neo_cgi.so) and then fixed it. I fixed and reinstalled the package on the target machine but the .egg with the broken neo_cgi.so was still around, and wasn't regenerated when I ran the code again. This caused trac to reference the old, broken .so in the old, broken .egg, causing it to fail despite it having a perfectly serviceable neo_cgi.so right where it knows where to find it. I would expect make(1)-like logic for regeneration of .egg files so that this sort of thing does not happen. Alternately, if I've misunderstood something, I would be very grateful to pointed at tfm so I can read. Thank you! ---------- messages: 694 nosy: cnehren priority: bug status: unread title: .egg files not regenerated when .so file changes _______________________________________________ Setuptools tracker <setuptools(a)bugs.python.org> <http://bugs.python.org/setuptools/issue145> _______________________________________________

1 0

zip imports ftw
by Daniel Holth 17 Sep '12

17 Sep '12

I'd like to take this moment to note how awesome importing from zipfile is. Did you know you can execute a zip file with Python, and if it has a __main__.py in the root it will be executed, with the zipfile's contents on the PYTHONPATH? file.zip: __main__.py somedep.py other/code.py python file.zip # runs!

6 13

Confused about distutils2/packaging
by Almar Klein 17 Sep '12

17 Sep '12

Hi, I'm eagerly looking forward to the improvements that distutils2/packaging will (hopefully) bring, and I very much appreciate the efforts of those involved. However, I'm a bit confused about the status. On http://pypi.python.org/pypi/Distutils2 it says packaging is included in Python 3.3, and on http://docs.python.org/dev/packaging/ there is guidance for how to write a .cfg file. On the other hand, on http://python.org/dev/peps/pep-0398/ (the 3.3 release schedule) it says that the inclusion of packaging is deferred. Which is it? Maybe it's a good idea to make the different docs consistent. Thanks, Almar -- Almar Klein, PhD Science Applied phone: +31 6 19268652 e-mail: a.klein(a)science-applied.nl

2 1

Re: [Distutils] [Python-Dev] packaging location ?
by Vinay Sajip 17 Sep '12

17 Sep '12

[moved from python-dev] > Possibly also useful (if only to standardise the approach) would be a > method to check hashes against installed files. Added as Distribution.check_installed_files() in distlib repo[1]. Docstring: """ Checks that the hashes and sizes of the files in ``RECORD`` are matched by the files themselves. Returns a (possibly empty) list of mismatches. Each entry in the mismatch list will be a tuple consisting of the path, 'exists', 'size' or 'hash' according to what didn't match (existence is checked first, then size, then hash), the expected value and the actual value. """ Regards, Vinay Sajip [1] https://bitbucket.org/vinay.sajip/distlib/

1 0

No >, >=, <, <= in environment markers spec
by Daniel Holth 16 Sep '12

16 Sep '12

Noticed that there is no >= etc. in the environment markers spec, but both environment markers implementations allow it. Shall I fix the spec? As of wheel 0.9.5, you can add to your setup.cfg lines to be included as Requires-Dist: in the output METADATA. They supplement setup.py's values. Only problem is they are only resolved during install from wheel, and not during an install directly from the sdist. [metadata] requires_dist = beaglevote; python_version == 2.7 reportlab quux; sys.platform == 'win32'

1 0

add hash algorithm agility to RECORD
by Daniel Holth 15 Sep '12

15 Sep '12

Proposed edits to https://bitbucket.org/dholth/python-peps/changeset/9c26fa50 In wheel I use urlsafe_b64encode_nopad() which omits the trailing = characters, but although very easy to implement isn't included in the stdlib. In this spec I use the stdlib urlsafe_b64encode(). diff -r 23f9640c2020 -r 9c26fa508424 pep-0376.txt --- a/pep-0376.txt Thu Sep 06 12:09:58 2012 -0400 +++ b/pep-0376.txt Thu Sep 06 12:24:40 2012 -0400 @@ -218,11 +218,16 @@ - an absolute path, using the local platform separator -- the **MD5** hash of the file, encoded in hex. Notice that `pyc` and `pyo` - generated files don't have any hash because they are automatically produced - from `py` files. So checking the hash of the corresponding `py` file is - enough to decide if the file and its associated `pyc` or `pyo` files have - changed. +- a hash of the file's contents. + Notice that `pyc` and `pyo` generated files don't have any hash because + they are automatically produced from `py` files. So checking the hash + of the corresponding `py` file is enough to decide if the file and + its associated `pyc` or `pyo` files have changed. + + The hash is either the empty string, the **MD5** hash of + the file, encoded in hex, or the hash algorithm as named in + ``hashlib.algorithms``, followed by the equals character ``=``, + followed by the hash digest as encoded with ``urlsafe_b64encode``. - the file's size in bytes @@ -391,9 +396,9 @@ And following methods: -- ``get_installed_files(local=False)`` -> iterator of (path, md5, size) +- ``get_installed_files(local=False)`` -> iterator of (path, hash, size) - Iterates over the `RECORD` entries and return a tuple ``(path, md5, size)`` + Iterates over the `RECORD` entries and return a tuple ``(path, hash, size)`` for each line. If ``local`` is ``True``, the path is transformed into a local absolute path. Otherwise the raw value from `RECORD` is returned.

2 5

Extension: Entry-Point
by Daniel Holth 14 Sep '12

14 Sep '12

I propose mapping a typical entry_points.txt to Metadata 1.3 like so: [pyramid.scaffold] starter=pyramid.scaffolds:StarterProjectTemplate zodb=pyramid.scaffolds:ZODBProjectTemplate alchemy=pyramid.scaffolds:AlchemyProjectTemplate [console_scripts] bfg2pyramid = pyramid.fixers.fix_bfg_imports:main pcreate = pyramid.scripts.pcreate:main pserve = pyramid.scripts.pserve:main pshell = pyramid.scripts.pshell:main proutes = pyramid.scripts.proutes:main pviews = pyramid.scripts.pviews:main ptweens = pyramid.scripts.ptweens:main prequest = pyramid.scripts.prequest:main [paste.server_runner] wsgiref = pyramid.scripts.pserve:wsgiref_server_runner cherrypy = pyramid.scripts.pserve:cherrypy_server_runner Extension: Entry-Point Entry-Point/pyramid.scaffold: starter=pyramid.scaffolds:StarterProjectTemplate Entry-Point/paste.server_runner: wsgiref = pyramid.scripts.pserve:wsgiref_server_runner

3 4

Re: [Distutils] [Python-Dev] packaging location ?
by Daniel Holth 14 Sep '12

14 Sep '12

Bento's documentation and source code should be required reading for anyone doing Python packaging work. It is actually well designed, which comes as something of a shock if you are accustomed to hacking on nested-subcommands-that-initialize-and-call-each-other. Bento cleanly separates the build and install phases, it includes an intermediate JSON metadata "ipkg.info" with the PEP metadata, install paths, and file lists, it is easy to hack on. It needs a better lexer. https://gist.github.com/3715068 - the intermediate (not human editable) package info.

2 1

Re: [Distutils] PEP386 Pre releases
by SAn 13 Sep '12

13 Sep '12

>On Wed Sep 12 19:47:39 CEST 2012, Donald Stufft wrote: > >On Wednesday, September 12, 2012 at 1:43 PM, Erik Bray wrote: >> That said, this doesn't match my workflow at all. After releasing >> "1.0" the next version is going to be "1.1", and any development >> pre-release will be "1.1.devX". "1.1a" might not ever even exist. I >> think others brought up this critique at the time PEP 386 was being >> discussed, but then nothing was ever done about it >_> > > Yea, this concerned me because 1.1.devX < 1.1a1 < 1.1b1 < 1.1c1 < 1.1 > is how i've seen it used in the wild. Looks like most everyone i've seen > using it so far has been doing it wrong. Don't think ive seen a single > person do it right. Hi, just yesterday i got bitten by this issue. FYI: # verlib "pep386" (from https://bitbucket.org/tarek/distutilsversion) >>> from verlib import NormalizedVersion as V >>> V("0.2a1") < V("0.2.dev0") < V("0.2") True Also there is a bug in that version of verlib because it contradicts pep386 here: >>> V("0.2rc1") < V("0.2") False # setuptools ¿? >>> from pkg_resources import parse_version as V >>> V("0.2.dev0") < V("0.2a1") < V("0.2") True # distutils >>> distutils.version.LooseVersion("0.2.dev0") < distutils.version.LooseVersion("0.2") False # O_O cheers, SAn PD: i wasn't subcribed to the list, sorry if this mail breaks the "thread".

2 2

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

Distutils-SIG September 2012