Mailman 3 June 2012 - Distutils-SIG

complete 'extras' proposal
by Daniel Holth June 25, 2012

June 25, 2012

The prose isn't PEP-compliant yet, but here is my whole proposal for adding extras to .dist-info-style / Metadata 1.2 distributions. An 'extra' is an optional feature that can augment a distribution's requirements. An extra name must be a valid Python identifier and is referenced using subscript notation, e.g. distribution[extra]. As a shortcut, multiple extras can be referenced as a comma-separated list distribution[extra1,extra2,...] which is expanded to "distribution distribution[extra1] distribution[extra2]". (It is not legal to require an extra without also requiring the providing distribution). The name 'extra' is added to the list of variables in PKG-INFO environment markers. During marker execution 'extra' is either 'None' or the name of the extra that is being installed. (The markers must be evaluated once for each extra that is being installed.) Extras are declared with the new Provides-Extra: extra field, repeated for each extra that a distribution provides. A distribution need not reference Provides-Extra: in any Requires-Dist: fields, a case that will most often occur to preserve compatibility with other distributions that require the named extra when a once-optional dependency becomes standard. The extra names 'test' and 'doc' are reserved for requirements that are needed for running automated tests or building documentation, respectively. Example lines from PKG-INFO: Requires-Dist: nose; extra == 'test' Provides-Extra: test

1 0

[issue139] pkg_resources.require can't deal with multiple installed versions if one is present in easy-install.pth
by mbogosian June 25, 2012

June 25, 2012

New submission from mbogosian <mtb19(a)columbia.edu>: Perhaps I have misunderstood the intended behavior, but I thought one should be able to do this: - - - - - - - - %< - - - - - - - - % easy_install BeautifulSoup==3.0.8 ... % easy_install -U BeautifulSoup [installs 3.1.0.1] ... % python -c 'import pkg_resources ; print pkg_resources.require("BeautifulSoup==3.0.8")' Traceback (most recent call last): File "<stdin>", line 1, in <module> File ".../site-packages/distribute-0.6.10-py2.5.egg/pkg_resources.py", line 648, in require needed = self.resolve(parse_requirements(requirements)) File ".../site-packages/distribute-0.6.10-py2.5.egg/pkg_resources.py", line 550, in resolve raise VersionConflict(dist,req) # XXX put more info here pkg_resources.VersionConflict: (BeautifulSoup 3.1.0.1 (.../site-packages/BeautifulSoup-3.1.0.1-py2.5.egg), Requirement.parse('BeautifulSoup==3.0.8')) - - - - - - - - >% - - - - - - - - If I use the '-m' tag (to keep it out of easy_install.pth), I have a different problem: - - - - - - - - %< - - - - - - - - % easy_install -m BeautifulSoup==3.0.8 ... % easy_install -U -m BeautifulSoup [installs 3.1.0.1] ... % python -c 'import pkg_resources ; print pkg_resources.require("BeautifulSoup==3.0.8")' [BeautifulSoup 3.0.8 (.../site-packages/BeautifulSoup-3.0.8-py2.5.egg)] % python -c 'import BeautifulSoup' Traceback (most recent call last): File "<string>", line 1, in <module> ImportError: No module named BeautifulSoup - - - - - - - - >% - - - - - - - - Can I not have my cake and eat it too? Meaning, can I not have multiple installed versions of a package with a default (i.e., an entry in easy-install.pth), but still be able to explicitly specify a previously installed version? ---------- messages: 659 nosy: mbogosian priority: bug status: unread title: pkg_resources.require can't deal with multiple installed versions if one is present in easy-install.pth _______________________________________________ Setuptools tracker <setuptools(a)bugs.python.org> <http://bugs.python.org/setuptools/issue139> _______________________________________________

2 1

unneeded warning about missing readme in case of rst
by Reinout van Rees June 25, 2012

June 25, 2012

Hi, I often get warnings like: warning: sdist: standard file not found: should have one of README, README.txt The reason for the missing file is that it is called README.rst now because bitbucket and github pick up that extension. Is there something that can be done about this warning? (See also a zest.releaser ticket about this: https://github.com/zestsoftware/zest.releaser/issues/14 ) Reinout -- Reinout van Rees http://reinout.vanrees.org/ reinout(a)vanrees.org http://www.nelen-schuurmans.nl/ "If you're not sure what to do, make something. -- Paul Graham"

2 2

[ANNOUNCE] pypiserver 0.6.0 - minimal private pypi server
by Ralf Schmitt June 14, 2012

June 14, 2012

Hi, I've just uploaded pypiserver 0.6.0 to the python package index. pypiserver is a minimal PyPI compatible server. It can be used to serve a set of packages and eggs to easy_install or pip. pypiserver is easy to install (i.e. just easy_install pypiserver). It doesn't have any external dependencies. http://pypi.python.org/pypi/pypiserver/ should contain enough information to easily get you started running your own PyPI server in a few minutes. The code is available on github: https://github.com/schmir/pypiserver Changes in version 0.6.0 ------------------------- - make pypiserver work with pip on windows - add support for password protected uploads - make pypiserver work with non-root paths - make pypiserver 'paste compatible' - allow to serve multiple package directories using paste -- Cheers, Ralf

1 0

plugin mechanism for packaging itself?
by Daniel Holth June 13, 2012

June 13, 2012

I would like to be able to add a command to packaging for all packages, not just my own. This is a mechanism that would stay out of the way of other packages. 1. Distribute a submodule of the namespace package 'packaging_hooks'. 'packaging_hooks.yourpackage' where 'yourpackage' is the pypi name of the package providing the hook 2. Include in the submodule a dictionary COMMANDS={'command_name':'class name' or class} and __all__ = ['COMMANDS', ...] Packaging calls pkgutil.iter_modules(prefix='packaging_hooks.') to import all the submodules of packaging_hooks, looking for COMMANDS= and resolving the dotted name of the command class if the value is not already the command class. This would only be a plugin mechanism for packaging itself, not a general-purpose replacement for entry_points.

2 3

building an application made from many internal packages
by Chris Jerdonek June 13, 2012

June 13, 2012

Hi, is there a recommended way of building and deploying a private Python application from many loosely coupled internal packages or modules as opposed to a single large package? Should this be done with a private PyPI server with each module/package having its own setup.py, or is zc.buildout (or some other tool) the more appropriate solution here? Thanks, --Chris

2 1

RFC: package multiple platforms into a single binary package
by Daniel Holth June 13, 2012

June 13, 2012

I've been trying to work out a filename convention and specification for packaging multiple platforms worth of Python into a single archive. The feature would reduce the number of mostly-redundant archives required for a package that is mostly Python but has a small extension module, or express that a package contains fat binaries a-la OS X. Now that Python includes the __pycache__ directory and ABI tags for extension modules, it will often be possible to copy installations of the same package compiled for different Python ABIs on top of each other without clobbering any files. For example, a directory could include .pyc files for Python 3.2 and 3.3, or extension modules for win32 and Linux at the same time. When copied into the same location, the only files clobbered by the second architecture should be the identical .py, .dist-info/METADATA and so forth, while the second platform's shared libraries will have a unique name. The proposed filename is {distribution}-{version}-{python tag}-{platform tag}.{archive extension} distribution: Distribution name, e.g. ‘django’, ‘pyramid’ version: PEP 386-compliant version, e.g. 1.0 Python implementation tag Platform tag or 'noarch' For example, package-1.0-py27-noarch.rar is compatible with Python 2.7 (any Python 2.7 implementation) on any CPU architecture. The Python implementation is abbreviated to save space. Each implementation has a two-letter code: py: Generic Python cp: CPython ip: IronPython pp: PyPy jy: Jython concatenated with the ABI tag: SOABI.split(‘-’)[-1] “33m”, or, for “pure Python” packages, py_version_nodot “27”. The platform tag is distutils.util.get_platform() with all periods and hyphens replaced with underscore, or the string ‘noarch’. The {python tag} field and {platform tag} field are actually .-delimited sets of tags. The filename indicates compatibility with the cartesian product of the two sets. For example, you would be able to combine a win32 and linux_x86_64 archive into package-1.0-py33-linux_x86_64.win32.tar.bz2 The part I'm unsure about is that 'noarch' and 'py27' don't quite fit with 'cp33dmu' (which says something about both the compiled code and the version of Python required by the .py files). Thanks, Daniel Holth

1 0

[issue138] setuptools.extension incompatible w/ Cython (patch included)
by JimC June 12, 2012

June 12, 2012

New submission from JimC <jim(a)carroll.com>: We discovered a minor problem with setuptools.extension.py that prevents being able to incrementally build extenion modules when working with Cython (meaning, edits to source do NOT result in a call to the cython compiler). A patch is attached to this ticket. The problem is the extension.Extension() class has logic that alters the filename the sources[] list if Pyrex is not installed. The code has a simple boolean test if Pyrex is installed, and if not, it remaps the extension from ".pyx" -> ".c". I believe the simple solution is to perform the same logic for Cython (to set a boolean if Cython is available, and to NOT remap the source extension if either Pyrex OR Cython is installed). Example to demonstrate the problem: setup('project', ..., ext_modules = Extension('Prj', ['Prj/pr1.pyx']), ..., cmdclass = {'build_ext': Cython.Distutils.build_ext}, ..., With a setup.py like this, the setuptools.extension.Extension class will re-write the sources[] list BEFORE the Cython.Distutils.build_ext module can get a shot at it. The result is that subsequent edits of 'Prj/pr1.pyx' will cause build_ext to skip the cython compile step. The only work around to force a recompile is to delete the contents of the 'build/' folder. See the suggested patch for our request for modification. Thanks ---------- files: extension.patch messages: 658 nosy: jim(a)carroll.com priority: bug status: unread title: setuptools.extension incompatible w/ Cython (patch included) Added file: http://bugs.python.org/setuptools/file80/extension.patch _______________________________________________ Setuptools tracker <setuptools(a)bugs.python.org> <http://bugs.python.org/setuptools/issue138> _______________________________________________

1 0

Packaging dependencies
by Keshav Kini June 12, 2012

June 12, 2012

Hi, We're using distribute to package our software, but we'd like to do something different with the "sdist" command. We would like to recursively find all packages necessary to build our package from a base Python install, download these packages, and store them all in a directory inside our source distribution tarball; then, when the tarball is unpacked and setup.py is run, we want "install" to use those cached packages. (This is to support installation on machines that can't or shouldn't access the network.) One solution I found in the distribute docs was to use `easy_install -zmaxd dirname package` on our package to generate and store dependency eggs, and then `easy_install -H none -f dirname` when installing the package. Unfortunately this actually builds packages which have C extensions and stuff like that in them, whereas we would like to only ship the source and build the dependencies when our package is built (to support multiple platforms). Also I guess we'd like `setup.py install` to just work in the directory without the user having to care about the fact that these source packages were shipped inside the tarball they just extracted. So I'm wondering if I can actually do this by importing something from distribute itself and writing some nontrivial code in setup.py. Can someone point me to what would be the best way to do this? I'm guessing I'll need to start by subclassing Environment to create a fake "default environment" which looks like how a bare Python 2.7.3 (say) install would. Then I'd need to somehow modify "sdist" somewhere to make it download the dependency tarballs that would be necessary if you were installing our packages into that Environment, and also need to modify "install" to make it look in the correct directory. Is that about right? Thanks and sorry for the simple question. -Keshav

2 1

bug#14651
by Rob Healey June 10, 2012

June 10, 2012

Greetings: I know that there are many many bugs out there right now, I must sincerely and grateful request that bug#14651's patch would please be accepted and applied before python-3.3.0 freeze happens! The only reasons that I must ask at this point is because without it, it completely stops the work that, me and others, are doing on our project??? -- Sincerely yours, Rob G. Healey

1 0