Distutils-SIG April 2014

distutils-sig@python.org

27 participants
30 discussions

by Charles Cazabon

Greetings, I'm having trouble getting setuptools to properly extract the svn revision and write it into the <package>.egg-info/PKG-INFO file in a new environment. It has worked correctly for years in our previous environments, so I'm not sure what I'm doing wrong at this point. The environments where this works are virtualenvs based on Python 2.6 (and previously to that, 2.5) on various versions of Ubuntu. The new environment is a Python 2.7.6 virtualenv on what will become Ubuntu 14.04 "Trusty Tahr" when that's finalized. The system-wide Python used to bootstrap the virtualenv has setuptools 3.3 installed. I've also installed setuptools 3.4.3 in the virtualenv to see if that helped, but it makes no difference. Our package's setup.cfg file contains the usual: [egg_info] tag_build = dev tag_svn_revision = true Running `python setup.py develop` (or egg_info directly, or one of the other commands that call egg_info) results in the <package>.egg-info/PKG-INFO containing: Metadata-Version: 1.0 Name: <packagename> Version: 5.0dev (with all other values being UNKNOWN, which is fine). In our other environments, the version properly gets the svn revision number appended, like this: Version: 5.0dev-r15364 The upcoming Ubuntu release features subversion 1.8.8 instead of the 1.6.x in the last long-term-support release, so I thought that might be the source of the problem -- but manually running the code from svn_utils.py shows it seems to work: >>> from setuptools.svn_utils import * >>> working_copy = '/path/to/srcdir/subdir' >>> SvnInfo().load(working_copy).get_revision() 15364 In case it's relevant, setup.py/setup.cfg and the package I'm dealing with are in a subdirectory of the working copy, so the .svn/ dir and its files etc are in the parent directory, not the directory setup.py runs in. `svnversion` correctly fetches the revision number regardless, of course. I haven't found any docs/FAQs/etc that would seem to be related - all the subversion-related issues seem to be from the earlier transition to 1.5 or 1.6, which is long past. Am I doing something wrong here? Can anyone suggest why the revision tagging is failing to work in my new environment? Thanks, Charles -- ------------------------------------------------------------------ Charles Cazabon <charlesc-distutils-python.org(a)pyropus.ca> Software, consulting, and services available at http://pyropus.ca/ ------------------------------------------------------------------

6 years, 9 months

Comments on PEP 426 and 459

by Daniel Holth

I read through the latest versions of PEP 426 and 459 "Metadata 2.0 and extensions". Here are my comments. The PEP suggests "setup.py dist_info" is a thing. Only "setup.py egg_info" works. It might make sense to refactor bdist_wheel to include a dist_info command, but if done badly it would break distributions that rely on the pluggable nature of .egg-info generation (most every part of the egg-info directory is written by a separate plugin). Why do we care about "Source labels MUST be unique within each project and MUST NOT match any defined version for the project."? I still think meta distributions are unnecessary and hard to understand. It's either a lot of trouble to try to discourage == dependencies while still allowing them sometimes, or an overcomplicated way to try to define requirements.txt 2.0 by allowing distributions that aren't exactly distributions. This sounds like a pain: " To avoid malicious hijacking of names, when interpreting metadata retrieved from a public index server, automated tools MUST NOT pay any attention to ``"provides"`` entries that do not correspond to a published distribution. " I'm not convinced that console scripts should not continue to be a specially named entry point / export rather than their own very similar thing. Overall both peps are quite good. We need to be able to put them to use to advance our goal of providing a transparently backwards-compatible Python packaging system. My peps repository on bitbucket has an updated json schema.

6 years, 9 months

mr.developer dev eggs not showing up in sys.path

by Reinout van Rees

Hi, A colleague came up with a problem yesterday (and last week) that only he seemed to have. He however found a stackoverflow question with the same problem: http://stackoverflow.com/q/21241451/27401 Basically: develop eggs from mr.developer don't show up in the sys.path of the bin/python (zc.recipe.egg script) and bin/django (djangorecipe part). Despite: - The eggs being mentioned as dependency in the project's setup.py. I triple-verified this; bog standard setup we use in all our projects. - One project uses buildout 1.4.4, the other 2.x. - Tried the latest mr.developer and one from half a year ago or so. - Yep, install_requires mentions them. I verified it with egg-info/requires.txt: they're there. - Yep, develop-egg links are present. - Upgraded setuptools, downgraded setuptools (all pretty recent versions, though): no change. - It all runs fine on my computer... The only solution is to add the missing eggs to the "eggs=" list of the relevant parts. Even though that should really be unnecessary as they're mentioned in the proper setup.py. => Anyone seen this behaviour before? Any idea of where to look? I'll try reproducing it locally, but it can't hurt to ask :-) Reinout -- Reinout van Rees http://reinout.vanrees.org/ reinout(a)vanrees.org http://www.nelen-schuurmans.nl/ "If you're not sure what to do, make something. -- Paul Graham"

6 years, 9 months

What is the syntax for passing conditional non-extra dependencies in setuptools?

by Daniel Holth

How do I specify a conditional (marker-guarded) non-extra dependency in setuptools? The syntax for a conditional extra dependency is currently: extras_require = { "ssl:sys_platform=='win32'": "wincertstore==0.2", "certs": "certifi==1.0.1", },

6 years, 9 months

install_requires vs. requires_dist, and setup.cfg

by Asheesh Laroia

>From what I understand in PEP 345, "Requires-Dist" is the name in PKG-INFO files for a distribution's dependencies on other distributions. >From what I also understand from personal experience, when using setuptools, install_requires is the name of the keyword argument to setup() that specifies a dependency on another distribution. Sanity-check: This doesn't mean that one should call setup() with a "required_dist" keyword argument, right? Relatedly, https://mail.python.org/pipermail/distutils-sig/2010-January/015211.htmlsee… to suggest that one day in our glorious future, when we can lay down our setup.py files and switch to setup.cfg, then we can call it requires_dist in that format. As I understand it. the setup.cfg revolution never happened, so I should still encourage new people packaging their own things to write install_requires in their calls to setuptools.setup(). Also, I just created a trivial setup.py that contains an "install_requires" and doing "python setup.py develop --user" does seem to have created a PKG-INFO file, but it doesn't have a "Requires-Dist" line. Details of my test case: ➜ rofl python Python 2.7.6 (default, Mar 22 2014, 15:40:47) [GCC 4.8.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import setuptools >>> print setuptools.__version__ 3.4.2 ➜ rofl cat setup.py from setuptools import setup setup(install_requires=['mwparser']) ➜ rofl grep mwparser UNKNOWN.egg-info/* UNKNOWN.egg-info/requires.txt:mwparser I expected to also see it in a Requires-Dist property within UNKNOWN.egg-info/PKG-INFO. Is that a mistaken expectation? If so, what if anything writes a "Requires-Dist" line anywhere? Thanks for your patience with me, and for your speedy responses so far! I'm really hoping to provide as much high quality information to people at PyCon as I can, and grateful that so many wise people are willing to have these conversations with me. -- Asheesh.

6 years, 9 months

setup.py should use if __name__ == '__main__', right?

by Asheesh Laroia

Hi nice distutils/PyPA people, I had a question that probably is worth showing up in the archives. Namely: It seems to me like bizarre bad form for the setup.py file to execute what amounts to a main() function at import time. I presume this is just some kind of historical accident, and if the early authors of the distutils docs were careful, they'd have recommended: from distutils.core import setup if __name__ == '__main__': setup() # FIXME args rather than just: from distutils.core import setup setup() # FIXME args Is that an accurate assessment? If so, that's great, because I plan to remark on this bemusedly in my talk. If there is a reason it is the way it is, however, then I will avoid making a joke that is wrong in an important way. -- Asheesh.

6 years, 9 months

If you have nothing, begin with with putting get-pip.py in ~/bin/pip ???

by Asheesh Laroia

Hi all, I'm trying to answer the question: "On a modern GNU/Linux, if you have no 'pip' command, and you want to live in the modern Python Packaging world, how do you start?" Hypothetically, let's say you're on the latest Python 2.7. Also, I am presuming that you either don't have root on the system, or you don't want to run things with sudo. One answer I've seen (and given...) is to download virtualenv.py by itself, and hope that it can create a virtualenv in e.g. ~/venv/base, and then always activate it. You can see that advice given here: http://dubroy.com/blog/so-you-want-to-install-a-python-package/#the-better-… This advice is not contradicted here < http://www.virtualenv.org/en/latest/virtualenv.html#installation> although it seems like the advice basically is, "Make sure you have pip already, but if you don't, then we have no particular advice for you." One alternative is to bootstrap via "get-pip.py", which happens (as far as I can tell) to operate properly as a ~/bin/pip. Then you can "pip install --user", and more. However, get-pip.py <http://pip.readthedocs.org/en/latest/installing.html> seems to, if you run it with no arguments, want to write to /usr/bin/pip. What I'm hoping exists is a variant of get-pip.py that I can suggest people put in ~/bin/pip (and tell them to fix their path to include $HOME/bin ), and then they can pip install --user to their heart's content. You could imagine this being called "heres-pip.py", rather than "get-pip.py", as in my imagination, running "heres-pip.py" would not be supposed to have side-effects like installing things unless you ask it to. You would just drop it into place. Shell transcripts follow, to help you see what I'm seeing, in case I'm perhaps doing something wrong: ➜ /tmp python get-pip.py Downloading/unpacking pip from https://pypi.python.org/packages/py2.py3/p/pip/pip-1.5.4-py2.py3-none-any.w… Downloading pip-1.5.4-py2.py3-none-any.whl (1.2MB): 1.2MB downloaded Installing collected packages: pip Found existing installation: pip 1.4.1 Uninstalling pip: Cleaning up... Exception: Traceback (most recent call last): File "/tmp/tmpL4ws00/pip.zip/pip/basecommand.py", line 122, in main status = self.run(options, args) File "/tmp/tmpL4ws00/pip.zip/pip/commands/install.py", line 279, in run requirement_set.install(install_options, global_options, root=options.root_path) File "/tmp/tmpL4ws00/pip.zip/pip/req.py", line 1376, in install requirement.uninstall(auto_confirm=True) File "/tmp/tmpL4ws00/pip.zip/pip/req.py", line 594, in uninstall paths_to_remove.remove(auto_confirm) File "/tmp/tmpL4ws00/pip.zip/pip/req.py", line 1781, in remove renames(path, new_path) File "/tmp/tmpL4ws00/pip.zip/pip/util.py", line 295, in renames shutil.move(old, new) File "/usr/lib/python2.7/shutil.py", line 303, in move os.unlink(src) OSError: [Errno 13] Permission denied: '/usr/bin/pip' ➜ /tmp python get-pip.py --user --upgrade mwparser Requirement already up-to-date: pip in /home/paulproteus/.local/lib/python2.7/site-packages Downloading/unpacking mwparser Downloading mwparser-0.1.tar.gz Running setup.py (path:/tmp/pip_build_paulproteus/mwparser/setup.py) egg_info for package mwparser Installing collected packages: mwparser Running setup.py install for mwparser /home/paulproteus/.local/lib/python2.7/site-packages/mwparser.py:13: SyntaxWarning: assertion is always true, perhaps remove parentheses? assert('<' not in tag and '>' not in tag, "sample tag is 'ol'") /home/paulproteus/.local/lib/python2.7/site-packages/mwparser.py:14: SyntaxWarning: assertion is always true, perhaps remove parentheses? assert(len(start_marker) == 1, "This code assumes start_marker's length is 1") Successfully installed mwparser Cleaning up... So it seems that you can (ab?)use get-pip.py as a personal ~/bin/pip. You just have to know not to type "install". I'm guessing that "get-pip.py" is a "pip" binary with "install pip" pre-pended to sys.argv, so by passing more things, I happen to be able to cause it to do what I want. Either way, I'd like to know more about what is sensible to recommend as a bootstrapping process. I presume that Mac OS users will have a similar issue as random GNU/Linux users who lack pip. I will say, if get-pip.py wanted to do "pip install --user" rather than "pip install", then I'd be happy recommending it as-is. (Other ideas welcome. I'm also semi-sure this is some kind of FAQ, but I don't see it in my reading of the distutils-sig list, so I thought I'd still ask it.) Cheers, Asheesh. P.S. I am *so* tempted to add a no-op distribution called "install" to PyPI so that "pip install install" would look like it works.

6 years, 9 months

Creating distribution for Python *module*, not package

by Paul Sokolovsky

Hello, I would like to create a PyPIable packaged disribution for a single-file python module (say, foo.py). How to achieve that? There doesn't seem to be "modules" param to setup() (as a mirror of "packages" param). Thanks, Paul mailto:pmiscml@gmail.com

6 years, 9 months

setuptools and git flow

by Charles Dimino

Hey all, I'm trying to use setuptools to manage my project, and this project uses git flow. Part of git flow (and in general) is that you don't know when you're working on develop branch what your next version will be yet. In a way, this means that work done on your develop branch doesn't have *any* version at all. Only once you create a release branch, will you know what your version will be. This is at odds with the way setuptools documentation talks about re: daily builds and snapshot releases. There's no option to change your project's "official" version identified entirely - only to append to it. So what ends up happening is a project has its release (1.0.0, for example), and eventually the released version identifier makes its way back into the develop branch. Then, when a new nightly develop build gets created, it's marked as 1.0.0-NIGHTLY-01012014. This is a problem, because it's *not* a 1.0.0 nightly build, it's a POST-1.0.0 nightly build. Since the develop branch isn't supposed to have an actual version, what are some recommendations? Should the version value in setup.py just be set to something like "DEVELOP" in the develop branch? That'd always create "DEVELOP-NIGHTLY-01012014" tags, which is acceptable. Thanks, Charlie

6 years, 9 months

wheel 0.23.0 released

by Daniel Holth

I've released version 0.23.0 of wheel, after verifying that its test pass against Python 2.6, 2.7, PyPy, Python 3.3 and Python 3.4. Most of the improvements in this release are from contributors. Thank you! The most surprising feature to anyone who relies on wheel's own installer is that bdist_wheel now relies on post-install setuptools scripts wrapper generation - which is automatically provided by recent wheel-supporting versions of pip, or manually by the wheel install-scripts [distribution [distribution ...]] command. This change also removes the --skip-scripts parameter from bdist_wheel. https://pypi.python.org/pypi/wheel/

6 years, 9 months

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

Distutils-SIG April 2014