Mailman 3 - Distutils-SIG

Indexing modules in Python distributions
by Thomas Kluyver Feb. 7, 2017

Feb. 7, 2017

For a variety of reasons, I would like to build an index of what modules/packages are contained in which distributions ('packages') on PyPI. For instance: - Identifying requirements by static analysis of code: 'import zmq' -> requires pyzmq - Finding corresponding packages from different packaging systems: pyzmq on PyPI corresponds to pyzmq in conda, and python[3]-zmq in Debian repositories. This is an oversimplification, but importable module names provide a common basis to compare packages. I'd like a tool that could pick between different ways of installing a given module. People often assume that the import name is the same as the name on PyPI. This is true in the vast majority of cases, but there's no requirement that they are the same, and there are cases where they're not - pyzmq is one example. The metadata field 'Provides' is, according to PEP 314, intended for this purpose, but the standard packaging tools don't make it easy to use, and consequently very few packages specify it. I have started putting together a tool to index wheels. It reads a .whl file, finds modules inside it, and tries to identify namespace packages. It's still quite rough, but it worked with the wheels I tried. https://github.com/takluyver/wheeldex Is this something that other people are interested in? One thing I'm trying to work out at the moment is how the data would be accessed: as a web service that tools can query online, or more like Linux packaging, where tools download and cache a list to do lookups locally. Or both? There's also, of course, the question of how the index would be built and updated. Thanks, Thomas

2 1

bdist_deb always creates 'all' architecture package for me
by varun mittal Feb. 6, 2017

Feb. 6, 2017

Hi all I am totally new to debian package building. Need to create a deb package from source, for Ubuntu. The package would contain mostly python code and a singular C file. My control file in 'debian' directory reads 'any' for Architecture. But running bdist_deb always creates _all.deb package. How to control that ? I tried forcing it to 'amd64' too, but didn't succeed Thanks n regards Mittal

1 0

pypa.io PR to document the actual current spec update process
by Nick Coghlan Feb. 6, 2017

Feb. 6, 2017

Hi folks, The "Specifications" section in the pypa.io developer's manual had fallen behind the process we've actually been using in recent times, so I've finally submitted a PR to bring it up to date: https://github.com/pypa/pypa.io/pull/19 General questions about the change are best asked on the list, while detailed comments on the specific wording in the PR are best submitted through GitHub. Cheers, Nick. P.S. See https://github.com/pypa/pypa.io/issues/11 for some additional background -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

1 0

Adding the "Description-Content-Type" metadata field
by Nick Coghlan Feb. 6, 2017

Feb. 6, 2017

Hi folks, Marc Abramowitz has prepared a PR for the Core Metadata section of the specifications page [1] that adds a new "Description-Content-Type" field: https://github.com/pypa/python-packaging-user-guide/pull/258 The draft text has now reached the point where I'm prepared to accept it, so this thread offers folks one last chance to provide feedback before we make it official. Full text of the new subsection ========================================= Description-Content-Type ~~~~~~~~~~~~~~~~~~~~~~~~ A string containing the format of the distribution's description, so that tools can intelligently render the description. Historically, PyPI supported descriptions in plain text and `reStructuredText (reST) <http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html>`_, and could render reST into HTML. However, it is common for distribution authors to write the description in `Markdown <https://daringfireball.net/projects/markdown/>`_ (`RFC 7763 <https://tools.ietf.org/html/rfc7763>`_) as many code hosting sites render Markdown READMEs, and authors would reuse the file for the description. PyPI didn't recognize the format and so could not render the description correctly. This resulted in many packages on PyPI with poorly-rendered descriptions when Markdown is left as plain text, or worse, was attempted to be rendered as reST. This field allows the distribution author to specify the format of their description, opening up the possibility for PyPI and other tools to be able to render Markdown and other formats. The format of this field is the same as the ``Content-Type`` header in HTTP (e.g.: `RFC 1341 <https://www.w3.org/Protocols/rfc1341/4_Content-Type.html>`_). Briefly, this means that it has a ``type/subtype`` part and then it can optionally have a number of parameters: Format:: Description-Content-Type: <type>/<subtype>; charset=<charset>[; <param_name>=<param value> ...] The ``type/subtype`` part has only a few legal values: - ``text/plain`` - ``text/x-rst`` - ``text/markdown`` The ``charset`` parameter can be used to specify whether the character set in use is UTF-8, ASCII, etc. If ``charset`` is not provided, then it is recommended that the implementation (e.g.: PyPI) treat the content as UTF-8. Other parameters might be specific to the chosen subtype. For example, for the ``markdown`` subtype, there is a ``variant`` parameter that allows specifying the variant of Markdown in use, such as: - ``CommonMark`` for `CommonMark` <https://tools.ietf.org/html/rfc7764#section-3.5>`_ - ``GFM`` for `GitHub Flavored Markdown (GFM) <https://tools.ietf.org/html/rfc7764#section-3.2>`_ - ``Original`` for `Gruber's original Markdown syntax <https://tools.ietf.org/html/rfc7763#section-6.1.4>`_ Example:: Description-Content-Type: text/plain; charset=UTF-8 Example:: Description-Content-Type: text/x-rst; charset=UTF-8 Example:: Description-Content-Type: text/markdown; charset=UTF-8; variant=CommonMark Example:: Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM Example:: Description-Content-Type: text/markdown; charset=UTF-8; variant=Original If a ``Description-Content-Type`` is not specified or it's set to an unrecognized value, then the assumed content type is ``text/x-rst; charset=UTF-8``. If the ``charset`` is not specified or it's set to an unrecognized value, then the assumed ``charset`` is ``UTF-8``. If the subtype is ``markdown`` and ``variant`` is not specified or it's set to an unrecognized value, then the assumed ``variant`` is ``CommonMark``. ========================================= [1] https://packaging.python.org/specifications/#core-metadata Regards, Nick. P.S. I know I still need to update https://www.pypa.io/en/latest/specifications/ to reflect the ability to make small backwards compatible adjustments to the specifications without a PEP, so I'll get that sorted today, since I've been talking about it for approximately forever. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

1 0

install questions and help requested. ---pyautogui
by Michael Strain Feb. 1, 2017

Feb. 1, 2017

I am looking for a simple short cut where I can automate a few clicks with a button via python automate the boring stuff. I go through data then I am seeking a short cut for the click inorder to be time efficient. I am having some errors. I am new to python. I am using windows. I have read that I have to run 'pip install pyautogui' I have used windows cmd. python shell and python command. I get a syntax error from python and a nothing from windows. Where should I put this? I found the pyautogui program and downloaded it and unzipped it. I try to jump straight to the import script of pyautogui like SyntaxError: multiple statements found while compiling a single statement Traceback (most recent call last): File "<pyshell#1>", line 1, in <module> import pyautogui ModuleNotFoundError: No module named 'pyautogui' >>> pip install pyautogui SyntaxError: invalid syntax Sincerely, Michael G. Strain Jr.

2 1

Re: [Distutils] latest setuptools appears to require six in a breaking way
by Donald Stufft Jan. 30, 2017

Jan. 30, 2017

> On Jan 27, 2017, at 2:05 PM, Eric Brunson <brunson(a)brunson.com> wrote: > > On Wed, Jan 25, 2017 at 5:52 PM Donald Stufft <donald(a)stufft.io <mailto:donald@stufft.io>> wrote: > >> On Jan 25, 2017, at 12:27 PM, Eric Brunson <brunson(a)brunson.com <mailto:brunson@brunson.com>> wrote: >> >> It wasn't until recently the I realized how quickly releases to setuptools and pip are being made, starting back in mid Dec when much of our dependency resolution started failing. There were three releases in the past two days. Four major and 22 minor releases in the past two months. While I applaud the speed of development and the improvement in these tools, don't you feel that breaking changes should be advertised better before release or perhaps we should slow down the cadence for release? >> >> I think an expectation that every setuptools user in the community start their day by checking to see if there was a release in the past 24 hours is a little unreasonable. I've spent a dozen hours since 32.0.0 resolving breakage in my own projects and assisting other developers in my org with their setuptools issues, all the while pushing setuptools as the best practice to do development and distribution. Is this period of breaking changes a short term thing that we just have to tough out for a few more weeks? >> >> Thanks, >> e. >> > > > I don’t believe that pip is really releasing that quickly. We generally make 1-2 “major” versions a year that include breaking changes, 2-4 “minor” releases a year that add new features, and 6-10 patch releases that fix bugs. To me that feels like a pretty decent pace of balancing not breaking people and getting new changes into people’s hands and getting rid of broken or less optimal parts of the code. > > Now, setuptools is releasing faster than pip is and whether that’s a good thing or not I don’t know. That’s a question for Jason largely :) > > Hey Donald, > > Thanks for the reply. > > Doesn't pip rely heavily on setuptools? I understand they have different origins, but I thought that since pip was moved under the purview of PYPA a lot of work was being done to converge the projects. When I run a pip -e one of the last message I see is "running setuptools.py develop", which isn't really a dependency, but can certainly cause people to infer that the problem is with pip and not know setuptools. Even the release notes Matthias references mentions pip as though it might be affected. > > If pip doesn't rely on setuptools, does that mean we have two separate and possibly different dependency resolution algorithms? > > Sincerely, > e. The answer here is… complex. Pip itself does not depend on setuptools though we _do_ depend on pkg_resources, but like all of our dependencies we bundle a copy internally and use that. Our uses of pkg_resources is primarily to inspect the currently installed packages to see what versions of what have been installed. If you’re installing from a Wheel file with pip, then you do not need to have setuptools installed at all (although if the wheel depends on setuptools we will obviously install it then). If you’re installing from a sdist, *then* you need setuptools installed because we’re going to execute the ``setup.py`` that is inside of that sdist and it will use setuptools (and we do some gross hacks to ensure it *always* uses setuptools, even if the project imports distutils). We are careful to pass in to setuptools the correct arguments that disables setuptools own handling of dependencies and we implement our own resolver. In the cases setuptools is being used (besides our bundled pkg_resources) it is primarily being used as a build tool. That being said, setuptools currently handles setup_requires on it’s own so the setuptools resolver handles that (though PEP 518 provides a way that in the future pip will use so that pip can handle build requirements instead of needing setuptools to do it). There is also a little known ``—egg`` parameter to pip that tells pip to stop preventing setuptools from doing it’s own resolving… but that is deprecated and is either removed or will be removed in the next version. So yes, they do have two separate and different dependency resolution algorithms (for instance, pip won’t install pre-releases by default unless it’s all that is available while setuptools will) but when you’re using ``pip install`` the setuptools one is carefully disabled *except* for setup_requires. — Donald Stufft

2 1

latest setuptools appears to require six in a breaking way
by Chris Withers Jan. 27, 2017

Jan. 27, 2017

Hi All, Anyone else seen this? |File "/home/travis/virtualenv/python2.6.9/lib/python2.6/site-packages/setuptools/__init__.py", line 10, in <module> from six.moves import filter, map ImportError: No module named six.moves| Started happening in my nightly builds on travis after the 34.0.2 release of setuptools. I've filed an issue here since I suspect it isn't just me: https://github.com/pypa/setuptools/issues/945 cheers, Chris

6 9

Virtalenv and setuptools [was: latest setuptools appears to require six in a breaking way]
by Paul Moore Jan. 25, 2017

Jan. 25, 2017

On 25 January 2017 at 17:20, Chris Withers <chris(a)simplistix.co.uk> wrote: > Right, so what's the recommended one-step way to set up a virtualenv now in > Py 2.6-3.6? This is the point I would consider most significant here. Virtualenv is deliberately built to allow use offline - pip, wheel and setuptools are bundled so that it's possible to create a virtualenv without needing Internet access. This change to setuptools will, if I understand it, break that expectation. While it's not a common scenario, I think it's something that should be considered. Going forward, I see a number of options for virtualenv: 1. Bundle all of setuptools' dependencies as well. 2. Drop the "no internet required" constraint - if we do this, it may be reasonable to only bundle pip, and get latest versions of everything else from PyPI. 3. Drop auto-installing setuptools (it's not needed unless you're installing from sdist, and it's only a "pip install setuptools" away for people who need it). 4. Document the changed behaviour by saying that no internet is required as long as you use --no-setuptools. Thoughts, anyone? Is the situation common enough to warrant anything other than (4)? It used to be for me, when pip didn't cache downloads and I had a secured proxy to deal with, but now I'd be OK with (4). Paul

3 7

Enquire for python
by ali refaee Jan. 25, 2017

Jan. 25, 2017

Dear Sir/Madam I’m beginner in python, how can I get python modules? when run some function such as (import numpy as np) message ImportError: No module named numpy another ex import matplotlib.pyplot as plt ImportError: No module named matplotlib.pyplot how we install modules? where are the modules? can I get modules to run python? Sent from Windows Mail

2 1

recommended strategy for missing development tools
by Robin Becker Jan. 25, 2017

Jan. 25, 2017

A reportlab user says his pip install fails to create an importable C extension. He said "The platform is Mac OS X 10.11.6, aka "El Capitan". Pip didn't complain" which is probably true since the extension is not required for reportlab's main usage. I tried to reproduce on OS X 10.10.5, but my machine has xcode installed and the extension was correctly produced. I don't have any code in setup.py to prevent compilation; is there a way to alert users to the non-build of the extension(s). -- Robin Becker

3 3