Mailman 3 June 2016 - Distutils-SIG

enscons, a prototype SCons-powered wheel & sdist builder
by Daniel Holth June 26, 2016

June 26, 2016

I've been working on a prototype Python packaging system powered by SCons called enscons. https://bitbucket.org/dholth/enscons . It is designed to be an easier way to experiment with packaging compared to hacking on distutils or setuptools which are notoriously difficult to extend. Now it is at the very earliest state where it might be interesting to others who are less familiar with the details of pip and Python package formats. Enscons is able to create wheels and source distributions that can be installed by pip. Source distributions built with enscons use pyproject.toml from PEP 518 ( https://www.python.org/dev/peps/pep-0518/ ) to install enscons before setup.py is run, and since pip does not yet implement PEP 518, include a setup.py shim for PEP 518. The same pyproject.toml includes most of the arguments normally passed to setuptools.setup(), which are used to create the metadata files needed by pip. Enscons converts pip's setup.py arguments to SCons ones and then invokes SCons, and the project's SConstruct does the rest. For a normal from-pypi installation, enscons generates just enough egg_info to let pip build a wheel, then pip builds and installs the wheel which has more complete metadata. Of course SCons is more interesting for more complicated projects. I've been working on building my pysdl2-cffi binding with SCons. It includes a custom Python code generation step that distutils knew nothing about. It's practically a one-liner to add my custom build step to SCons, and it knows how to rebuild each part of the project only when its dependencies have changed. I don't know how to do that with setuptools. It is still a very early prototype. Don't expect it to be very good. Here are some of its limitations: - There is no error checking. - The sdists can only be built inside virtualenvs; otherwise pip complains that --user and --target cannot be used together. - It also doesn't work if one of the pyproject.toml build requirements conflicts with something that's already installed. - 'pip install .' doesn't work; it still tries to invoke 'setup.py install', which is not implemented. - 'setup.py develop' is not implemented. Try PYTHONPATH. - I am not an experienced SCons user. It also relies on what I'm sure will be a short-lived repackaging of SCons called 'import_scons' that makes 'python -m SCons' work, and includes a Python 3 version of SCons from github.com/timj/scons that seems to work well enough for my purposes. On the plus side, it's short. If you're interested in implementing Python packaging in SCons, or are interested in the straightforward but tedious task of copying just enough distutils trickery to implement a robust Python C extension compiler in SCons, then perhaps you should check out enscons. Here is a SConstruct for enscons itself: import pytoml as tomlimport ensconsmetadata = dict(toml.load(open('pyproject.toml')))['tool']['enscons'] env = Environment(tools=['default', 'packaging', enscons.generate], PACKAGE_METADATA=metadata, WHEEL_TAG='py2.py3-none-any', ROOT_IS_PURELIB=True)py_source = Glob('enscons/*.py') sdist = env.Package( NAME=env['PACKAGE_NAME'], VERSION=env['PACKAGE_METADATA']['version'], PACKAGETYPE='src_zip', target=['dist/' + env['PACKAGE_NAME'] + '-' + env['PACKAGE_VERSION']], source=FindSourceFiles() + ['PKG-INFO', 'setup.py'], ) env.NoClean(sdist) env.Alias('sdist', sdist)env.Whl('purelib', py_source, root='.') Thanks!

2 2

Re: [Distutils] Benchmark regression feeds
by Pauli Virtanen June 24, 2016

June 24, 2016

Sorry for the unrelated noise --- sent to the wrong mailing list :/

1 0

Benchmark regression feeds
by Pauli Virtanen June 24, 2016

June 24, 2016

Hi, In case someone is interested in getting notifications of performance regressions in the Numpy and Scipy benchmarks, this is available as Atom feeds at: https://pv.github.io/numpy-bench/regressions.xml https://pv.github.io/scipy-bench/regressions.xml

1 0

Notice: PyPI APIs now return 403 when accessed via HTTP
by Donald Stufft June 24, 2016

June 24, 2016

In part of an ongoing effort to improve the security of PyPI, instead of redirecting (or silently allowing) requests made over HTTP to PyPI APIs, these APIs will now return a 403 and require people to make the initial request over HTTPS. This does not affect the UI portions of the site that are designed to be used by humans, for these we will still redirect (which will cause the browser to see the HSTS header and force the user to use HTTPS from then on out). Thanks! — Donald Stufft

2 1

Proposal: using /etc/os-release in the "platform tag" definition for wheel files
by Vitaly Kruglikov June 23, 2016

June 23, 2016

On 22 June 2016 at 08:51, Nathaniel Smith <njs at pobox.com> wrote: > I believe the status is that there's general consensus that something like > this would be useful, but there's no one who is currently actively working > in it. Thanks everyone for following up on my questions. This helps a lot. In particular, thank you for pointing me to manylinux1. https://www.python.org/dev/peps/pep-0513 appears to be incredibly well written, with excellent insights. I will likely adopt the manylinux1 approach. <https://www.python.org/dev/peps/pep-0513/>

1 0

Re: [Distutils] Proposal: using /etc/os-release in the "platform tag" definition for wheel files
by Nathaniel Smith June 23, 2016

June 23, 2016

I believe the status is that there's general consensus that something like this would be useful, but there's no one who is currently actively working in it. On Jun 22, 2016 5:53 AM, "Vitaly Kruglikov" <vkruglikov(a)numenta.com> wrote: There have been no updates in over a year. Has this effort died, or transitioned to another medium? Thx _______________________________________________ Distutils-SIG maillist - Distutils-SIG(a)python.org https://mail.python.org/mailman/listinfo/distutils-sig

3 2

Proposal: using /etc/os-release in the "platform tag" definition for wheel files
by Vitaly Kruglikov June 23, 2016

June 23, 2016

Thanks everyone for following up on my questions. This helps a lot. In particular, thank you for pointing me to manylinux1. https://www.python.org/dev/peps/pep-0513 appears to be incredibly well written, with excellent insights. I will likely adopt the manylinux1 approach. <https://www.python.org/dev/peps/pep-0513/>

1 0

Re: [Distutils] Proposal: using /etc/os-release in the "platform tag" definition for wheel files
by Nathaniel Smith June 22, 2016

June 22, 2016

There are still use cases for distro-specific wheels, though -- some examples include Raspbian wheels (manylinux1 is x86/x86-64 only), Alpine Linux wheels (manylinux1 is glibc only), internal deploys that want to build on Ubuntu 14.04 and deploy on 14.04 and don't need the hassle of generating manylinux-style binaries but would like a more meaningful platform tag than "linux", and for everyone who wants to extend wheel metadata to allow dependencies on external distro packages then having distro-specific wheels is probably a necessary first step. -n On Jun 22, 2016 09:49, "Noah Kantrowitz" <noah(a)coderanger.net> wrote: Manylinux has mostly replaced it as that covers the platforms 99% of people worry about. The tooling for manylinux is more complex than this would have been, but sunk cost etc etc and now that we have it might as well save everyone some headache. --Noah > On Jun 22, 2016, at 8:51 AM, Nathaniel Smith <njs(a)pobox.com> wrote: > > I believe the status is that there's general consensus that something like this would be useful, but there's no one who is currently actively working in it. > > On Jun 22, 2016 5:53 AM, "Vitaly Kruglikov" <vkruglikov(a)numenta.com> wrote: > There have been no updates in over a year. Has this effort died, or transitioned to another medium? Thx > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG(a)python.org > https://mail.python.org/mailman/listinfo/distutils-sig > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG(a)python.org > https://mail.python.org/mailman/listinfo/distutils-sig _______________________________________________ Distutils-SIG maillist - Distutils-SIG(a)python.org https://mail.python.org/mailman/listinfo/distutils-sig

5 6

Proposal: using /etc/os-release in the "platform tag" definition for wheel files
by Vitaly Kruglikov June 22, 2016

June 22, 2016

There have been no updates in over a year. Has this effort died, or transitioned to another medium? Thx

1 0

Removal of wheels deleting more than the data files
by Sylvain Corlay June 20, 2016

June 20, 2016

I discovered a quite serious bug in wheels ( http://bugs.python.org/issue27317) When specifying an empty list for the list of data_files in a given directory, the entire directory is being deleted on uninstall of the wheel, even if it contained other resources from other pacakges. Example: from setuptools import setup > setup(name='remover', data_files=[('share/plugins', [])]) The expected behavior is that only the specified list of files is removed, (which is empty in that case). When the list is not empty, the behavior is the one expected. For example, from setuptools import setup > setup(name='remover', data_files=[('share/plugins', ['foobar.json'])]) will only remove `foobar.json` on uninstall and the `plugins` directory will not be removed if it is not empty. Thanks, Sylvain

2 2