enscons, a prototype SCons-powered wheel & sdist builder

I've been working on a prototype Python packaging system powered by SCons called enscons. https://bitbucket.org/dholth/enscons . It is designed to be an easier way to experiment with packaging compared to hacking on distutils or setuptools which are notoriously difficult to extend. Now it is at the very earliest state where it might be interesting to others who are less familiar with the details of pip and Python package formats.
Enscons is able to create wheels and source distributions that can be installed by pip. Source distributions built with enscons use pyproject.toml from PEP 518 ( https://www.python.org/dev/peps/pep-0518/ ) to install enscons before setup.py is run, and since pip does not yet implement PEP 518, include a setup.py shim for PEP 518. The same pyproject.toml includes most of the arguments normally passed to setuptools.setup(), which are used to create the metadata files needed by pip.
Enscons converts pip's setup.py arguments to SCons ones and then invokes SCons, and the project's SConstruct does the rest. For a normal from-pypi installation, enscons generates just enough egg_info to let pip build a wheel, then pip builds and installs the wheel which has more complete metadata.
Of course SCons is more interesting for more complicated projects. I've been working on building my pysdl2-cffi binding with SCons. It includes a custom Python code generation step that distutils knew nothing about. It's practically a one-liner to add my custom build step to SCons, and it knows how to rebuild each part of the project only when its dependencies have changed. I don't know how to do that with setuptools.
It is still a very early prototype. Don't expect it to be very good. Here are some of its limitations:
- There is no error checking. - The sdists can only be built inside virtualenvs; otherwise pip complains that --user and --target cannot be used together. - It also doesn't work if one of the pyproject.toml build requirements conflicts with something that's already installed. - 'pip install .' doesn't work; it still tries to invoke 'setup.py install', which is not implemented. - 'setup.py develop' is not implemented. Try PYTHONPATH. - I am not an experienced SCons user.
It also relies on what I'm sure will be a short-lived repackaging of SCons called 'import_scons' that makes 'python -m SCons' work, and includes a Python 3 version of SCons from github.com/timj/scons that seems to work well enough for my purposes.
On the plus side, it's short.
If you're interested in implementing Python packaging in SCons, or are interested in the straightforward but tedious task of copying just enough distutils trickery to implement a robust Python C extension compiler in SCons, then perhaps you should check out enscons.
Here is a SConstruct for enscons itself:
import pytoml as tomlimport ensconsmetadata = dict(toml.load(open('pyproject.toml')))['tool']['enscons'] env = Environment(tools=['default', 'packaging', enscons.generate], PACKAGE_METADATA=metadata, WHEEL_TAG='py2.py3-none-any', ROOT_IS_PURELIB=True)py_source = Glob('enscons/*.py') sdist = env.Package( NAME=env['PACKAGE_NAME'], VERSION=env['PACKAGE_METADATA']['version'], PACKAGETYPE='src_zip', target=['dist/' + env['PACKAGE_NAME'] + '-' + env['PACKAGE_VERSION']], source=FindSourceFiles() + ['PKG-INFO', 'setup.py'], )
env.NoClean(sdist)
env.Alias('sdist', sdist)env.Whl('purelib', py_source, root='.')
Thanks!

On Sun, Jun 26, 2016 at 10:45 PM, Daniel Holth dholth@gmail.com wrote:
I've been working on a prototype Python packaging system powered by SCons called enscons. https://bitbucket.org/dholth/enscons . It is designed to be an easier way to experiment with packaging compared to hacking on distutils or setuptools which are notoriously difficult to extend. Now it is at the very earliest state where it might be interesting to others who are less familiar with the details of pip and Python package formats.
Interesting, thanks Daniel.
This does immediately bring back memories of the now deceased Numscons: https://github.com/cournape/numscons. David Cournapeau wrote quite a bit about it on his blog: https://cournape.wordpress.com/?s=numscons Are you aware of it? It was able to build numpy and scipy, so maybe there's still something worth stealing from it (it's 6 years old by now though).
Cheers, Ralf

I hadn't seen it. It looks very thorough. One difference is that mine is not a distutils extension. It might have some good tricks.
On Sun, Jun 26, 2016, 17:40 Ralf Gommers ralf.gommers@gmail.com wrote:
On Sun, Jun 26, 2016 at 10:45 PM, Daniel Holth dholth@gmail.com wrote:
I've been working on a prototype Python packaging system powered by SCons called enscons. https://bitbucket.org/dholth/enscons . It is designed to be an easier way to experiment with packaging compared to hacking on distutils or setuptools which are notoriously difficult to extend. Now it is at the very earliest state where it might be interesting to others who are less familiar with the details of pip and Python package formats.
Interesting, thanks Daniel.
This does immediately bring back memories of the now deceased Numscons: https://github.com/cournape/numscons. David Cournapeau wrote quite a bit about it on his blog: https://cournape.wordpress.com/?s=numscons Are you aware of it? It was able to build numpy and scipy, so maybe there's still something worth stealing from it (it's 6 years old by now though).
Cheers, Ralf
participants (2)
-
Daniel Holth
-
Ralf Gommers