[Distutils] Comments on PEP 426

[Oscar Benjamin]

On 29 August 2013 16:49, Paul Moore <p.f.moore at gmail.com> wrote:
> On 29 August 2013 16:02, Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote:
>>On 29 August 2013 15:30, Antoine Pitrou <antoine at python.org> wrote:
> [...]
>>> (after all, it's just "python setup.py build_bdist", or something :-))
>>
>> The point is that pip and other packaging tools will use 'python
>> setup.py ...' to do all the building and wheel making and so on.
>> However the required interface that setup.py should expose is not
>> documented anywhere and is essentially implementation defined where
>> the implementation is the setup() function from a recent version of
>> setuptools. In the interest of standardising the required parts of
>> existing practice the required subset of this interface should be
>> documented.
>
> Specifically, the command is
>
>     python setup.py bdist_wheel
>
> But that requires the wheel project and setuptools to be installed,
> and we're not going to require all users to have those available.
>
> Also, other projects can build wheels with different commands/interfaces:
> * distlib says put all your built files in a set of directories then
> do wheel.build(paths=path_mapping) - no setup.py needed at all
> * pip says pip wheel requirement (but that uses setuptools/wheel under the hood)
> * bento might do something completely different

Yes, but whatever is used if the required interface from setup.py is
documented it's easy enough for a distribution author to create a
setup.py that would satisfy those commands. It could be as easy as:

if sys.argv[1] == 'bdist_wheel':
    sys.exit(subprocess.call(['bentomaker', 'build_wheel'])

or whatever. Then, if bento is build-required (or distributed in the
sdist), 'pip wheel' would work right? Bento could even ship/generate
setup.py files for bento-using distributions to use (I assume
'bentomaker sdist' does actually do this but I got an error installing
bento; see below...).

However, right now it's not clear exactly what the command line
interface would need to be e.g.: Should setup.py process any optional
arguments? How should it know what filename to give the wheel and what
directory to put it in? Should the setup.py assume that its current
working directory is the VCS checkout or unpacked sdist directory?
Will pip et al. infer sucess/failure from the return code? Who is
supposed to be responsible for any cleanup if necessary?

> The whole question of standardising the command line API for building
> (sdists and) wheels is being avoided at the moment, as it's going to
> be another long debate (setup.py is too closely associated with
> distutils and/or setuptools for some people).

Yes but rather than try to think of something better I'm just
suggesting to document what is *already* required, with some guarantee
of backward compatibility that will be respected in the future. Even
if wheels become commonplace and are used by the most significant
projects there will still be a need to build some distributions from
source e.g. because the authors didn't build a wheel for your
architecture, or the user/author prefer to make build-time
optimisations etc.

> AIUI, we're sort of moving towards the "official" command line API
> being pip's (so "pip wheel XXX") but that's not a complete answer as
> currently pip internally just uses the setup.py command line, and the
> intention is to decouple the two so that alternative build tools (like
> bento, I guess) get a look in. It's all a bit vague at the moment,
> though, because nobody has even looked at what alternative build tools
> might even be needed.
>
> I could have this completely wrong, though - we're trying very hard to
> keep the work in small chunks, and building is not one of those chunks
> yet.

Leaving the build infrastructure alone for now seems reasonable to me.
However if a static target is created for third-party build tools then
there could be more progress on that front.

I just tried to install bento to test it out and:

$ pip install bento
Downloading/unpacking bento
  Downloading bento-0.1.1.tar.gz (582kB): 582kB downloaded
  Running setup.py egg_info for package bento
Installing collected packages: bento
  Running setup.py install for bento
  Could not find .egg-info directory in install record for bento
Cleaning up...
Exception:
Traceback (most recent call last):
  File "Q:\tools\Python27\lib\site-packages\pip\basecommand.py", line
134, in main
    status = self.run(options, args)
  File "Q:\tools\Python27\lib\site-packages\pip\commands\install.py",
line 241, in run
    requirement_set.install(install_options, global_options,
root=options.root_path)
  File "Q:\tools\Python27\lib\site-packages\pip\req.py", line 1298, in install
    requirement.install(install_options, global_options, *args, **kwargs)
  File "Q:\tools\Python27\lib\site-packages\pip\req.py", line 668, in install
    os.remove(record_filename)
WindowsError: [Error 32] The process cannot access the file because it
is being used by another process:
'c:\\docume~1\\enojb\\locals~1\\temp\\pip-aae65s-record\\install-record.txt'

Storing complete log in c:/Documents and Settings/enojb\pip\pip.log

I tried deleting the mentioned file but just got the same error
message again. Is that a bento/pip/setuptools bug? I notice that the
bento docs don't mention pip on the installation page:
http://cournape.github.io/Bento/html/install.html

Here's the appropriate version information:

$ pip --version
pip 1.4.1 from q:\tools\python27\lib\site-packages (python 2.7)
$ python --version
Python 2.7.5
$ python -c 'import setuptools; print(setuptools.__version__)'
1.1

(I just very carefully updated pip/setuptools based on Paul's previous
instructions).

The bento setup.py uses bento's own setup() command:
https://github.com/cournape/Bento/blob/master/setup.py


Oscar