On Thu, Jul 10, 2014 at 11:52 +1000, Richard Jones wrote:
> Is it possible (or desirable) that devpi be able to trigger builds of
> wheels for packages that it caches? I realise it'd be possible as part of a
> Jenkins trigger for an upload, but this is for cached packages from PyPI.
>
> The actual building could be complex - in the case of current Linux wheels,
> we'd need to have separate indexes for the wheels for Debian, RHEL, Ubuntu,
> SUSE, etc. since the wheels are typically not compatible. That complexity
> could be external to devpi - it's just a case of getting the trigger to
> indicate that a build is desired.
>
> Quite how this integrates with a user just doing a "pip install package" is
> a little beyond me for the moment - the delay between the cache event and
> build completing could be significant. In light of that it might be
> desirable to have a separate process which does all this work (fetches the
> latest package and performs the build). Has anyone done this? How would it
> keep up to date?

I think it's fine to discuss it here because the desirable flow of events is
not clear, to me at least.

My own current scenario looks like this:

- i have an Index X on a remote server which I use for uploading
  and preparing releases for open source packages.

- i want to install numpy and pandas but don't want to recompile on
  every installation but instead "cache" it in wheel format related
  to the computer i am working on.

Apart from triggering the building of the wheel the question is where
or how is this wheel uploaded, exactly.  If we just say I can also use index X
for uploading, then the required functionality boils down to downloading
numpy/pandas packages, creating the wheel and uploading it like a normal
release file.

FYI "devpi test" already downloads a package, tests it and uploads test results
back to the devpi server.  That's a very similar flow of data.  We might
just consider a "devpi wheelify NAME" command which implements the
downloading, building of wheel and uploading, reusing some of the "devpi
test" code.  It would just use the current index X (*).

If this were to somewhat happen transparently it's clear that the building
needs to happen on the client side as devpi-2.0 doesn't have means to
build packages and might run on a different platform anyway.  I guess
we could think about making "devpi install" grow build/cache-wheel
functionality -- it is currently just a dumb wrapper around pip.

be a base for my X but more generally holds wheels of a certain platform.
For example, users on ubuntu-14.04 64bit systems could always inherit
from an index Y which holds wheels specific to that system.  This could
help with disambiguating platforms because i think the wheel filename does
not contain enough information (thus pypi.python.org refuses to accept
linux wheels IIRC).