On Thu, Jul 10, 2014 at 11:52 +1000, Richard Jones wrote:
Is it possible (or desirable) that devpi be able to trigger builds of wheels for packages that it caches? I realise it'd be possible as part of a Jenkins trigger for an upload, but this is for cached packages from PyPI.
The actual building could be complex - in the case of current Linux wheels, we'd need to have separate indexes for the wheels for Debian, RHEL, Ubuntu, SUSE, etc. since the wheels are typically not compatible. That complexity could be external to devpi - it's just a case of getting the trigger to indicate that a build is desired.
Quite how this integrates with a user just doing a "pip install package" is a little beyond me for the moment - the delay between the cache event and build completing could be significant. In light of that it might be desirable to have a separate process which does all this work (fetches the latest package and performs the build). Has anyone done this? How would it keep up to date?
I think it's fine to discuss it here because the desirable flow of events is not clear, to me at least. My own current scenario looks like this: - i have an Index X on a remote server which I use for uploading and preparing releases for open source packages. - i want to install numpy and pandas but don't want to recompile on every installation but instead "cache" it in wheel format related to the computer i am working on. Apart from triggering the building of the wheel the question is where or how is this wheel uploaded, exactly. If we just say I can also use index X for uploading, then the required functionality boils down to downloading numpy/pandas packages, creating the wheel and uploading it like a normal release file. FYI "devpi test" already downloads a package, tests it and uploads test results back to the devpi server. That's a very similar flow of data. We might just consider a "devpi wheelify NAME" command which implements the downloading, building of wheel and uploading, reusing some of the "devpi test" code. It would just use the current index X (*). If this were to somewhat happen transparently it's clear that the building needs to happen on the client side as devpi-2.0 doesn't have means to build packages and might run on a different platform anyway. I guess we could think about making "devpi install" grow build/cache-wheel functionality -- it is currently just a dumb wrapper around pip. Any insights and thoughts on the matter welcome. best, holger (*) we could also think about uploading to another stage Y which might be a base for my X but more generally holds wheels of a certain platform. For example, users on ubuntu-14.04 64bit systems could always inherit from an index Y which holds wheels specific to that system. This could help with disambiguating platforms because i think the wheel filename does not contain enough information (thus pypi.python.org refuses to accept linux wheels IIRC).
Richard
-- You received this message because you are subscribed to the Google Groups "devpi-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to devpi-dev+...@googlegroups.com. To post to this group, send email to devp...@googlegroups.com. Visit this group at http://groups.google.com/group/devpi-dev. For more options, visit https://groups.google.com/d/optout.