On Thu, Oct 09, 2014 at 16:25 +0100, Paul Moore wrote:
On 9 October 2014 16:02, holger krekel
wrote: Alternatively, and just as good from my POV, would be for the plugin to add the wheels to a local index of my choosing. That would preserve /root/pypi as a PyPI mirror, but I could add my wheel index as an extra index URL, to get the wheels that way.
The second option seems to fit better how things work internally currently. So something like:
1. setup X/pypi_with_wheels, inherit from root/pypi 2. implement a wheel builder plugin that implements a new devpi_server_on_pypi_upload(link) hook which triggers an async script that does download, convert, upload to root/pypi_with_wheels.
should be implementable without too much effort.
Cool. When you say "implements a new hook", can plugins implement hooks? If that's what you mean, I'll give it a go and see if I can write something (expect requests for help, I've not looked into writing plugins yet!) If you mean that devpi would need to implement a new hook internally first, I'm not sure I know enough to be able to do that...
Indeed, you'd need to implement a new hook within devpi-server. They are defined in devpi_server/config.py's PluginManager. And would need to be called from views.py's pkgserv where it calls into entry.cache_remote_file() -- entry.url points to the remote file and entry.basename is the raw basename. If you rather open an issue to introduce this hook and specify the info/semantics that you think are needed from the plugin, i can see to get to it sometime (but no time promises).
Somehow it'd be nice, i guess, if we could make things more real-time, i.e. you would always see wheel links on the simple page to begin with. There would be wheels made out of sdists, exe, whatnot and they would be created dynamically when actually accessed. This would probably make most sense for per-machine devpi deployments. Implementing this requires probably some more effort.
(Near) real-time would be best for me. My ideal would be (for example) to have "pip install lxml" install a wheel built from the exe that's available on PyPI. If the wheel convert happens asynchronously, that would mean the first install would fail, but if I repeated the install it would work.
real-time could be done maybe using two new plugin points: - devpi_server_get_releaselinks(stage) which returns the links to release files and is used to create the simple page for an index. A plugin could add canonical wheel filenames, generated from the existing links. - devpi_server_retrieve_file(entry) would remotely retrieve or build a file -- this happens during access to the release file so if it takes very long (lxml, ...) the "pip install" command would time out, however. Because of the time out issue, it may make sense to rather stick with the first async and overall somewhat simpler approach. best, holger
Paul
-- You received this message because you are subscribed to the Google Groups "devpi-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to devpi-dev+...@googlegroups.com. To post to this group, send email to devp...@googlegroups.com. Visit this group at http://groups.google.com/group/devpi-dev. For more options, visit https://groups.google.com/d/optout.