Vagrant VM for building NumPy (1.7.x) Windows binaries
Hi, I've created this repository: https://github.com/certik/numpy-vendor which uses Vagrant and Fabric to fully automate the setup & creation of NumPy binaries for Windows. The setup is especially tricky, I've thought several times already that I nailed it, and then always new things pop up. One can of course install things directly in Ubuntu, but it's tricky, there are a lot of things that can go wrong. The above approach should be 100% reproducible. So hopefully this repository will be useful for somebody new (like I am) to numpy releases. Also my hope is that more people can help out with the release just by running it on their machines and/or sending PRs against this repository. All essential logic is in these two files: https://github.com/certik/numpy-vendor/blob/master/fabfile.py https://github.com/certik/numpy-vendor/blob/master/setup-wine.sh The setup-wine.sh can be used directly in Ubuntu as well (it erases ~/.wine and reinstalls it). Times: on my computer and internet speed it takes a little less than 1h to prepare the VM (it needs to download the 300MB Ubuntu, then about 600MB of Latex, and few MB of other stuff, and then about 12 minutes of configuring), and about 0.5 for each binary, so about 6h total (the docs only take a few minutes). After doing all this, I just realized that I forgot to build numpy against atlas. So I will need to fix that. But apart from that, it should be ok. Travis, should I upload the binaries to sourceforge and do the beta release with this? Or should we wait until I fix the atlas. Also I need to do the Mac binaries. I'll have the Win binaries in about 6h from now, I am running one final build from scratch now. My apologies that it took me longer than expected. Ondrej P.S. Here is a PR for review (you need to apply it manually if you want the above to work): https://github.com/numpy/numpy/pull/384
On Mon, Aug 13, 2012 at 5:13 AM, Ondřej Čertík <ondrej.certik@gmail.com> wrote:
All essential logic is in these two files:
https://github.com/certik/numpy-vendor/blob/master/fabfile.py https://github.com/certik/numpy-vendor/blob/master/setup-wine.sh
The setup-wine.sh can be used directly in Ubuntu as well (it erases ~/.wine and reinstalls it).
If you want setup-wine.sh to be generally useful, I'd suggest - moving the dangerous rm -rf logic to the fabfile instead - having it set up into a user-specified directory instead of ~/.wine. My ~/.wine already has stuff in it... With wine it is very trivial to create a new "windows VM" -- you just set the environment variable WINEPREFIX to point to any directory, which then acts like ~/.wine. It's often recommended therefore to install every windows program into its own pristine WINEPREFIX, so as to avoid cross-contamination, make it easier to blow things away when necessary, and generally avoid the difficulty of actually administering windows. Cheers, -n
Hi Ondrej, On Mon, Aug 13, 2012 at 5:13 AM, Ondřej Čertík <ondrej.certik@gmail.com> wrote:
Hi,
I've created this repository:
https://github.com/certik/numpy-vendor
which uses Vagrant and Fabric to fully automate the setup & creation of NumPy binaries for Windows. The setup is especially tricky, I've thought several times already that I nailed it, and then always new things pop up. One can of course install things directly in Ubuntu, but it's tricky, there are a lot of things that can go wrong. The above approach should be 100% reproducible. So hopefully this repository will be useful for somebody new (like I am) to numpy releases. Also my hope is that more people can help out with the release just by running it on their machines and/or sending PRs against this repository.
Thanks for doing this. I think vagrant is the way to go. I myself have some stuff for native windows and vagrant (much more painful, but sometimes necessary unfortunately). Did you see veewee to create vagrant boxes ? It simplifies quite a few things, but maybe they matter more on windows than on linux, where this kind of things is much simpler. David
On Mon, Aug 13, 2012 at 6:46 AM, David Cournapeau <cournape@gmail.com> wrote:
Hi Ondrej,
On Mon, Aug 13, 2012 at 5:13 AM, Ondřej Čertík <ondrej.certik@gmail.com> wrote:
Hi,
I've created this repository:
https://github.com/certik/numpy-vendor
which uses Vagrant and Fabric to fully automate the setup & creation of NumPy binaries for Windows. The setup is especially tricky, I've thought several times already that I nailed it, and then always new things pop up. One can of course install things directly in Ubuntu, but it's tricky, there are a lot of things that can go wrong. The above approach should be 100% reproducible. So hopefully this repository will be useful for somebody new (like I am) to numpy releases. Also my hope is that more people can help out with the release just by running it on their machines and/or sending PRs against this repository.
Thanks for doing this. I think vagrant is the way to go. I myself have some stuff for native windows and vagrant (much more painful, but sometimes necessary unfortunately).
Did you see veewee to create vagrant boxes ? It simplifies quite a few things, but maybe they matter more on windows than on linux, where this kind of things is much simpler.
Nice, I'll try it. So far I was using https://github.com/cal/vagrant-ubuntu-precise-64 and it doesn't work for me: https://github.com/cal/vagrant-ubuntu-precise-64/issues/10 Nathan, thanks for the WINEPREFIX trick, I didn't know about that. You are right, the wine itself is pretty much isolated VM. Unfortunately the things around it aren't. I'll see after the release, if there is interest, I'll be happy to make the scripts more general. Ondrej P.S. The total building time for all binaries was actually less than 2h (I miscalculated it). So that's very usable.
On Mon, Aug 13, 2012 at 8:06 AM, Ondřej Čertík <ondrej.certik@gmail.com> wrote:
On Mon, Aug 13, 2012 at 6:46 AM, David Cournapeau <cournape@gmail.com> wrote: [...]
Did you see veewee to create vagrant boxes ? It simplifies quite a few things, but maybe they matter more on windows than on linux, where this kind of things is much simpler.
Nice, I'll try it.
Sadly, it does not work either for me: https://github.com/jedi4ever/veewee/issues/361 Ondrej
participants (3)
-
David Cournapeau -
Nathaniel Smith -
Ondřej Čertík