Any plans for windows 64-bit installer for 1.7?
Hi,
I see there is no Windows 64 bit installer for the 1.7 rc1.
Is there any prospect of a 64 bit installer for the full release?
Can I help? I have a Windows 7 64 bit machine that I use as a build slave; I am happy to give access.
Cheers,
Matthew
On Sat, Feb 2, 2013 at 6:14 PM, Matthew Brett matthew.brett@gmail.com wrote:
Hi,
I see there is no Windows 64 bit installer for the 1.7 rc1.
related: Is there any chance to get newer mingw or mingw-w64 support "soonish"?
Josef
Is there any prospect of a 64 bit installer for the full release?
Can I help? I have a Windows 7 64 bit machine that I use as a build slave; I am happy to give access.
Cheers,
Matthew _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Sun, Feb 3, 2013 at 12:28 AM, josef.pktd@gmail.com wrote:
On Sat, Feb 2, 2013 at 6:14 PM, Matthew Brett matthew.brett@gmail.com wrote:
Hi,
I see there is no Windows 64 bit installer for the 1.7 rc1.
related: Is there any chance to get newer mingw or mingw-w64 support "soonish"?
The problem has no solution until we can restrict support to windows 7 and above. Otherwise, any acceptable solution would require user to be an admin.
David
On Sun, Feb 3, 2013 at 2:57 AM, David Cournapeau cournape@gmail.com wrote:
On Sun, Feb 3, 2013 at 12:28 AM, josef.pktd@gmail.com wrote:
On Sat, Feb 2, 2013 at 6:14 PM, Matthew Brett matthew.brett@gmail.com wrote:
Hi,
I see there is no Windows 64 bit installer for the 1.7 rc1.
related: Is there any chance to get newer mingw or mingw-w64 support "soonish"?
The problem has no solution until we can restrict support to windows 7 and above. Otherwise, any acceptable solution would require user to be an admin.
The installer is built with this VM/scripts:
https://github.com/certik/numpy-vendor
currently the VM itself is 32 bit. I think that might be upgraded to 64bit, and maybe it's possible to use 64 bit Wine:
but then we would need to figure out how to use Mingw with 64 bits.
I would be very happy to accept patches to the above repository.
Alternatively, if the actual Windows 64bit machine would have to be used, is there any way to automate the process? Would you compile it from command line (cmd.exe), just like I do in Wine? I would much prefer if we can figure out how to do this in Wine, so that the process can be automated and other people can easily reproduce it.
Ondrej
Hi,
On Mon, Feb 4, 2013 at 12:27 PM, Ondřej Čertík ondrej.certik@gmail.com wrote:
On Sun, Feb 3, 2013 at 2:57 AM, David Cournapeau cournape@gmail.com wrote:
On Sun, Feb 3, 2013 at 12:28 AM, josef.pktd@gmail.com wrote:
On Sat, Feb 2, 2013 at 6:14 PM, Matthew Brett matthew.brett@gmail.com wrote:
Hi,
I see there is no Windows 64 bit installer for the 1.7 rc1.
related: Is there any chance to get newer mingw or mingw-w64 support "soonish"?
The problem has no solution until we can restrict support to windows 7 and above. Otherwise, any acceptable solution would require user to be an admin.
The installer is built with this VM/scripts:
https://github.com/certik/numpy-vendor
currently the VM itself is 32 bit. I think that might be upgraded to 64bit, and maybe it's possible to use 64 bit Wine:
but then we would need to figure out how to use Mingw with 64 bits.
I would be very happy to accept patches to the above repository.
Alternatively, if the actual Windows 64bit machine would have to be used, is there any way to automate the process? Would you compile it from command line (cmd.exe), just like I do in Wine? I would much prefer if we can figure out how to do this in Wine, so that the process can be automated and other people can easily reproduce it.
I wonder whether getting ming64 to work on 64 bit Wine is too hard to get working before the release? I often can't get 32-bit Wine working, and we've apparently got problems with mingw64 on native windows.
As a short term fix, how about an Amazon image with the Windows 64 bit compilers on it?
Or can Christophe Gohlke help us out here?
It seems a shame not to provide these builds.
Cheers,
Matthew
On Mon, Feb 4, 2013 at 12:36 PM, Matthew Brett matthew.brett@gmail.com wrote:
Hi,
On Mon, Feb 4, 2013 at 12:27 PM, Ondřej Čertík ondrej.certik@gmail.com wrote:
On Sun, Feb 3, 2013 at 2:57 AM, David Cournapeau cournape@gmail.com wrote:
On Sun, Feb 3, 2013 at 12:28 AM, josef.pktd@gmail.com wrote:
On Sat, Feb 2, 2013 at 6:14 PM, Matthew Brett matthew.brett@gmail.com wrote:
Hi,
I see there is no Windows 64 bit installer for the 1.7 rc1.
related: Is there any chance to get newer mingw or mingw-w64 support "soonish"?
The problem has no solution until we can restrict support to windows 7 and above. Otherwise, any acceptable solution would require user to be an admin.
The installer is built with this VM/scripts:
https://github.com/certik/numpy-vendor
currently the VM itself is 32 bit. I think that might be upgraded to 64bit, and maybe it's possible to use 64 bit Wine:
but then we would need to figure out how to use Mingw with 64 bits.
I would be very happy to accept patches to the above repository.
Alternatively, if the actual Windows 64bit machine would have to be used, is there any way to automate the process? Would you compile it from command line (cmd.exe), just like I do in Wine? I would much prefer if we can figure out how to do this in Wine, so that the process can be automated and other people can easily reproduce it.
I wonder whether getting ming64 to work on 64 bit Wine is too hard to get working before the release? I often can't get 32-bit Wine working, and we've apparently got problems with mingw64 on native windows.
As a short term fix, how about an Amazon image with the Windows 64 bit compilers on it?
Or can Christophe Gohlke help us out here?
As a temporary measure it might make sense to just deem the cgolke builds official and upload them to the usual places -- or at least, it seems like it might make sense to me, but it depends on what Christophe thinks :-).
-n
On Mon, Feb 4, 2013 at 9:39 PM, Nathaniel Smith njs@pobox.com wrote:
On Mon, Feb 4, 2013 at 12:36 PM, Matthew Brett matthew.brett@gmail.com wrote:
Hi,
On Mon, Feb 4, 2013 at 12:27 PM, Ondřej Čertík ondrej.certik@gmail.com
wrote:
On Sun, Feb 3, 2013 at 2:57 AM, David Cournapeau cournape@gmail.com
wrote:
On Sun, Feb 3, 2013 at 12:28 AM, josef.pktd@gmail.com wrote:
On Sat, Feb 2, 2013 at 6:14 PM, Matthew Brett <
matthew.brett@gmail.com> wrote:
Hi,
I see there is no Windows 64 bit installer for the 1.7 rc1.
related: Is there any chance to get newer mingw or mingw-w64 support "soonish"?
The problem has no solution until we can restrict support to windows 7 and above. Otherwise, any acceptable solution would require user to be an admin.
The installer is built with this VM/scripts:
https://github.com/certik/numpy-vendor
currently the VM itself is 32 bit. I think that might be upgraded to
64bit,
and maybe it's possible to use 64 bit Wine:
but then we would need to figure out how to use Mingw with 64 bits.
I would be very happy to accept patches to the above repository.
Alternatively, if the actual Windows 64bit machine would have to be
used,
is there any way to automate the process? Would you compile it from
command line
(cmd.exe), just like I do in Wine? I would much prefer if we can figure
out
how to do this in Wine, so that the process can be automated and other
people
can easily reproduce it.
I wonder whether getting ming64 to work on 64 bit Wine is too hard to get working before the release? I often can't get 32-bit Wine working, and we've apparently got problems with mingw64 on native windows.
As a short term fix, how about an Amazon image with the Windows 64 bit compilers on it?
-1 on providing an "official" solution which will require admin rights for the produced installer and not work for scipy.
Or can Christophe Gohlke help us out here?
As a temporary measure it might make sense to just deem the cgolke builds official and upload them to the usual places -- or at least, it seems like it might make sense to me, but it depends on what Christophe thinks :-).
I'm +0 on that. MSVC + MKL is currently the only real option for 64-bit binaries, so providing Christoph's installers as "official" could make sense. On the other hand, Christoph has a matching set of other packages on his site, so users will be better off being redirected there than just grabbing only a numpy installer from SF and not finding a scipy one there.
Ralf
Hi,
On Mon, Feb 4, 2013 at 12:55 PM, Ralf Gommers ralf.gommers@gmail.com wrote:
On Mon, Feb 4, 2013 at 9:39 PM, Nathaniel Smith njs@pobox.com wrote:
On Mon, Feb 4, 2013 at 12:36 PM, Matthew Brett matthew.brett@gmail.com wrote:
Hi,
On Mon, Feb 4, 2013 at 12:27 PM, Ondřej Čertík ondrej.certik@gmail.com wrote:
On Sun, Feb 3, 2013 at 2:57 AM, David Cournapeau cournape@gmail.com wrote:
On Sun, Feb 3, 2013 at 12:28 AM, josef.pktd@gmail.com wrote:
On Sat, Feb 2, 2013 at 6:14 PM, Matthew Brett matthew.brett@gmail.com wrote: > Hi, > > I see there is no Windows 64 bit installer for the 1.7 rc1.
related: Is there any chance to get newer mingw or mingw-w64 support "soonish"?
The problem has no solution until we can restrict support to windows 7 and above. Otherwise, any acceptable solution would require user to be an admin.
The installer is built with this VM/scripts:
https://github.com/certik/numpy-vendor
currently the VM itself is 32 bit. I think that might be upgraded to 64bit, and maybe it's possible to use 64 bit Wine:
but then we would need to figure out how to use Mingw with 64 bits.
I would be very happy to accept patches to the above repository.
Alternatively, if the actual Windows 64bit machine would have to be used, is there any way to automate the process? Would you compile it from command line (cmd.exe), just like I do in Wine? I would much prefer if we can figure out how to do this in Wine, so that the process can be automated and other people can easily reproduce it.
I wonder whether getting ming64 to work on 64 bit Wine is too hard to get working before the release? I often can't get 32-bit Wine working, and we've apparently got problems with mingw64 on native windows.
As a short term fix, how about an Amazon image with the Windows 64 bit compilers on it?
-1 on providing an "official" solution which will require admin rights for the produced installer and not work for scipy.
Sorry if I am being slow, but I don't follow.
Am I right in thinking that we can currently build numpy 64 bit installers with the Microsoft tools, and that these would be distributable without admin rights for windows >=XP ?
Why would this solution not work for scipy?
Cheers,
Matthew
On Mon, Feb 4, 2013 at 10:06 PM, Matthew Brett matthew.brett@gmail.comwrote:
Hi,
On Mon, Feb 4, 2013 at 12:55 PM, Ralf Gommers ralf.gommers@gmail.com wrote:
On Mon, Feb 4, 2013 at 9:39 PM, Nathaniel Smith njs@pobox.com wrote:
On Mon, Feb 4, 2013 at 12:36 PM, Matthew Brett <matthew.brett@gmail.com
wrote:
Hi,
On Mon, Feb 4, 2013 at 12:27 PM, Ondřej Čertík <
ondrej.certik@gmail.com>
wrote:
On Sun, Feb 3, 2013 at 2:57 AM, David Cournapeau <cournape@gmail.com
wrote:
On Sun, Feb 3, 2013 at 12:28 AM, josef.pktd@gmail.com wrote: > On Sat, Feb 2, 2013 at 6:14 PM, Matthew Brett > matthew.brett@gmail.com wrote: >> Hi, >> >> I see there is no Windows 64 bit installer for the 1.7 rc1. > > related: > Is there any chance to get newer mingw or mingw-w64 support > "soonish"?
The problem has no solution until we can restrict support to
windows 7
and above. Otherwise, any acceptable solution would require user to
be
an admin.
The installer is built with this VM/scripts:
https://github.com/certik/numpy-vendor
currently the VM itself is 32 bit. I think that might be upgraded to 64bit, and maybe it's possible to use 64 bit Wine:
but then we would need to figure out how to use Mingw with 64 bits.
I would be very happy to accept patches to the above repository.
Alternatively, if the actual Windows 64bit machine would have to be used, is there any way to automate the process? Would you compile it from command line (cmd.exe), just like I do in Wine? I would much prefer if we can
figure
out how to do this in Wine, so that the process can be automated and
other
people can easily reproduce it.
I wonder whether getting ming64 to work on 64 bit Wine is too hard to get working before the release? I often can't get 32-bit Wine working, and we've apparently got problems with mingw64 on native windows.
As a short term fix, how about an Amazon image with the Windows 64 bit compilers on it?
-1 on providing an "official" solution which will require admin rights
for
the produced installer and not work for scipy.
Sorry if I am being slow, but I don't follow.
Am I right in thinking that we can currently build numpy 64 bit installers with the Microsoft tools, and that these would be distributable without admin rights for windows >=XP ?
MSVC + Intel Fortran + MKL, yes. But those aren't free. So how can you provide an Amazon image for those?
Why would this solution not work for scipy?
It would. gfortran doesn't. Looking at your mail, I still read it as providing an image with mingw64 + gfortran.
Ralf
On Mon, Feb 4, 2013 at 4:15 PM, Ralf Gommers ralf.gommers@gmail.com wrote:
On Mon, Feb 4, 2013 at 10:06 PM, Matthew Brett matthew.brett@gmail.com wrote:
Hi,
On Mon, Feb 4, 2013 at 12:55 PM, Ralf Gommers ralf.gommers@gmail.com wrote:
On Mon, Feb 4, 2013 at 9:39 PM, Nathaniel Smith njs@pobox.com wrote:
On Mon, Feb 4, 2013 at 12:36 PM, Matthew Brett matthew.brett@gmail.com wrote:
Hi,
On Mon, Feb 4, 2013 at 12:27 PM, Ondřej Čertík ondrej.certik@gmail.com wrote:
On Sun, Feb 3, 2013 at 2:57 AM, David Cournapeau cournape@gmail.com wrote: > On Sun, Feb 3, 2013 at 12:28 AM, josef.pktd@gmail.com wrote: >> On Sat, Feb 2, 2013 at 6:14 PM, Matthew Brett >> matthew.brett@gmail.com wrote: >>> Hi, >>> >>> I see there is no Windows 64 bit installer for the 1.7 rc1. >> >> related: >> Is there any chance to get newer mingw or mingw-w64 support >> "soonish"? > > The problem has no solution until we can restrict support to > windows 7 > and above. Otherwise, any acceptable solution would require user to > be > an admin.
The installer is built with this VM/scripts:
https://github.com/certik/numpy-vendor
currently the VM itself is 32 bit. I think that might be upgraded to 64bit, and maybe it's possible to use 64 bit Wine:
but then we would need to figure out how to use Mingw with 64 bits.
I would be very happy to accept patches to the above repository.
Alternatively, if the actual Windows 64bit machine would have to be used, is there any way to automate the process? Would you compile it from command line (cmd.exe), just like I do in Wine? I would much prefer if we can figure out how to do this in Wine, so that the process can be automated and other people can easily reproduce it.
I wonder whether getting ming64 to work on 64 bit Wine is too hard to get working before the release? I often can't get 32-bit Wine working, and we've apparently got problems with mingw64 on native windows.
As a short term fix, how about an Amazon image with the Windows 64 bit compilers on it?
-1 on providing an "official" solution which will require admin rights for the produced installer and not work for scipy.
Sorry if I am being slow, but I don't follow.
Am I right in thinking that we can currently build numpy 64 bit installers with the Microsoft tools, and that these would be distributable without admin rights for windows >=XP ?
MSVC + Intel Fortran + MKL, yes. But those aren't free. So how can you provide an Amazon image for those?
Related question Would scipy and similar packages complain when we try to build them without MKL against an MKL numpy. Gohlke has the compatibility notes to warn users. If incompatible files are on numpy sourceforge for download, then users might install by accident incompatible versions, or not?
Josef
Why would this solution not work for scipy?
It would. gfortran doesn't. Looking at your mail, I still read it as providing an image with mingw64 + gfortran.
Ralf
NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Hi,
On Mon, Feb 4, 2013 at 1:15 PM, Ralf Gommers ralf.gommers@gmail.com wrote:
On Mon, Feb 4, 2013 at 10:06 PM, Matthew Brett matthew.brett@gmail.com wrote:
Hi,
On Mon, Feb 4, 2013 at 12:55 PM, Ralf Gommers ralf.gommers@gmail.com wrote:
On Mon, Feb 4, 2013 at 9:39 PM, Nathaniel Smith njs@pobox.com wrote:
On Mon, Feb 4, 2013 at 12:36 PM, Matthew Brett matthew.brett@gmail.com wrote:
Hi,
On Mon, Feb 4, 2013 at 12:27 PM, Ondřej Čertík ondrej.certik@gmail.com wrote:
On Sun, Feb 3, 2013 at 2:57 AM, David Cournapeau cournape@gmail.com wrote: > On Sun, Feb 3, 2013 at 12:28 AM, josef.pktd@gmail.com wrote: >> On Sat, Feb 2, 2013 at 6:14 PM, Matthew Brett >> matthew.brett@gmail.com wrote: >>> Hi, >>> >>> I see there is no Windows 64 bit installer for the 1.7 rc1. >> >> related: >> Is there any chance to get newer mingw or mingw-w64 support >> "soonish"? > > The problem has no solution until we can restrict support to > windows 7 > and above. Otherwise, any acceptable solution would require user to > be > an admin.
The installer is built with this VM/scripts:
https://github.com/certik/numpy-vendor
currently the VM itself is 32 bit. I think that might be upgraded to 64bit, and maybe it's possible to use 64 bit Wine:
but then we would need to figure out how to use Mingw with 64 bits.
I would be very happy to accept patches to the above repository.
Alternatively, if the actual Windows 64bit machine would have to be used, is there any way to automate the process? Would you compile it from command line (cmd.exe), just like I do in Wine? I would much prefer if we can figure out how to do this in Wine, so that the process can be automated and other people can easily reproduce it.
I wonder whether getting ming64 to work on 64 bit Wine is too hard to get working before the release? I often can't get 32-bit Wine working, and we've apparently got problems with mingw64 on native windows.
As a short term fix, how about an Amazon image with the Windows 64 bit compilers on it?
-1 on providing an "official" solution which will require admin rights for the produced installer and not work for scipy.
Sorry if I am being slow, but I don't follow.
Am I right in thinking that we can currently build numpy 64 bit installers with the Microsoft tools, and that these would be distributable without admin rights for windows >=XP ?
MSVC + Intel Fortran + MKL, yes. But those aren't free. So how can you provide an Amazon image for those?
You can make an image that is not public, I guess. I suppose anyone who uses the image would have to have their own licenses for the Intel stuff? Does anyone have experience of this?
Does ATLAS / openBLAS not build for windows? Sorry for my ignorance.
Cheers,
Matthew
On Mon, Feb 4, 2013 at 10:38 PM, Matthew Brett matthew.brett@gmail.com wrote:
Hi,
On Mon, Feb 4, 2013 at 1:15 PM, Ralf Gommers ralf.gommers@gmail.com wrote:
MSVC + Intel Fortran + MKL, yes. But those aren't free. So how can you provide an Amazon image for those?
You can make an image that is not public, I guess. I suppose anyone who uses the image would have to have their own licenses for the Intel stuff? Does anyone have experience of this?
You need to purchase one license per developer:
http://software.intel.com/en-us/articles/intel-math-kernel-library-licensing...
-- Robert Kern
On Mon, Feb 4, 2013 at 4:04 PM, Robert Kern robert.kern@gmail.com wrote:
On Mon, Feb 4, 2013 at 10:38 PM, Matthew Brett matthew.brett@gmail.com wrote:
Hi,
On Mon, Feb 4, 2013 at 1:15 PM, Ralf Gommers ralf.gommers@gmail.com
wrote:
MSVC + Intel Fortran + MKL, yes. But those aren't free. So how can you provide an Amazon image for those?
You can make an image that is not public, I guess. I suppose anyone who uses the image would have to have their own licenses for the Intel stuff? Does anyone have experience of this?
You need to purchase one license per developer:
http://software.intel.com/en-us/articles/intel-math-kernel-library-licensing...
I think 64 bits on windows is best pushed off to 1.7.1 or 1.8. It would be a bit much to get it implemented in the next week or two.
Chuck
Hi,
On Mon, Feb 4, 2013 at 3:46 PM, Charles R Harris charlesr.harris@gmail.com wrote:
On Mon, Feb 4, 2013 at 4:04 PM, Robert Kern robert.kern@gmail.com wrote:
On Mon, Feb 4, 2013 at 10:38 PM, Matthew Brett matthew.brett@gmail.com wrote:
Hi,
On Mon, Feb 4, 2013 at 1:15 PM, Ralf Gommers ralf.gommers@gmail.com wrote:
MSVC + Intel Fortran + MKL, yes. But those aren't free. So how can you provide an Amazon image for those?
You can make an image that is not public, I guess. I suppose anyone who uses the image would have to have their own licenses for the Intel stuff? Does anyone have experience of this?
You need to purchase one license per developer:
http://software.intel.com/en-us/articles/intel-math-kernel-library-licensing...
I think 64 bits on windows is best pushed off to 1.7.1 or 1.8. It would be a bit much to get it implemented in the next week or two.
The problem with not providing these binaries is that they are at the bottom of everyone's stack, so a delay in numpy holds everyone back.
I can't find completely convincing stats, but it looks as though 64 bit windows 7 is now the most common version of Windows, at least for Gamers [1] around now, and it was getting that way for everyone in 2010 [2].
It don't think it reflects well on on us that we don't appear to support 64 bits out of the box; just for example, R already has a 32 bit / 64 bit installer.
If I understand correctly, the options for doing this right now are:
1) Minimal cost in time : ask Christophe nicely whether we can distribute his binaries via the Numpy page 2) Small cost in time / money : pay for licenses for Ondrej or me or someone to install the dependencies on my Berkeley machine / an Amazon image.
Ralf : I suppose we qualify for the free licenses you referred to earlier ? [3] . I guess that covers us for the Numpy build? Then it's only a question of paying for ifort licenses when it comes to do the Scipy build?
So, if the cost of option 2 is too high, how about option 1?
Cheers,
Matthew
[1] http://store.steampowered.com/hwsurvey [2] http://blogs.windows.com/windows/b/bloggingwindows/archive/2010/07/08/64-bit... [3] http://numpy-discussion.10968.n7.nabble.com/MKL-licenses-for-core-scientific...
Hi,
On Mon, Feb 4, 2013 at 5:09 PM, Matthew Brett matthew.brett@gmail.com wrote:
Hi,
On Mon, Feb 4, 2013 at 3:46 PM, Charles R Harris charlesr.harris@gmail.com wrote:
On Mon, Feb 4, 2013 at 4:04 PM, Robert Kern robert.kern@gmail.com wrote:
On Mon, Feb 4, 2013 at 10:38 PM, Matthew Brett matthew.brett@gmail.com wrote:
Hi,
On Mon, Feb 4, 2013 at 1:15 PM, Ralf Gommers ralf.gommers@gmail.com wrote:
MSVC + Intel Fortran + MKL, yes. But those aren't free. So how can you provide an Amazon image for those?
You can make an image that is not public, I guess. I suppose anyone who uses the image would have to have their own licenses for the Intel stuff? Does anyone have experience of this?
You need to purchase one license per developer:
http://software.intel.com/en-us/articles/intel-math-kernel-library-licensing...
I think 64 bits on windows is best pushed off to 1.7.1 or 1.8. It would be a bit much to get it implemented in the next week or two.
The problem with not providing these binaries is that they are at the bottom of everyone's stack, so a delay in numpy holds everyone back.
I can't find completely convincing stats, but it looks as though 64 bit windows 7 is now the most common version of Windows, at least for Gamers [1] around now, and it was getting that way for everyone in 2010 [2].
It don't think it reflects well on on us that we don't appear to support 64 bits out of the box; just for example, R already has a 32 bit / 64 bit installer.
If I understand correctly, the options for doing this right now are:
- Minimal cost in time : ask Christophe nicely whether we can
distribute his binaries via the Numpy page 2) Small cost in time / money : pay for licenses for Ondrej or me or someone to install the dependencies on my Berkeley machine / an Amazon image.
David - obviously if you were willing to do this - that would be far preferable to me stumbling along. Can provide machine and beer IOUs.
Cheers,
Matthew
On 02/04/2013 06:09 PM, Matthew Brett wrote:
The problem with not providing these binaries is that they are at the bottom of everyone's stack, so a delay in numpy holds everyone back.
OTOH, so far it's been an *excellent* excuse for those of us further up the stack not to make a 64-bit binary. It sounds like we're completely out of luck on 64-bit Windows with free tools? So the rest of the community is going to face shelling out for compiler licenses as well?
Hi,
On Tue, Feb 5, 2013 at 6:03 AM, Jonathan T. Niehof jniehof@lanl.gov wrote:
On 02/04/2013 06:09 PM, Matthew Brett wrote:
The problem with not providing these binaries is that they are at the bottom of everyone's stack, so a delay in numpy holds everyone back.
OTOH, so far it's been an *excellent* excuse for those of us further up the stack not to make a 64-bit binary. It sounds like we're completely out of luck on 64-bit Windows with free tools? So the rest of the community is going to face shelling out for compiler licenses as well?
Normally you'd only need the free (as in beer) MS C compilers. You'd need a (possibly free) MKL license if you wanted to link against an optimized blas / lapack (it seems), and you'd need to use the Intel ifort compiler if you had fortran code in there (I think).
Cheers,
Matthew
Hi,
On Mon, Feb 4, 2013 at 5:09 PM, Matthew Brett matthew.brett@gmail.com wrote:
Hi,
On Mon, Feb 4, 2013 at 3:46 PM, Charles R Harris charlesr.harris@gmail.com wrote:
On Mon, Feb 4, 2013 at 4:04 PM, Robert Kern robert.kern@gmail.com wrote:
On Mon, Feb 4, 2013 at 10:38 PM, Matthew Brett matthew.brett@gmail.com wrote:
Hi,
On Mon, Feb 4, 2013 at 1:15 PM, Ralf Gommers ralf.gommers@gmail.com wrote:
MSVC + Intel Fortran + MKL, yes. But those aren't free. So how can you provide an Amazon image for those?
You can make an image that is not public, I guess. I suppose anyone who uses the image would have to have their own licenses for the Intel stuff? Does anyone have experience of this?
You need to purchase one license per developer:
http://software.intel.com/en-us/articles/intel-math-kernel-library-licensing...
I think 64 bits on windows is best pushed off to 1.7.1 or 1.8. It would be a bit much to get it implemented in the next week or two.
The problem with not providing these binaries is that they are at the bottom of everyone's stack, so a delay in numpy holds everyone back.
I can't find completely convincing stats, but it looks as though 64 bit windows 7 is now the most common version of Windows, at least for Gamers [1] around now, and it was getting that way for everyone in 2010 [2].
It don't think it reflects well on on us that we don't appear to support 64 bits out of the box; just for example, R already has a 32 bit / 64 bit installer.
If I understand correctly, the options for doing this right now are:
- Minimal cost in time : ask Christophe nicely whether we can
distribute his binaries via the Numpy page 2) Small cost in time / money : pay for licenses for Ondrej or me or someone to install the dependencies on my Berkeley machine / an Amazon image.
In order not to leave this discussion without a resolution:
Christophe - would you allow us to distribute your numpy binaries for 1.7 from the numpy sourceforge page?
Cheers,
Matthew
On 2/5/2013 10:51 AM, Matthew Brett wrote:
Hi,
On Mon, Feb 4, 2013 at 5:09 PM, Matthew Brett matthew.brett@gmail.com wrote:
Hi,
On Mon, Feb 4, 2013 at 3:46 PM, Charles R Harris charlesr.harris@gmail.com wrote:
On Mon, Feb 4, 2013 at 4:04 PM, Robert Kern robert.kern@gmail.com wrote:
On Mon, Feb 4, 2013 at 10:38 PM, Matthew Brett matthew.brett@gmail.com wrote:
Hi,
On Mon, Feb 4, 2013 at 1:15 PM, Ralf Gommers ralf.gommers@gmail.com wrote:
MSVC + Intel Fortran + MKL, yes. But those aren't free. So how can you provide an Amazon image for those?
You can make an image that is not public, I guess. I suppose anyone who uses the image would have to have their own licenses for the Intel stuff? Does anyone have experience of this?
You need to purchase one license per developer:
http://software.intel.com/en-us/articles/intel-math-kernel-library-licensing...
I think 64 bits on windows is best pushed off to 1.7.1 or 1.8. It would be a bit much to get it implemented in the next week or two.
The problem with not providing these binaries is that they are at the bottom of everyone's stack, so a delay in numpy holds everyone back.
I can't find completely convincing stats, but it looks as though 64 bit windows 7 is now the most common version of Windows, at least for Gamers [1] around now, and it was getting that way for everyone in 2010 [2].
It don't think it reflects well on on us that we don't appear to support 64 bits out of the box; just for example, R already has a 32 bit / 64 bit installer.
If I understand correctly, the options for doing this right now are:
- Minimal cost in time : ask Christophe nicely whether we can
distribute his binaries via the Numpy page 2) Small cost in time / money : pay for licenses for Ondrej or me or someone to install the dependencies on my Berkeley machine / an Amazon image.
In order not to leave this discussion without a resolution:
Christophe - would you allow us to distribute your numpy binaries for 1.7 from the numpy sourceforge page?
Cheers,
Matthew
I am OK with providing 64 bit "numpy-MKL" binaries (that is numpy compiled with MSVC compilers and linked to Intel's MKL) for official numpy releases.
However:
1) There seems to be no real consensus and urge for doing this. Using a free toolchain capable of building the whole scipy-stack would be much preferred. Several 64 bit Python distributions containing numpy-MKL are already available, some for free.
2) Releasing 64 bit numpy without matching scipy binaries would make little sense to me.
3) Please do not just redistribute the binaries from my website and declare them official. They might contain unreleased fixes from git master and pull requests that are needed for my work and other packages.
4) Numpy-MKL requires the Intel runtime DLLs (MKL is linked statically btw). I ship those with the installers and append the directory containing the DLLs to os.environ['PATH'] in numpy/__init__.py. This is a big no-no according to numpy developers. I don't agree. Anyway, those changes are not in the numpy source repositories.
5) My numpy-MKL installers are Python distutils bdist_wininst installers. That means if Python was installed for all users, installing numpy-MKL on Windows >6.0 will prompt for UAC elevation. Another no-no?
Christoph
Hi,
On Tue, Feb 5, 2013 at 3:04 PM, Christoph Gohlke cgohlke@uci.edu wrote:
On 2/5/2013 10:51 AM, Matthew Brett wrote:
Hi,
On Mon, Feb 4, 2013 at 5:09 PM, Matthew Brett matthew.brett@gmail.com wrote:
Hi,
On Mon, Feb 4, 2013 at 3:46 PM, Charles R Harris charlesr.harris@gmail.com wrote:
On Mon, Feb 4, 2013 at 4:04 PM, Robert Kern robert.kern@gmail.com wrote:
On Mon, Feb 4, 2013 at 10:38 PM, Matthew Brett matthew.brett@gmail.com wrote:
Hi,
On Mon, Feb 4, 2013 at 1:15 PM, Ralf Gommers ralf.gommers@gmail.com wrote:
> MSVC + Intel Fortran + MKL, yes. But those aren't free. So how can you > provide an Amazon image for those?
You can make an image that is not public, I guess. I suppose anyone who uses the image would have to have their own licenses for the Intel stuff? Does anyone have experience of this?
You need to purchase one license per developer:
http://software.intel.com/en-us/articles/intel-math-kernel-library-licensing...
I think 64 bits on windows is best pushed off to 1.7.1 or 1.8. It would be a bit much to get it implemented in the next week or two.
The problem with not providing these binaries is that they are at the bottom of everyone's stack, so a delay in numpy holds everyone back.
I can't find completely convincing stats, but it looks as though 64 bit windows 7 is now the most common version of Windows, at least for Gamers [1] around now, and it was getting that way for everyone in 2010 [2].
It don't think it reflects well on on us that we don't appear to support 64 bits out of the box; just for example, R already has a 32 bit / 64 bit installer.
If I understand correctly, the options for doing this right now are:
- Minimal cost in time : ask Christophe nicely whether we can
distribute his binaries via the Numpy page 2) Small cost in time / money : pay for licenses for Ondrej or me or someone to install the dependencies on my Berkeley machine / an Amazon image.
In order not to leave this discussion without a resolution:
Christophe - would you allow us to distribute your numpy binaries for 1.7 from the numpy sourceforge page?
Cheers,
Matthew
I am OK with providing 64 bit "numpy-MKL" binaries (that is numpy compiled with MSVC compilers and linked to Intel's MKL) for official numpy releases.
Thank you - that is great.
However:
- There seems to be no real consensus and urge for doing this.
I certainly feel the urge and feel it strongly. As a packager for two or three projects myself, it's a royal pain having to tell someone to go to two different places for binaries depending on the number of bits of their Windows. I think Chuck was worried about the time it would take to do it, and I think you've already solved this problem. Ralf was worried about Scipy - see below.
Using a free toolchain capable of building the whole scipy-stack would be much preferred.
That's true, but there seems general agreement this is not practical in the very near future.
Several 64 bit Python distributions containing numpy-MKL are already available, some for free.
You mean EPD and AnacondaCE? I don't think we should withhold easily available vanilla builds because they are also available in company-sponsored projects. Python.org provides windows builds even though ActiveState is free-as-in-beer.
- Releasing 64 bit numpy without matching scipy binaries would make
little sense to me.
Would you consider also releasing your scipy binaries?
- Please do not just redistribute the binaries from my website and
declare them official. They might contain unreleased fixes from git master and pull requests that are needed for my work and other packages.
Right - would you consider then being the release provider for numpy / scipy binaries on windows, much as it appears that Martin v Lowis supplies Windows builds for Python?
- Numpy-MKL requires the Intel runtime DLLs (MKL is linked statically
btw). I ship those with the installers and append the directory containing the DLLs to os.environ['PATH'] in numpy/__init__.py. This is a big no-no according to numpy developers. I don't agree. Anyway, those changes are not in the numpy source repositories.
- My numpy-MKL installers are Python distutils bdist_wininst
installers. That means if Python was installed for all users, installing numpy-MKL on Windows >6.0 will prompt for UAC elevation. Another no-no?
I defer to others on these ones,
Thanks a lot,
Matthew
On Tue, Feb 5, 2013 at 4:32 PM, Matthew Brett matthew.brett@gmail.com wrote:
- Numpy-MKL requires the Intel runtime DLLs (MKL is linked statically
btw). I ship those with the installers and append the directory containing the DLLs to os.environ['PATH'] in numpy/__init__.py. This is a big no-no according to numpy developers. I don't agree. Anyway, those changes are not in the numpy source repositories.
I think you pointed out that another option is to load the dlls with ctypes -- is it much work to make that change?
- My numpy-MKL installers are Python distutils bdist_wininst
installers. That means if Python was installed for all users, installing numpy-MKL on Windows >6.0 will prompt for UAC elevation. Another no-no?
not sure about the UAC elevation -- but:
1) most folks use bdist_wininst for Windows binaries -- including the current numpy builds, and python.org python -- yes?
2) UAC aside, It would be great to have binaries that could be used with virtualenv -- binary eggs?
-Chris
Hi,
On Tue, Feb 5, 2013 at 4:55 PM, Chris Barker - NOAA Federal chris.barker@noaa.gov wrote:
On Tue, Feb 5, 2013 at 4:32 PM, Matthew Brett matthew.brett@gmail.com wrote:
- Numpy-MKL requires the Intel runtime DLLs (MKL is linked statically
btw). I ship those with the installers and append the directory containing the DLLs to os.environ['PATH'] in numpy/__init__.py. This is a big no-no according to numpy developers. I don't agree. Anyway, those changes are not in the numpy source repositories.
I think you pointed out that another option is to load the dlls with ctypes -- is it much work to make that change?
- My numpy-MKL installers are Python distutils bdist_wininst
installers. That means if Python was installed for all users, installing numpy-MKL on Windows >6.0 will prompt for UAC elevation. Another no-no?
not sure about the UAC elevation -- but:
- most folks use bdist_wininst for Windows binaries -- including the
current numpy builds, and python.org python -- yes?
- UAC aside, It would be great to have binaries that could be used
with virtualenv -- binary eggs?
easy_install can install into virtualenvs from bdist_wininst installers, at least the ones I have built...
See you,
Matthew
On Tue, Feb 5, 2013 at 5:01 PM, Matthew Brett matthew.brett@gmail.com wrote:
easy_install can install into virtualenvs from bdist_wininst installers, at least the ones I have built...
really? cool! I never thought to try that!
Thanks, -Chris
On 6 February 2013 01:55, Chris Barker - NOAA Federal wrote:
On Tue, Feb 5, 2013 at 4:32 PM, Matthew Brett matthew.brett@gmail.com wrote:
- Numpy-MKL requires the Intel runtime DLLs (MKL is linked statically
btw). I ship those with the installers and append the directory containing the DLLs to os.environ['PATH'] in numpy/__init__.py. This is a big no-no according to numpy developers. I don't agree. Anyway, those changes are not in the numpy source repositories.
I think you pointed out that another option is to load the dlls with ctypes -- is it much work to make that change?
- My numpy-MKL installers are Python distutils bdist_wininst
installers. That means if Python was installed for all users, installing numpy-MKL on Windows >6.0 will prompt for UAC elevation. Another no-no?
not sure about the UAC elevation -- but:
- most folks use bdist_wininst for Windows binaries -- including the
current numpy builds, and python.org python -- yes?
Even the current approach is off-limits for the few haggards out there (like me at work), who can not update numpy/scipy/anything else that requires an installation procedure (like pretty much all the bdist_wininst distributions for Windows 64 bits/Python 64 bits). The only (partial) solution I found is to install at home and bring the site-packages folder on a USB stick with me at work.
Which is a bit sad overall: if I can make dozens of installers of my applications for my colleagues with InnoSetup/NSIS which do *not* require any UAC crap/elevation, what's stopping the bdist_wininst to do the same? The same holds (unfortunately) for the excellent distributions from Christoph Gohlke.
The fact that Windows 64 bits/Python 64 bits UAC-free installers are pretty much non-existent in the Python world does not play very nicely with the fact that 64 bits architectures have been available for 783648660236729 years.
I'd rather prefer if someone would upload to sourceforge/scipy.org/whatever a zipped folder containing numpy as it was in their site-packages directory. Now that would be a huge plus :-)
Andrea.
"Imagination Is The Only Weapon In The War Against Reality." http://www.infinity77.net
# ------------------------------------------------------------- # def ask_mailing_list_support(email):
if mention_platform_and_version() and include_sample_app(): send_message(email) else: install_malware() erase_hard_drives() # ------------------------------------------------------------- #
On Wed, Feb 6, 2013 at 1:32 AM, Matthew Brett matthew.brett@gmail.comwrote:
Hi,
On Tue, Feb 5, 2013 at 3:04 PM, Christoph Gohlke cgohlke@uci.edu wrote:
On 2/5/2013 10:51 AM, Matthew Brett wrote:
Hi,
On Mon, Feb 4, 2013 at 5:09 PM, Matthew Brett matthew.brett@gmail.com
wrote:
Hi,
On Mon, Feb 4, 2013 at 3:46 PM, Charles R Harris charlesr.harris@gmail.com wrote:
On Mon, Feb 4, 2013 at 4:04 PM, Robert Kern robert.kern@gmail.com
wrote:
On Mon, Feb 4, 2013 at 10:38 PM, Matthew Brett <
matthew.brett@gmail.com>
wrote: > Hi, > > On Mon, Feb 4, 2013 at 1:15 PM, Ralf Gommers <
ralf.gommers@gmail.com>
> wrote:
>> MSVC + Intel Fortran + MKL, yes. But those aren't free. So how can
you
>> provide an Amazon image for those? > > You can make an image that is not public, I guess. I suppose
anyone
> who uses the image would have to have their own licenses for the
Intel
> stuff? Does anyone have experience of this?
You need to purchase one license per developer:
http://software.intel.com/en-us/articles/intel-math-kernel-library-licensing...
I think 64 bits on windows is best pushed off to 1.7.1 or 1.8. It
would be a
bit much to get it implemented in the next week or two.
The problem with not providing these binaries is that they are at the bottom of everyone's stack, so a delay in numpy holds everyone back.
I can't find completely convincing stats, but it looks as though 64 bit windows 7 is now the most common version of Windows, at least for Gamers [1] around now, and it was getting that way for everyone in 2010 [2].
It don't think it reflects well on on us that we don't appear to support 64 bits out of the box; just for example, R already has a 32 bit / 64 bit installer.
If I understand correctly, the options for doing this right now are:
- Minimal cost in time : ask Christophe nicely whether we can
distribute his binaries via the Numpy page 2) Small cost in time / money : pay for licenses for Ondrej or me or someone to install the dependencies on my Berkeley machine / an Amazon image.
In order not to leave this discussion without a resolution:
Christophe - would you allow us to distribute your numpy binaries for 1.7 from the numpy sourceforge page?
Cheers,
Matthew
I am OK with providing 64 bit "numpy-MKL" binaries (that is numpy compiled with MSVC compilers and linked to Intel's MKL) for official numpy releases.
Thank you - that is great.
However:
- There seems to be no real consensus and urge for doing this.
I certainly feel the urge and feel it strongly. As a packager for two or three projects myself, it's a royal pain having to tell someone to go to two different places for binaries depending on the number of bits of their Windows.
If you're relying on .exe installers on SF, then you have to send your users to more places than that probably. Really the separate installers are a poor alternative to the available scientific distributions. And the more packages you need as a user, the more annoying these separate installers are.
I think Chuck was worried about the time it would take to do it, and I think you've already solved this problem. Ralf was worried about Scipy - see below.
Not just Scipy - that would be my worry number one, but the same holds for all packages based on Numpy. You double the number of Windows installers that every single project needs to provide.
Using a free toolchain capable of building the whole scipy-stack would be much preferred.
That's true, but there seems general agreement this is not practical in the very near future.
Several 64 bit Python distributions containing numpy-MKL are already available, some for free.
You mean EPD and AnacondaCE? I don't think we should withhold easily available vanilla builds because they are also available in company-sponsored projects. Python.org provides windows builds even though ActiveState is free-as-in-beer.
If the company-sponsored bit bothers you, there's also a 64-bit Python(x,y) now.
Ralf
- Releasing 64 bit numpy without matching scipy binaries would make
little sense to me.
Would you consider also releasing your scipy binaries?
- Please do not just redistribute the binaries from my website and
declare them official. They might contain unreleased fixes from git master and pull requests that are needed for my work and other packages.
Right - would you consider then being the release provider for numpy / scipy binaries on windows, much as it appears that Martin v Lowis supplies Windows builds for Python?
- Numpy-MKL requires the Intel runtime DLLs (MKL is linked statically
btw). I ship those with the installers and append the directory containing the DLLs to os.environ['PATH'] in numpy/__init__.py. This is a big no-no according to numpy developers. I don't agree. Anyway, those changes are not in the numpy source repositories.
- My numpy-MKL installers are Python distutils bdist_wininst
installers. That means if Python was installed for all users, installing numpy-MKL on Windows >6.0 will prompt for UAC elevation. Another no-no?
I defer to others on these ones,
Thanks a lot,
Matthew _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Hi,
On Wed, Feb 6, 2013 at 10:48 AM, Ralf Gommers ralf.gommers@gmail.com wrote:
On Wed, Feb 6, 2013 at 1:32 AM, Matthew Brett matthew.brett@gmail.com wrote:
Hi,
On Tue, Feb 5, 2013 at 3:04 PM, Christoph Gohlke cgohlke@uci.edu wrote:
On 2/5/2013 10:51 AM, Matthew Brett wrote:
Hi,
On Mon, Feb 4, 2013 at 5:09 PM, Matthew Brett matthew.brett@gmail.com wrote:
Hi,
On Mon, Feb 4, 2013 at 3:46 PM, Charles R Harris charlesr.harris@gmail.com wrote:
On Mon, Feb 4, 2013 at 4:04 PM, Robert Kern robert.kern@gmail.com wrote: > > On Mon, Feb 4, 2013 at 10:38 PM, Matthew Brett > matthew.brett@gmail.com > wrote: >> Hi, >> >> On Mon, Feb 4, 2013 at 1:15 PM, Ralf Gommers >> ralf.gommers@gmail.com >> wrote: > >>> MSVC + Intel Fortran + MKL, yes. But those aren't free. So how can >>> you >>> provide an Amazon image for those? >> >> You can make an image that is not public, I guess. I suppose >> anyone >> who uses the image would have to have their own licenses for the >> Intel >> stuff? Does anyone have experience of this? > > You need to purchase one license per developer: > > > > http://software.intel.com/en-us/articles/intel-math-kernel-library-licensing... >
I think 64 bits on windows is best pushed off to 1.7.1 or 1.8. It would be a bit much to get it implemented in the next week or two.
The problem with not providing these binaries is that they are at the bottom of everyone's stack, so a delay in numpy holds everyone back.
I can't find completely convincing stats, but it looks as though 64 bit windows 7 is now the most common version of Windows, at least for Gamers [1] around now, and it was getting that way for everyone in 2010 [2].
It don't think it reflects well on on us that we don't appear to support 64 bits out of the box; just for example, R already has a 32 bit / 64 bit installer.
If I understand correctly, the options for doing this right now are:
- Minimal cost in time : ask Christophe nicely whether we can
distribute his binaries via the Numpy page 2) Small cost in time / money : pay for licenses for Ondrej or me or someone to install the dependencies on my Berkeley machine / an Amazon image.
In order not to leave this discussion without a resolution:
Christophe - would you allow us to distribute your numpy binaries for 1.7 from the numpy sourceforge page?
Cheers,
Matthew
I am OK with providing 64 bit "numpy-MKL" binaries (that is numpy compiled with MSVC compilers and linked to Intel's MKL) for official numpy releases.
Thank you - that is great.
However:
- There seems to be no real consensus and urge for doing this.
I certainly feel the urge and feel it strongly. As a packager for two or three projects myself, it's a royal pain having to tell someone to go to two different places for binaries depending on the number of bits of their Windows.
If you're relying on .exe installers on SF, then you have to send your users to more places than that probably. Really the separate installers are a poor alternative to the available scientific distributions. And the more packages you need as a user, the more annoying these separate installers are.
I think Chuck was worried about the time it would take to do it, and I think you've already solved this problem. Ralf was worried about Scipy - see below.
Not just Scipy - that would be my worry number one, but the same holds for all packages based on Numpy. You double the number of Windows installers that every single project needs to provide.
Using a free toolchain capable of building the whole scipy-stack would be much preferred.
That's true, but there seems general agreement this is not practical in the very near future.
Several 64 bit Python distributions containing numpy-MKL are already available, some for free.
You mean EPD and AnacondaCE? I don't think we should withhold easily available vanilla builds because they are also available in company-sponsored projects. Python.org provides windows builds even though ActiveState is free-as-in-beer.
If the company-sponsored bit bothers you, there's also a 64-bit Python(x,y) now.
I'm sure you've seen that the question 'where are the 64-bit installers' comes up often for Numpy.
It seems to me that we'd have to have a good reason not provide them. There will always be some number of people like me who like to install the various parts by hand, and don't like using large packages, free or or open or not. For example, I don't use macports. At the moment, the reasons I see you are giving are:
1) Then everyone would have to provide 64-bit binaries 2) You should use a super-package instead
On those arguments, we should withdraw all binary installers. Or withdraw the 32-bit ones, on the basis that 64-bit is likely the more common now. Of course we shouldn't do that, because it will put off some measurable number of users, and convey the impression that the numpy stack is a bit shaky, because it cannot be easily installed without a monolithic build framework.
I think that's a real shame, and would harm numpy, to no benefit.
I guess I'm saying, please, please, pretty please, oh please oh please oh please can we have a Windows 64-bit installer?
Cheers,
Matthew
On Wed, Feb 6, 2013 at 8:46 PM, Matthew Brett matthew.brett@gmail.comwrote:
Hi,
On Wed, Feb 6, 2013 at 10:48 AM, Ralf Gommers ralf.gommers@gmail.com wrote:
On Wed, Feb 6, 2013 at 1:32 AM, Matthew Brett matthew.brett@gmail.com wrote:
Hi,
On Tue, Feb 5, 2013 at 3:04 PM, Christoph Gohlke cgohlke@uci.edu
wrote:
On 2/5/2013 10:51 AM, Matthew Brett wrote:
Hi,
On Mon, Feb 4, 2013 at 5:09 PM, Matthew Brett <
matthew.brett@gmail.com>
wrote:
Hi,
On Mon, Feb 4, 2013 at 3:46 PM, Charles R Harris charlesr.harris@gmail.com wrote: > > > On Mon, Feb 4, 2013 at 4:04 PM, Robert Kern <robert.kern@gmail.com
> wrote: >> >> On Mon, Feb 4, 2013 at 10:38 PM, Matthew Brett >> matthew.brett@gmail.com >> wrote: >>> Hi, >>> >>> On Mon, Feb 4, 2013 at 1:15 PM, Ralf Gommers >>> ralf.gommers@gmail.com >>> wrote: >> >>>> MSVC + Intel Fortran + MKL, yes. But those aren't free. So how
can
>>>> you >>>> provide an Amazon image for those? >>> >>> You can make an image that is not public, I guess. I suppose >>> anyone >>> who uses the image would have to have their own licenses for the >>> Intel >>> stuff? Does anyone have experience of this? >> >> You need to purchase one license per developer: >> >> >> >>
http://software.intel.com/en-us/articles/intel-math-kernel-library-licensing...
>> > > I think 64 bits on windows is best pushed off to 1.7.1 or 1.8. It > would be a > bit much to get it implemented in the next week or two.
The problem with not providing these binaries is that they are at
the
bottom of everyone's stack, so a delay in numpy holds everyone back.
I can't find completely convincing stats, but it looks as though 64 bit windows 7 is now the most common version of Windows, at least
for
Gamers [1] around now, and it was getting that way for everyone in 2010 [2].
It don't think it reflects well on on us that we don't appear to support 64 bits out of the box; just for example, R already has a 32 bit / 64 bit installer.
If I understand correctly, the options for doing this right now are:
- Minimal cost in time : ask Christophe nicely whether we can
distribute his binaries via the Numpy page 2) Small cost in time / money : pay for licenses for Ondrej or me or someone to install the dependencies on my Berkeley machine / an
Amazon
image.
In order not to leave this discussion without a resolution:
Christophe - would you allow us to distribute your numpy binaries for 1.7 from the numpy sourceforge page?
Cheers,
Matthew
I am OK with providing 64 bit "numpy-MKL" binaries (that is numpy compiled with MSVC compilers and linked to Intel's MKL) for official numpy releases.
Thank you - that is great.
However:
- There seems to be no real consensus and urge for doing this.
I certainly feel the urge and feel it strongly. As a packager for two or three projects myself, it's a royal pain having to tell someone to go to two different places for binaries depending on the number of bits of their Windows.
If you're relying on .exe installers on SF, then you have to send your
users
to more places than that probably. Really the separate installers are a
poor
alternative to the available scientific distributions. And the more
packages
you need as a user, the more annoying these separate installers are.
I think Chuck was worried about the time it would take to do it, and I think you've already solved this problem. Ralf was worried about Scipy - see below.
Not just Scipy - that would be my worry number one, but the same holds
for
all packages based on Numpy. You double the number of Windows installers that every single project needs to provide.
Using a free toolchain capable of building the whole scipy-stack would be much preferred.
That's true, but there seems general agreement this is not practical in the very near future.
Several 64 bit Python distributions containing numpy-MKL are already available, some for free.
You mean EPD and AnacondaCE? I don't think we should withhold easily available vanilla builds because they are also available in company-sponsored projects. Python.org provides windows builds even though ActiveState is free-as-in-beer.
If the company-sponsored bit bothers you, there's also a 64-bit
Python(x,y)
now.
I'm sure you've seen that the question 'where are the 64-bit installers' comes up often for Numpy.
It seems to me that we'd have to have a good reason not provide them. There will always be some number of people like me who like to install the various parts by hand, and don't like using large packages, free or or open or not. For example, I don't use macports. At the moment, the reasons I see you are giving are:
- Then everyone would have to provide 64-bit binaries
Indeed. And since many packages won't do that because there's no free compilers, users just get stuck a bit later in the "installing the stack" process. I'm sure providing the binaries will help some people, but I expect it to cause as many problems as it solves.
Maybe there's a volunteer to help with building official binaries for a large number of packages? If not, I'm simply not convinced that this will be a net gain for users.
2) You should use a super-package instead
On those arguments, we should withdraw all binary installers.
32-bit is a very different situation.
Or withdraw the 32-bit ones, on the basis that 64-bit is likely the more
common now. Of course we shouldn't do that, because it will put off some measurable number of users, and convey the impression that the numpy stack is a bit shaky, because it cannot be easily installed without a monolithic build framework.
The installation of Python-based stacks *is* a bit shaky, that's no big secret.
Ralf
I think that's a real shame, and would harm numpy, to no benefit.
I guess I'm saying, please, please, pretty please, oh please oh please oh please can we have a Windows 64-bit installer?
Hi,
On Wed, Feb 6, 2013 at 1:36 PM, Ralf Gommers ralf.gommers@gmail.com wrote:
On Wed, Feb 6, 2013 at 8:46 PM, Matthew Brett matthew.brett@gmail.com wrote:
Hi,
On Wed, Feb 6, 2013 at 10:48 AM, Ralf Gommers ralf.gommers@gmail.com wrote:
On Wed, Feb 6, 2013 at 1:32 AM, Matthew Brett matthew.brett@gmail.com wrote:
Hi,
On Tue, Feb 5, 2013 at 3:04 PM, Christoph Gohlke cgohlke@uci.edu wrote:
On 2/5/2013 10:51 AM, Matthew Brett wrote:
Hi,
On Mon, Feb 4, 2013 at 5:09 PM, Matthew Brett matthew.brett@gmail.com wrote: > Hi, > > On Mon, Feb 4, 2013 at 3:46 PM, Charles R Harris > charlesr.harris@gmail.com wrote: >> >> >> On Mon, Feb 4, 2013 at 4:04 PM, Robert Kern >> robert.kern@gmail.com >> wrote: >>> >>> On Mon, Feb 4, 2013 at 10:38 PM, Matthew Brett >>> matthew.brett@gmail.com >>> wrote: >>>> Hi, >>>> >>>> On Mon, Feb 4, 2013 at 1:15 PM, Ralf Gommers >>>> ralf.gommers@gmail.com >>>> wrote: >>> >>>>> MSVC + Intel Fortran + MKL, yes. But those aren't free. So how >>>>> can >>>>> you >>>>> provide an Amazon image for those? >>>> >>>> You can make an image that is not public, I guess. I suppose >>>> anyone >>>> who uses the image would have to have their own licenses for the >>>> Intel >>>> stuff? Does anyone have experience of this? >>> >>> You need to purchase one license per developer: >>> >>> >>> >>> >>> http://software.intel.com/en-us/articles/intel-math-kernel-library-licensing... >>> >> >> I think 64 bits on windows is best pushed off to 1.7.1 or 1.8. It >> would be a >> bit much to get it implemented in the next week or two. > > The problem with not providing these binaries is that they are at > the > bottom of everyone's stack, so a delay in numpy holds everyone > back. > > I can't find completely convincing stats, but it looks as though 64 > bit windows 7 is now the most common version of Windows, at least > for > Gamers [1] around now, and it was getting that way for everyone in > 2010 [2]. > > It don't think it reflects well on on us that we don't appear to > support 64 bits out of the box; just for example, R already has a > 32 > bit / 64 bit installer. > > If I understand correctly, the options for doing this right now > are: > > 1) Minimal cost in time : ask Christophe nicely whether we can > distribute his binaries via the Numpy page > 2) Small cost in time / money : pay for licenses for Ondrej or me > or > someone to install the dependencies on my Berkeley machine / an > Amazon > image.
In order not to leave this discussion without a resolution:
Christophe - would you allow us to distribute your numpy binaries for 1.7 from the numpy sourceforge page?
Cheers,
Matthew
I am OK with providing 64 bit "numpy-MKL" binaries (that is numpy compiled with MSVC compilers and linked to Intel's MKL) for official numpy releases.
Thank you - that is great.
However:
- There seems to be no real consensus and urge for doing this.
I certainly feel the urge and feel it strongly. As a packager for two or three projects myself, it's a royal pain having to tell someone to go to two different places for binaries depending on the number of bits of their Windows.
If you're relying on .exe installers on SF, then you have to send your users to more places than that probably. Really the separate installers are a poor alternative to the available scientific distributions. And the more packages you need as a user, the more annoying these separate installers are.
I think Chuck was worried about the time it would take to do it, and I think you've already solved this problem. Ralf was worried about Scipy - see below.
Not just Scipy - that would be my worry number one, but the same holds for all packages based on Numpy. You double the number of Windows installers that every single project needs to provide.
Using a free toolchain capable of building the whole scipy-stack would be much preferred.
That's true, but there seems general agreement this is not practical in the very near future.
Several 64 bit Python distributions containing numpy-MKL are already available, some for free.
You mean EPD and AnacondaCE? I don't think we should withhold easily available vanilla builds because they are also available in company-sponsored projects. Python.org provides windows builds even though ActiveState is free-as-in-beer.
If the company-sponsored bit bothers you, there's also a 64-bit Python(x,y) now.
I'm sure you've seen that the question 'where are the 64-bit installers' comes up often for Numpy.
It seems to me that we'd have to have a good reason not provide them. There will always be some number of people like me who like to install the various parts by hand, and don't like using large packages, free or or open or not. For example, I don't use macports. At the moment, the reasons I see you are giving are:
- Then everyone would have to provide 64-bit binaries
Indeed. And since many packages won't do that because there's no free compilers, users just get stuck a bit later in the "installing the stack" process. I'm sure providing the binaries will help some people, but I expect it to cause as many problems as it solves.
Can you clarify the people you think will get stuck? I think I'm right in saying that anyone with a C extension should be able to build them against numpy, by installing the free (as-in-beer) MS tools? So do you just mean people needing a Fortran compiler? That's a small constituency, I think.
It seems to me against the give-it-a-go spirit of open source to say 'sure we can build installers, but you might get stuck later on so we won't give them to you'. And, if we start providing installers, I suspect we'll find that people start solving the remaining issues much more quickly than would happen if we don't provide them.
Cheers,
Matthew
On Wed, Feb 6, 2013 at 3:16 PM, Matthew Brett matthew.brett@gmail.com wrote:
Can you clarify the people you think will get stuck? I think I'm right in saying that anyone with a C extension should be able to build them against numpy, by installing the free (as-in-beer) MS tools?
yup -- and that's the recommended (easiest) way to do it against the the python.org python.
and you can use mingw to compile extensions that will run with the python.org python (built with MSVC), can you not use mingw to build extensions that will work with a MSVC-build numpy?
So do you just mean people needing a Fortran compiler? That's a small constituency, I think.
particularly small overlap between needing fortran (and knowing how to deal with building it) and needing binaries of numpy....
if someone has a fortran-extension-building solution -- is it likeley not to work with a MSVC-built numpy?
And, if we start providing installers, I suspect we'll find that people start solving the remaining issues much more quickly than would happen if we don't provide them.
+1
-Chris
On 02/07/2013 12:16 AM, Matthew Brett wrote:
Hi,
On Wed, Feb 6, 2013 at 1:36 PM, Ralf Gommers ralf.gommers@gmail.com wrote:
On Wed, Feb 6, 2013 at 8:46 PM, Matthew Brett matthew.brett@gmail.com wrote:
Hi,
On Wed, Feb 6, 2013 at 10:48 AM, Ralf Gommers ralf.gommers@gmail.com wrote:
On Wed, Feb 6, 2013 at 1:32 AM, Matthew Brett matthew.brett@gmail.com wrote:
Hi,
On Tue, Feb 5, 2013 at 3:04 PM, Christoph Gohlke cgohlke@uci.edu wrote:
On 2/5/2013 10:51 AM, Matthew Brett wrote: > Hi, > > On Mon, Feb 4, 2013 at 5:09 PM, Matthew Brett > matthew.brett@gmail.com > wrote: >> Hi, >> >> On Mon, Feb 4, 2013 at 3:46 PM, Charles R Harris >> charlesr.harris@gmail.com wrote: >>> >>> >>> On Mon, Feb 4, 2013 at 4:04 PM, Robert Kern >>> robert.kern@gmail.com >>> wrote: >>>> >>>> On Mon, Feb 4, 2013 at 10:38 PM, Matthew Brett >>>> matthew.brett@gmail.com >>>> wrote: >>>>> Hi, >>>>> >>>>> On Mon, Feb 4, 2013 at 1:15 PM, Ralf Gommers >>>>> ralf.gommers@gmail.com >>>>> wrote: >>>> >>>>>> MSVC + Intel Fortran + MKL, yes. But those aren't free. So how >>>>>> can >>>>>> you >>>>>> provide an Amazon image for those? >>>>> >>>>> You can make an image that is not public, I guess. I suppose >>>>> anyone >>>>> who uses the image would have to have their own licenses for the >>>>> Intel >>>>> stuff? Does anyone have experience of this? >>>> >>>> You need to purchase one license per developer: >>>> >>>> >>>> >>>> >>>> http://software.intel.com/en-us/articles/intel-math-kernel-library-licensing... >>>> >>> >>> I think 64 bits on windows is best pushed off to 1.7.1 or 1.8. It >>> would be a >>> bit much to get it implemented in the next week or two. >> >> The problem with not providing these binaries is that they are at >> the >> bottom of everyone's stack, so a delay in numpy holds everyone >> back. >> >> I can't find completely convincing stats, but it looks as though 64 >> bit windows 7 is now the most common version of Windows, at least >> for >> Gamers [1] around now, and it was getting that way for everyone in >> 2010 [2]. >> >> It don't think it reflects well on on us that we don't appear to >> support 64 bits out of the box; just for example, R already has a >> 32 >> bit / 64 bit installer. >> >> If I understand correctly, the options for doing this right now >> are: >> >> 1) Minimal cost in time : ask Christophe nicely whether we can >> distribute his binaries via the Numpy page >> 2) Small cost in time / money : pay for licenses for Ondrej or me >> or >> someone to install the dependencies on my Berkeley machine / an >> Amazon >> image. > > In order not to leave this discussion without a resolution: > > Christophe - would you allow us to distribute your numpy binaries > for > 1.7 from the numpy sourceforge page? > > Cheers, > > Matthew
I am OK with providing 64 bit "numpy-MKL" binaries (that is numpy compiled with MSVC compilers and linked to Intel's MKL) for official numpy releases.
Thank you - that is great.
However:
- There seems to be no real consensus and urge for doing this.
I certainly feel the urge and feel it strongly. As a packager for two or three projects myself, it's a royal pain having to tell someone to go to two different places for binaries depending on the number of bits of their Windows.
If you're relying on .exe installers on SF, then you have to send your users to more places than that probably. Really the separate installers are a poor alternative to the available scientific distributions. And the more packages you need as a user, the more annoying these separate installers are.
I think Chuck was worried about the time it would take to do it, and I think you've already solved this problem. Ralf was worried about Scipy - see below.
Not just Scipy - that would be my worry number one, but the same holds for all packages based on Numpy. You double the number of Windows installers that every single project needs to provide.
Using a free toolchain capable of building the whole scipy-stack would be much preferred.
That's true, but there seems general agreement this is not practical in the very near future.
Several 64 bit Python distributions containing numpy-MKL are already available, some for free.
You mean EPD and AnacondaCE? I don't think we should withhold easily available vanilla builds because they are also available in company-sponsored projects. Python.org provides windows builds even though ActiveState is free-as-in-beer.
If the company-sponsored bit bothers you, there's also a 64-bit Python(x,y) now.
I'm sure you've seen that the question 'where are the 64-bit installers' comes up often for Numpy.
It seems to me that we'd have to have a good reason not provide them. There will always be some number of people like me who like to install the various parts by hand, and don't like using large packages, free or or open or not. For example, I don't use macports. At the moment, the reasons I see you are giving are:
- Then everyone would have to provide 64-bit binaries
Indeed. And since many packages won't do that because there's no free compilers, users just get stuck a bit later in the "installing the stack" process. I'm sure providing the binaries will help some people, but I expect it to cause as many problems as it solves.
Can you clarify the people you think will get stuck? I think I'm right in saying that anyone with a C extension should be able to build them against numpy, by installing the free (as-in-beer) MS tools? So do you just mean people needing a Fortran compiler? That's a small constituency, I think.
Off the top of my head there's SciPy and pymc...
Anyway, I'm butting in because I wish this discussion could separate between the user perspective and the developer perspective.
FWIW,
1) From a user's perspective, I don't understand this either. If you are already using a closed source, not-free-as-in-beer operating system, why would you not use (or buy!) a closed source, not-free-as-in-beer Fortran compiler?
2) BUT, the argument I've seen that I can at least understand is that the release manager should be able to do a release using only open source tools (even using Wine instead of Windows) and not rely on a limited number of licenses. And that the release manager must be able to perform all the official builds directly.
Dag Sverre
Hi,
On Wed, Feb 6, 2013 at 9:20 PM, Dag Sverre Seljebotn d.s.seljebotn@astro.uio.no wrote:
On 02/07/2013 12:16 AM, Matthew Brett wrote:
Hi,
On Wed, Feb 6, 2013 at 1:36 PM, Ralf Gommers ralf.gommers@gmail.com wrote:
On Wed, Feb 6, 2013 at 8:46 PM, Matthew Brett matthew.brett@gmail.com wrote:
Hi,
On Wed, Feb 6, 2013 at 10:48 AM, Ralf Gommers ralf.gommers@gmail.com wrote:
On Wed, Feb 6, 2013 at 1:32 AM, Matthew Brett matthew.brett@gmail.com wrote:
Hi,
On Tue, Feb 5, 2013 at 3:04 PM, Christoph Gohlke cgohlke@uci.edu wrote: > On 2/5/2013 10:51 AM, Matthew Brett wrote: >> Hi, >> >> On Mon, Feb 4, 2013 at 5:09 PM, Matthew Brett >> matthew.brett@gmail.com >> wrote: >>> Hi, >>> >>> On Mon, Feb 4, 2013 at 3:46 PM, Charles R Harris >>> charlesr.harris@gmail.com wrote: >>>> >>>> >>>> On Mon, Feb 4, 2013 at 4:04 PM, Robert Kern >>>> robert.kern@gmail.com >>>> wrote: >>>>> >>>>> On Mon, Feb 4, 2013 at 10:38 PM, Matthew Brett >>>>> matthew.brett@gmail.com >>>>> wrote: >>>>>> Hi, >>>>>> >>>>>> On Mon, Feb 4, 2013 at 1:15 PM, Ralf Gommers >>>>>> ralf.gommers@gmail.com >>>>>> wrote: >>>>> >>>>>>> MSVC + Intel Fortran + MKL, yes. But those aren't free. So how >>>>>>> can >>>>>>> you >>>>>>> provide an Amazon image for those? >>>>>> >>>>>> You can make an image that is not public, I guess. I suppose >>>>>> anyone >>>>>> who uses the image would have to have their own licenses for the >>>>>> Intel >>>>>> stuff? Does anyone have experience of this? >>>>> >>>>> You need to purchase one license per developer: >>>>> >>>>> >>>>> >>>>> >>>>> http://software.intel.com/en-us/articles/intel-math-kernel-library-licensing... >>>>> >>>> >>>> I think 64 bits on windows is best pushed off to 1.7.1 or 1.8. It >>>> would be a >>>> bit much to get it implemented in the next week or two. >>> >>> The problem with not providing these binaries is that they are at >>> the >>> bottom of everyone's stack, so a delay in numpy holds everyone >>> back. >>> >>> I can't find completely convincing stats, but it looks as though 64 >>> bit windows 7 is now the most common version of Windows, at least >>> for >>> Gamers [1] around now, and it was getting that way for everyone in >>> 2010 [2]. >>> >>> It don't think it reflects well on on us that we don't appear to >>> support 64 bits out of the box; just for example, R already has a >>> 32 >>> bit / 64 bit installer. >>> >>> If I understand correctly, the options for doing this right now >>> are: >>> >>> 1) Minimal cost in time : ask Christophe nicely whether we can >>> distribute his binaries via the Numpy page >>> 2) Small cost in time / money : pay for licenses for Ondrej or me >>> or >>> someone to install the dependencies on my Berkeley machine / an >>> Amazon >>> image. >> >> In order not to leave this discussion without a resolution: >> >> Christophe - would you allow us to distribute your numpy binaries >> for >> 1.7 from the numpy sourceforge page? >> >> Cheers, >> >> Matthew > > > I am OK with providing 64 bit "numpy-MKL" binaries (that is numpy > compiled with MSVC compilers and linked to Intel's MKL) for official > numpy releases.
Thank you - that is great.
> However: > > 1) There seems to be no real consensus and urge for doing this.
I certainly feel the urge and feel it strongly. As a packager for two or three projects myself, it's a royal pain having to tell someone to go to two different places for binaries depending on the number of bits of their Windows.
If you're relying on .exe installers on SF, then you have to send your users to more places than that probably. Really the separate installers are a poor alternative to the available scientific distributions. And the more packages you need as a user, the more annoying these separate installers are.
I think Chuck was worried about the time it would take to do it, and I think you've already solved this problem. Ralf was worried about Scipy - see below.
Not just Scipy - that would be my worry number one, but the same holds for all packages based on Numpy. You double the number of Windows installers that every single project needs to provide.
> Using a > free toolchain capable of building the whole scipy-stack would be > much > preferred.
That's true, but there seems general agreement this is not practical in the very near future.
> Several 64 bit Python distributions containing numpy-MKL are > already available, some for free.
You mean EPD and AnacondaCE? I don't think we should withhold easily available vanilla builds because they are also available in company-sponsored projects. Python.org provides windows builds even though ActiveState is free-as-in-beer.
If the company-sponsored bit bothers you, there's also a 64-bit Python(x,y) now.
I'm sure you've seen that the question 'where are the 64-bit installers' comes up often for Numpy.
It seems to me that we'd have to have a good reason not provide them. There will always be some number of people like me who like to install the various parts by hand, and don't like using large packages, free or or open or not. For example, I don't use macports. At the moment, the reasons I see you are giving are:
- Then everyone would have to provide 64-bit binaries
Indeed. And since many packages won't do that because there's no free compilers, users just get stuck a bit later in the "installing the stack" process. I'm sure providing the binaries will help some people, but I expect it to cause as many problems as it solves.
Can you clarify the people you think will get stuck? I think I'm right in saying that anyone with a C extension should be able to build them against numpy, by installing the free (as-in-beer) MS tools? So do you just mean people needing a Fortran compiler? That's a small constituency, I think.
Off the top of my head there's SciPy and pymc...
We covered Scipy a while back in this thread. The proposal was that Christophe also supply these.
Perhaps we can at least agree that the large majority of numpy-based projects that require compilation do not need FORTRAN. And if they do, then, as Chris said, they are likely to have sophisticated developers who are not likely to be confused by a new Windows build for numpy.
Anyway, I'm butting in because I wish this discussion could separate between the user perspective and the developer perspective.
FWIW,
- From a user's perspective, I don't understand this either. If you are
already using a closed source, not-free-as-in-beer operating system, why would you not use (or buy!) a closed source, not-free-as-in-beer Fortran compiler?
I think you might be asking whether Windows users care about having free (beer and or freedom) software? I think the answer to this is yes. As indeed OSX users care about having free software.
- BUT, the argument I've seen that I can at least understand is that
the release manager should be able to do a release using only open source tools (even using Wine instead of Windows) and not rely on a limited number of licenses. And that the release manager must be able to perform all the official builds directly.
I haven't heard that argument made. I've heard the general wish that we could build numpy with open-source tools. I haven't heard anyone assert that we won't release anything that can't be build with open-source tools. Python itself is not built with open-source tools, so I can't personally see the reason for this restriction. Perhaps you can explain it?
I haven't heard the argument that the release manager should be able to make all the builds. That would be nice, obviously, but as an absolute rule it seems unnecessary. For example, is it your understanding that all the Python builds are made by the same person? Why would we want to be more restrictive than that?
I don't know if anyone has estimated the number of people running Numpy via windows, but I notice from the download stats on Sourceforge, that there are around 800 downloads of OSX binary installers and over 4000 downloads of windows binaries.
To be clear, what we seem to be heading for at the moment is this:
* We have a experienced builder in Christophe building Windows 64 binaries for numpy and scipy * He has offered us at least Numpy and maybe Scipy * We are saying no to this zero-cost-to-us build because of one of the following reasons
1) Anyone using Windows 64 should be using EPD or AnacondaCE [1]. To enforce this we should not provide our own build 2) If we provide a Windows 64 build then other people will have to provide them too 3) We insist for some reason on only using open-source build tools 4) We insist for some reason that the release manager personally builds all the builds.
This means that there is no way that I has a packager of - say - nipy - can give you a binary installer for Windows 64 bits without making you install one of (Christophe's builds, EPD, AnacondaCE). Which one should I build and installer for? All three? You have to give up your Python.org python to use my package?
We are, because of this, leaving more than half of what appears to be our largest user base without a standard binary installer.
I suppose, in 6 months time, someone will look for a Windows 64 bit installer, and they will not see one, and they think 'I wonder why?'. Then they will look at this thread. I wonder if they'll think we made the right decision or not.
Cheers,
Matthew
[1] I can't find a 64-bit Python X Y but maybe one exists.
On Wed, Feb 6, 2013 at 9:20 PM, Dag Sverre Seljebotn d.s.seljebotn@astro.uio.no wrote:
On 02/07/2013 12:16 AM, Matthew Brett wrote:
[...]
Can you clarify the people you think will get stuck? I think I'm right in saying that anyone with a C extension should be able to build them against numpy, by installing the free (as-in-beer) MS tools? So do you just mean people needing a Fortran compiler? That's a small constituency, I think.
Off the top of my head there's SciPy and pymc...
Anyway, I'm butting in because I wish this discussion could separate between the user perspective and the developer perspective.
FWIW,
- From a user's perspective, I don't understand this either. If you are
already using a closed source, not-free-as-in-beer operating system, why would you not use (or buy!) a closed source, not-free-as-in-beer Fortran compiler?
Indeed. Though I really have no clue on the Windows use cases. Maybe most Windows users don't want to compile anything, just use numpy and scipy from Python?
- BUT, the argument I've seen that I can at least understand is that
the release manager should be able to do a release using only open source tools (even using Wine instead of Windows) and not rely on a limited number of licenses. And that the release manager must be able to perform all the official builds directly.
As the release manager, I really only have two requirements:
* I want to ssh in there from my Ubuntu * I want to automate the whole process
For Mac, linux and Wine I can do that. So I have just spend few hours browsing the net and it looks like that the combination of Windows PowerShell 2.0:
http://en.wikipedia.org/wiki/Windows_PowerShell
and some SSH server, there are quite a few, one commercial but free for one user one connection (perfect for me!):
http://www.powershellinside.com/powershell/ssh/
So if I understand the pages correctly, I can login there from linux, and then I use the PowerShell commands to script anything. It looks like I can even use my Fabric fabfiles with powershell:
https://gist.github.com/diyan/2850866
I can also use git with PowerShell:
http://windows.github.com/ http://haacked.com/archive/2011/12/13/better-git-with-powershell.aspx
So the final problem is how to execute MSVC and Fortran from Power Shell on Windows. These links might help for MSVC:
http://stackoverflow.com/questions/4398136/use-powershell-for-visual-studio-... http://geekswithblogs.net/dwdii/archive/2011/05/20/automating-a-visual-studi...
Finally, for Intel Fortran + powershell:
http://software.intel.com/en-us/forums/topic/284425
So I think it is all possible. If somebody can provide a machine with Windows, MSVC, PowerShell2.0, SSH server and some Fortran compiler, it should be possible for me to automate everything from Ubuntu using my Fabric files (https://github.com/certik/numpy-vendor).
Ondrej
Hi,
On Wed, Feb 6, 2013 at 10:21 PM, Ondřej Čertík ondrej.certik@gmail.com wrote:
On Wed, Feb 6, 2013 at 9:20 PM, Dag Sverre Seljebotn d.s.seljebotn@astro.uio.no wrote:
On 02/07/2013 12:16 AM, Matthew Brett wrote:
[...]
Can you clarify the people you think will get stuck? I think I'm right in saying that anyone with a C extension should be able to build them against numpy, by installing the free (as-in-beer) MS tools? So do you just mean people needing a Fortran compiler? That's a small constituency, I think.
Off the top of my head there's SciPy and pymc...
Anyway, I'm butting in because I wish this discussion could separate between the user perspective and the developer perspective.
FWIW,
- From a user's perspective, I don't understand this either. If you are
already using a closed source, not-free-as-in-beer operating system, why would you not use (or buy!) a closed source, not-free-as-in-beer Fortran compiler?
Indeed. Though I really have no clue on the Windows use cases. Maybe most Windows users don't want to compile anything, just use numpy and scipy from Python?
Well - yes - as a packager I really want to be able to provide a binary so my binary consumers don't have to have a C compiler installed. I imagine it's the same for all of us packagers out there.
- BUT, the argument I've seen that I can at least understand is that
the release manager should be able to do a release using only open source tools (even using Wine instead of Windows) and not rely on a limited number of licenses. And that the release manager must be able to perform all the official builds directly.
As the release manager, I really only have two requirements:
- I want to ssh in there from my Ubuntu
- I want to automate the whole process
For Mac, linux and Wine I can do that. So I have just spend few hours browsing the net and it looks like that the combination of Windows PowerShell 2.0:
http://en.wikipedia.org/wiki/Windows_PowerShell
and some SSH server, there are quite a few, one commercial but free for one user one connection (perfect for me!):
http://www.powershellinside.com/powershell/ssh/
So if I understand the pages correctly, I can login there from linux, and then I use the PowerShell commands to script anything. It looks like I can even use my Fabric fabfiles with powershell:
https://gist.github.com/diyan/2850866
I can also use git with PowerShell:
http://windows.github.com/ http://haacked.com/archive/2011/12/13/better-git-with-powershell.aspx
So the final problem is how to execute MSVC and Fortran from Power Shell on Windows. These links might help for MSVC:
http://stackoverflow.com/questions/4398136/use-powershell-for-visual-studio-... http://geekswithblogs.net/dwdii/archive/2011/05/20/automating-a-visual-studi...
Finally, for Intel Fortran + powershell:
http://software.intel.com/en-us/forums/topic/284425
So I think it is all possible. If somebody can provide a machine with Windows, MSVC, PowerShell2.0, SSH server and some Fortran compiler, it should be possible for me to automate everything from Ubuntu using my Fabric files (https://github.com/certik/numpy-vendor).
Many many thanks for trying to solve this. I had really started to give up hope.
I think you will need a developer's license for MKL for Numpy. Ralf - any ETA for those?
I think I'm right in thinking you'll need a Fortran compiler for Scipy but not Numpy? Can we defer the Scipy build until after the Numpy build?
I will try to get you set up with ssh on my Windows 7 machine in case you can use it. It has the MS tools.
Thanks again,
Matthew
On Wed, Feb 6, 2013 at 10:41 PM, Matthew Brett matthew.brett@gmail.com wrote:
Hi,
On Wed, Feb 6, 2013 at 10:21 PM, Ondřej Čertík ondrej.certik@gmail.com wrote:
On Wed, Feb 6, 2013 at 9:20 PM, Dag Sverre Seljebotn d.s.seljebotn@astro.uio.no wrote:
On 02/07/2013 12:16 AM, Matthew Brett wrote:
[...]
Can you clarify the people you think will get stuck? I think I'm right in saying that anyone with a C extension should be able to build them against numpy, by installing the free (as-in-beer) MS tools? So do you just mean people needing a Fortran compiler? That's a small constituency, I think.
Off the top of my head there's SciPy and pymc...
Anyway, I'm butting in because I wish this discussion could separate between the user perspective and the developer perspective.
FWIW,
- From a user's perspective, I don't understand this either. If you are
already using a closed source, not-free-as-in-beer operating system, why would you not use (or buy!) a closed source, not-free-as-in-beer Fortran compiler?
Indeed. Though I really have no clue on the Windows use cases. Maybe most Windows users don't want to compile anything, just use numpy and scipy from Python?
Well - yes - as a packager I really want to be able to provide a binary so my binary consumers don't have to have a C compiler installed. I imagine it's the same for all of us packagers out there.
- BUT, the argument I've seen that I can at least understand is that
the release manager should be able to do a release using only open source tools (even using Wine instead of Windows) and not rely on a limited number of licenses. And that the release manager must be able to perform all the official builds directly.
As the release manager, I really only have two requirements:
- I want to ssh in there from my Ubuntu
- I want to automate the whole process
For Mac, linux and Wine I can do that. So I have just spend few hours browsing the net and it looks like that the combination of Windows PowerShell 2.0:
http://en.wikipedia.org/wiki/Windows_PowerShell
and some SSH server, there are quite a few, one commercial but free for one user one connection (perfect for me!):
http://www.powershellinside.com/powershell/ssh/
So if I understand the pages correctly, I can login there from linux, and then I use the PowerShell commands to script anything. It looks like I can even use my Fabric fabfiles with powershell:
https://gist.github.com/diyan/2850866
I can also use git with PowerShell:
http://windows.github.com/ http://haacked.com/archive/2011/12/13/better-git-with-powershell.aspx
So the final problem is how to execute MSVC and Fortran from Power Shell on Windows. These links might help for MSVC:
http://stackoverflow.com/questions/4398136/use-powershell-for-visual-studio-... http://geekswithblogs.net/dwdii/archive/2011/05/20/automating-a-visual-studi...
Finally, for Intel Fortran + powershell:
http://software.intel.com/en-us/forums/topic/284425
So I think it is all possible. If somebody can provide a machine with Windows, MSVC, PowerShell2.0, SSH server and some Fortran compiler, it should be possible for me to automate everything from Ubuntu using my Fabric files (https://github.com/certik/numpy-vendor).
Many many thanks for trying to solve this. I had really started to give up hope.
I think you will need a developer's license for MKL for Numpy. Ralf - any ETA for those?
I think I'm right in thinking you'll need a Fortran compiler for Scipy but not Numpy? Can we defer the Scipy build until after the Numpy build?
I will try to get you set up with ssh on my Windows 7 machine in case you can use it. It has the MS tools.
That would be amazing! If you can set me up with the Power Shell and some ssh server, I'll start playing with this right away.
Ondrej
Hi,
On Wed, Feb 6, 2013 at 10:59 PM, Ondřej Čertík ondrej.certik@gmail.com wrote:
On Wed, Feb 6, 2013 at 10:41 PM, Matthew Brett matthew.brett@gmail.com wrote:
Hi,
On Wed, Feb 6, 2013 at 10:21 PM, Ondřej Čertík ondrej.certik@gmail.com wrote:
On Wed, Feb 6, 2013 at 9:20 PM, Dag Sverre Seljebotn d.s.seljebotn@astro.uio.no wrote:
On 02/07/2013 12:16 AM, Matthew Brett wrote:
[...]
Can you clarify the people you think will get stuck? I think I'm right in saying that anyone with a C extension should be able to build them against numpy, by installing the free (as-in-beer) MS tools? So do you just mean people needing a Fortran compiler? That's a small constituency, I think.
Off the top of my head there's SciPy and pymc...
Anyway, I'm butting in because I wish this discussion could separate between the user perspective and the developer perspective.
FWIW,
- From a user's perspective, I don't understand this either. If you are
already using a closed source, not-free-as-in-beer operating system, why would you not use (or buy!) a closed source, not-free-as-in-beer Fortran compiler?
Indeed. Though I really have no clue on the Windows use cases. Maybe most Windows users don't want to compile anything, just use numpy and scipy from Python?
Well - yes - as a packager I really want to be able to provide a binary so my binary consumers don't have to have a C compiler installed. I imagine it's the same for all of us packagers out there.
- BUT, the argument I've seen that I can at least understand is that
the release manager should be able to do a release using only open source tools (even using Wine instead of Windows) and not rely on a limited number of licenses. And that the release manager must be able to perform all the official builds directly.
As the release manager, I really only have two requirements:
- I want to ssh in there from my Ubuntu
- I want to automate the whole process
For Mac, linux and Wine I can do that. So I have just spend few hours browsing the net and it looks like that the combination of Windows PowerShell 2.0:
http://en.wikipedia.org/wiki/Windows_PowerShell
and some SSH server, there are quite a few, one commercial but free for one user one connection (perfect for me!):
http://www.powershellinside.com/powershell/ssh/
So if I understand the pages correctly, I can login there from linux, and then I use the PowerShell commands to script anything. It looks like I can even use my Fabric fabfiles with powershell:
https://gist.github.com/diyan/2850866
I can also use git with PowerShell:
http://windows.github.com/ http://haacked.com/archive/2011/12/13/better-git-with-powershell.aspx
So the final problem is how to execute MSVC and Fortran from Power Shell on Windows. These links might help for MSVC:
http://stackoverflow.com/questions/4398136/use-powershell-for-visual-studio-... http://geekswithblogs.net/dwdii/archive/2011/05/20/automating-a-visual-studi...
Finally, for Intel Fortran + powershell:
http://software.intel.com/en-us/forums/topic/284425
So I think it is all possible. If somebody can provide a machine with Windows, MSVC, PowerShell2.0, SSH server and some Fortran compiler, it should be possible for me to automate everything from Ubuntu using my Fabric files (https://github.com/certik/numpy-vendor).
Many many thanks for trying to solve this. I had really started to give up hope.
I think you will need a developer's license for MKL for Numpy. Ralf - any ETA for those?
I think I'm right in thinking you'll need a Fortran compiler for Scipy but not Numpy? Can we defer the Scipy build until after the Numpy build?
I will try to get you set up with ssh on my Windows 7 machine in case you can use it. It has the MS tools.
That would be amazing! If you can set me up with the Power Shell and some ssh server, I'll start playing with this right away.
I've set up a Cygwin SSH server on the box, and powershell 2 comes with windows 7, I believe. At least, that's the version I'm getting.
However, it's hard to run powershell scripts interactively via cygwin :
http://hivearchive.com/2006/07/03/using-powershell-through-ssh/ http://cygwin.com/ml/cygwin/2008-10/msg00393.html
so you might need to debug the scripts interactively via remote desktop protocol and then run them non-interactively.
Could you send me your ssh public key off list or give me a call to get set up?
Thanks again,
Matthew
On Thu, Feb 7, 2013 at 7:41 AM, Matthew Brett matthew.brett@gmail.comwrote:
I think you will need a developer's license for MKL for Numpy. Ralf - any ETA for those?
No, I'll have to ask again.
I think I'm right in thinking you'll need a Fortran compiler for Scipy but not Numpy?
Correct.
Can we defer the Scipy build until after the Numpy build?
That doesn't sound like a good idea to me.
I will try to get you set up with ssh on my Windows 7 machine in case you can use it. It has the MS tools.
As pointed out before by Robert, sharing the machine between you requires multiple licenses. Intel has promised us some licenses, but those are given to specific people.
Ralf
Hi,
On Thu, Feb 7, 2013 at 10:47 AM, Ralf Gommers ralf.gommers@gmail.com wrote:
On Thu, Feb 7, 2013 at 7:41 AM, Matthew Brett matthew.brett@gmail.com wrote:
I think you will need a developer's license for MKL for Numpy. Ralf - any ETA for those?
No, I'll have to ask again.
I think I'm right in thinking you'll need a Fortran compiler for Scipy but not Numpy?
Correct.
Can we defer the Scipy build until after the Numpy build?
That doesn't sound like a good idea to me.
I will try to get you set up with ssh on my Windows 7 machine in case you can use it. It has the MS tools.
As pointed out before by Robert, sharing the machine between you requires multiple licenses. Intel has promised us some licenses, but those are given to specific people.
Right. Ondrej has his own account on my machine, and it will be Ondrej's license, when it is available.
Cheers,
Matthew
Hi,
On Thu, Feb 7, 2013 at 10:47 AM, Ralf Gommers ralf.gommers@gmail.com wrote:
On Thu, Feb 7, 2013 at 7:41 AM, Matthew Brett matthew.brett@gmail.com wrote:
I think you will need a developer's license for MKL for Numpy. Ralf - any ETA for those?
No, I'll have to ask again.
I think I'm right in thinking you'll need a Fortran compiler for Scipy but not Numpy?
Correct.
Can we defer the Scipy build until after the Numpy build?
That doesn't sound like a good idea to me.
I must say I'm a little confused as to how we're going to make the decisions here.
I'm sure you agree that there's an opposite argument to be made, and I would make it if I thought it would make a difference, but I'm losing faith in my ability to keep the discussion on track, and I don't know what to do about that.
Cheers,
Matthew
On Thu, Feb 7, 2013 at 10:59 AM, Matthew Brett matthew.brett@gmail.com wrote:
Hi,
On Thu, Feb 7, 2013 at 10:47 AM, Ralf Gommers ralf.gommers@gmail.com wrote:
On Thu, Feb 7, 2013 at 7:41 AM, Matthew Brett matthew.brett@gmail.com wrote:
I think you will need a developer's license for MKL for Numpy. Ralf - any ETA for those?
No, I'll have to ask again.
I think I'm right in thinking you'll need a Fortran compiler for Scipy but not Numpy?
Correct.
Can we defer the Scipy build until after the Numpy build?
That doesn't sound like a good idea to me.
I must say I'm a little confused as to how we're going to make the decisions here.
I'm sure you agree that there's an opposite argument to be made, and I would make it if I thought it would make a difference, but I'm losing faith in my ability to keep the discussion on track, and I don't know what to do about that.
Matthew I don't see any problem here. I agree with Ralf that we need to figure out how to build scipy with Fortran pretty much at the same time, to see if the solution is a viable solution. With Christoph help and experience, I am sure it can get done in a satisfactory way.
Ondrej
On Thu, Feb 7, 2013 at 7:59 PM, Matthew Brett matthew.brett@gmail.comwrote:
Can we defer the Scipy build until after the Numpy build?
That doesn't sound like a good idea to me.
I must say I'm a little confused as to how we're going to make the decisions here.
How about: attempt to reach consensus? David's concern on DLLs hasn't been addressed yet, nor has mine on packages being unavailable. I was actually still answering another of your emails, but I can't seem to reply fast enough.
I'm sure you agree that there's an opposite argument to be made, and I would make it if I thought it would make a difference, but I'm losing faith in my ability to keep the discussion on track, and I don't know what to do about that.
I don't see the problem. Before you offered to put in work. Ondrej is willing to help, so is Christoph. So why is it impossible to do Scipy builds?
I can see us getting to a solution here, but offering Numpy installers without Scipy ones is not a solution in my book.
Ralf
On Thu, Feb 7, 2013 at 11:05 AM, Ralf Gommers ralf.gommers@gmail.com wrote:
On Thu, Feb 7, 2013 at 7:59 PM, Matthew Brett matthew.brett@gmail.com wrote:
Can we defer the Scipy build until after the Numpy build?
That doesn't sound like a good idea to me.
I must say I'm a little confused as to how we're going to make the decisions here.
How about: attempt to reach consensus? David's concern on DLLs hasn't been addressed yet, nor has mine on packages being unavailable. I was actually still answering another of your emails, but I can't seem to reply fast enough.
Yep, we will need to address those.
I'm sure you agree that there's an opposite argument to be made, and I would make it if I thought it would make a difference, but I'm losing faith in my ability to keep the discussion on track, and I don't know what to do about that.
I don't see the problem. Before you offered to put in work. Ondrej is willing to help, so is Christoph. So why is it impossible to do Scipy builds?
I can see us getting to a solution here, but offering Numpy installers without Scipy ones is not a solution in my book.
Exactly. There is no problem here. Fortran needs to be working as a first class citizen. I personally use modern Fortran a lot. I've setup this page:
with a relevant FAQ about binary compatibility:
http://fortran90.org/src/faq.html#are-fortran-compilers-abi-compatible
and based on how things work on Windows, I'll be happy to extend the information there.
Ondrej
Hi,
On Thu, Feb 7, 2013 at 11:05 AM, Ralf Gommers ralf.gommers@gmail.com wrote:
On Thu, Feb 7, 2013 at 7:59 PM, Matthew Brett matthew.brett@gmail.com wrote:
Can we defer the Scipy build until after the Numpy build?
That doesn't sound like a good idea to me.
I must say I'm a little confused as to how we're going to make the decisions here.
How about: attempt to reach consensus? David's concern on DLLs hasn't been addressed yet, nor has mine on packages being unavailable. I was actually still answering another of your emails, but I can't seem to reply fast enough.
Right - consensus is good - but at the moment I keep getting lost because the arguments seem to shift and get lost, and sometimes they are not made.
So, here is the summary as I understand it, please correct if I am wrong
I think we agree that:
1) Having a binary installer for numpy Windows 64 bit is desirable 2) It is desirable to have a matching binary installer for Scipy as soon as possible 3) It is preferable to build with free tools 4) It is acceptable to use non-free tools 5) The build will need to do some run-time linking to MKL and / or mingw 6) It is preferable that the build should be fully automated 7) It is preferable that one person can build all numpy / scipy builds
The points of potential disagreement are:
a) If we cannot build Scipy now, it may or may not be acceptable to release numpy now. I think it is, you (Ralf) think it isn't, we haven't discussed that. It may not come up. b) It may or may not be acceptable for someone other than Ondrej to be responsible for the Windows 64-bit builds. I think it should be, if necessary, we haven't really discussed that, it may not come up. c) It may or may not be acceptable for the build to be only partially automated. Ditto. d) It may or may not be acceptable to add the DLL directory to the PATH on numpy import. David says not, Christophe disagrees, we haven't really discussed that.
Is that right?
Cheers,
Matthew
On Thu, Feb 7, 2013 at 8:38 PM, Matthew Brett matthew.brett@gmail.comwrote:
Hi,
On Thu, Feb 7, 2013 at 11:05 AM, Ralf Gommers ralf.gommers@gmail.com wrote:
On Thu, Feb 7, 2013 at 7:59 PM, Matthew Brett matthew.brett@gmail.com wrote:
Can we defer the Scipy build until after the Numpy build?
That doesn't sound like a good idea to me.
I must say I'm a little confused as to how we're going to make the decisions here.
How about: attempt to reach consensus? David's concern on DLLs hasn't
been
addressed yet, nor has mine on packages being unavailable. I was actually still answering another of your emails, but I can't seem to reply fast enough.
Right - consensus is good - but at the moment I keep getting lost because the arguments seem to shift and get lost, and sometimes they are not made.
So, here is the summary as I understand it, please correct if I am wrong
I think we agree that:
- Having a binary installer for numpy Windows 64 bit is desirable
- It is desirable to have a matching binary installer for Scipy as
soon as possible 3) It is preferable to build with free tools 4) It is acceptable to use non-free tools 5) The build will need to do some run-time linking to MKL and / or mingw 6) It is preferable that the build should be fully automated 7) It is preferable that one person can build all numpy / scipy builds
The points of potential disagreement are:
a) If we cannot build Scipy now, it may or may not be acceptable to release numpy now. I think it is, you (Ralf) think it isn't, we haven't discussed that. It may not come up. b) It may or may not be acceptable for someone other than Ondrej to be responsible for the Windows 64-bit builds. I think it should be, if necessary, we haven't really discussed that, it may not come up. c) It may or may not be acceptable for the build to be only partially automated. Ditto. d) It may or may not be acceptable to add the DLL directory to the PATH on numpy import. David says not, Christophe disagrees, we haven't really discussed that.
Is that right?
Good summary, looks complete to me.
Ralf
On Thu, Feb 7, 2013 at 11:38 AM, Matthew Brett matthew.brett@gmail.com wrote:
a) If we cannot build Scipy now, it may or may not be acceptable to release numpy now. I think it is, you (Ralf) think it isn't, we haven't discussed that. It may not come up.
Is anyone suggesting we hold the whole release for this? I fnot, then why would you say we shouldnt' put out a 64 bit Windows numpy binary because we dont' ahve a matching Scipy one? folks that need Scipy will have to find a way to built it themselves anyway, and folks that use numpy without scipy would have a nice binary to use -- That seems like value-added to me.
d) It may or may not be acceptable to add the DLL directory to the PATH on numpy import. David says not, Christophe disagrees, we haven't really discussed that.
I like the "load them with ctypes" approach better, but I don't know how much effort it would be to implement that.
I'm still not totally clear if there is an issue with buildings other extensions tht rely on numpy/scipy -- can they be built with mingGW if numpy/scipy were built with MSVC (Or any other compiler for that matter).
-Chris
Hi,
On Thu, Feb 7, 2013 at 12:29 PM, Chris Barker - NOAA Federal chris.barker@noaa.gov wrote:
On Thu, Feb 7, 2013 at 11:38 AM, Matthew Brett matthew.brett@gmail.com wrote:
a) If we cannot build Scipy now, it may or may not be acceptable to release numpy now. I think it is, you (Ralf) think it isn't, we haven't discussed that. It may not come up.
Is anyone suggesting we hold the whole release for this? I fnot, then why would you say we shouldnt' put out a 64 bit Windows numpy binary because we dont' ahve a matching Scipy one? folks that need Scipy will have to find a way to built it themselves anyway, and folks that use numpy without scipy would have a nice binary to use -- That seems like value-added to me.
Right - me too - but we could hold off that question until Ondrej has had a chance to build both I guess?
d) It may or may not be acceptable to add the DLL directory to the PATH on numpy import. David says not, Christophe disagrees, we haven't really discussed that.
I like the "load them with ctypes" approach better, but I don't know how much effort it would be to implement that.
I'm still not totally clear if there is an issue with buildings other extensions tht rely on numpy/scipy -- can they be built with mingGW if numpy/scipy were built with MSVC (Or any other compiler for that matter).
I had the impression that the problem with Mingw64 was the need to load Mingw DLLs at run time - if that's the same for MSVC / MKL, maybe there's no advantage to using the MS tools? But I don't understand the issues very well.
Cheers,
Matthew
On Thu, Feb 7, 2013 at 12:29 PM, Chris Barker - NOAA Federal chris.barker@noaa.gov wrote:
On Thu, Feb 7, 2013 at 11:38 AM, Matthew Brett matthew.brett@gmail.com wrote:
a) If we cannot build Scipy now, it may or may not be acceptable to release numpy now. I think it is, you (Ralf) think it isn't, we haven't discussed that. It may not come up.
Is anyone suggesting we hold the whole release for this? I fnot, then
Just to make it clear, I do not plan to hold the whole release because of this. Previous releases also didn't have this official 64bit Windows binary, so there is no regression.
Once we figure out how to create 64bit binaries, then we'll start uploading them.
Ondrej
Hi Ondrej,
On Thu, Feb 7, 2013 at 3:18 PM, Ondřej Čertík ondrej.certik@gmail.com wrote:
On Thu, Feb 7, 2013 at 12:29 PM, Chris Barker - NOAA Federal chris.barker@noaa.gov wrote:
On Thu, Feb 7, 2013 at 11:38 AM, Matthew Brett matthew.brett@gmail.com wrote:
a) If we cannot build Scipy now, it may or may not be acceptable to release numpy now. I think it is, you (Ralf) think it isn't, we haven't discussed that. It may not come up.
Is anyone suggesting we hold the whole release for this? I fnot, then
Just to make it clear, I do not plan to hold the whole release because of this. Previous releases also didn't have this official 64bit Windows binary, so there is no regression.
Once we figure out how to create 64bit binaries, then we'll start uploading them.
Did you make any progress with this? Worth making some notes? Anything we can do to help?
Cheers,
Matthew
Hi Matthew,
On Tue, Mar 26, 2013 at 5:48 PM, Matthew Brett matthew.brett@gmail.com wrote:
Hi Ondrej,
On Thu, Feb 7, 2013 at 3:18 PM, Ondřej Čertík ondrej.certik@gmail.com wrote:
On Thu, Feb 7, 2013 at 12:29 PM, Chris Barker - NOAA Federal chris.barker@noaa.gov wrote:
On Thu, Feb 7, 2013 at 11:38 AM, Matthew Brett matthew.brett@gmail.com wrote:
a) If we cannot build Scipy now, it may or may not be acceptable to release numpy now. I think it is, you (Ralf) think it isn't, we haven't discussed that. It may not come up.
Is anyone suggesting we hold the whole release for this? I fnot, then
Just to make it clear, I do not plan to hold the whole release because of this. Previous releases also didn't have this official 64bit Windows binary, so there is no regression.
Once we figure out how to create 64bit binaries, then we'll start uploading them.
Did you make any progress with this? Worth making some notes? Anything we can do to help?
Unfortunately I've been too busy the last month to push this through, so right now I am just concentrating on getting 1.7.1 out of the door, as that is higher priority.
I am starting a new job on Monday, so once things settle down, I should be able to get back to this. I will post notes once I get to this again.
Ondrej
On Thu, Feb 7, 2013 at 7:38 PM, Matthew Brett matthew.brett@gmail.com wrote:
Hi,
On Thu, Feb 7, 2013 at 11:05 AM, Ralf Gommers ralf.gommers@gmail.com wrote:
On Thu, Feb 7, 2013 at 7:59 PM, Matthew Brett matthew.brett@gmail.com wrote:
Can we defer the Scipy build until after the Numpy build?
That doesn't sound like a good idea to me.
I must say I'm a little confused as to how we're going to make the decisions here.
How about: attempt to reach consensus? David's concern on DLLs hasn't been addressed yet, nor has mine on packages being unavailable. I was actually still answering another of your emails, but I can't seem to reply fast enough.
Right - consensus is good - but at the moment I keep getting lost because the arguments seem to shift and get lost, and sometimes they are not made.
So, here is the summary as I understand it, please correct if I am wrong
I think we agree that:
- Having a binary installer for numpy Windows 64 bit is desirable
- It is desirable to have a matching binary installer for Scipy as
soon as possible 3) It is preferable to build with free tools 4) It is acceptable to use non-free tools 5) The build will need to do some run-time linking to MKL and / or mingw 6) It is preferable that the build should be fully automated 7) It is preferable that one person can build all numpy / scipy builds
The points of potential disagreement are:
a) If we cannot build Scipy now, it may or may not be acceptable to release numpy now. I think it is, you (Ralf) think it isn't, we haven't discussed that. It may not come up. b) It may or may not be acceptable for someone other than Ondrej to be responsible for the Windows 64-bit builds. I think it should be, if necessary, we haven't really discussed that, it may not come up. c) It may or may not be acceptable for the build to be only partially automated. Ditto. d) It may or may not be acceptable to add the DLL directory to the PATH on numpy import. David says not, Christophe disagrees, we haven't really discussed that.
I assumed this was obvious, but looks like it isn't: modifying the process user environment when importing a python package is quite user hostile. os.environ is a global variable, and it could cause quite hard to diagnose issues when something goes wrong.
I would see a library doing this kind of things, especially at import time, quite badly.
David
On Thu, Feb 7, 2013 at 6:21 AM, Ondřej Čertík ondrej.certik@gmail.com wrote:
On Wed, Feb 6, 2013 at 9:20 PM, Dag Sverre Seljebotn d.s.seljebotn@astro.uio.no wrote:
On 02/07/2013 12:16 AM, Matthew Brett wrote:
[...]
Can you clarify the people you think will get stuck? I think I'm right in saying that anyone with a C extension should be able to build them against numpy, by installing the free (as-in-beer) MS tools? So do you just mean people needing a Fortran compiler? That's a small constituency, I think.
Off the top of my head there's SciPy and pymc...
Anyway, I'm butting in because I wish this discussion could separate between the user perspective and the developer perspective.
FWIW,
- From a user's perspective, I don't understand this either. If you are
already using a closed source, not-free-as-in-beer operating system, why would you not use (or buy!) a closed source, not-free-as-in-beer Fortran compiler?
Indeed. Though I really have no clue on the Windows use cases. Maybe most Windows users don't want to compile anything, just use numpy and scipy from Python?
- BUT, the argument I've seen that I can at least understand is that
the release manager should be able to do a release using only open source tools (even using Wine instead of Windows) and not rely on a limited number of licenses. And that the release manager must be able to perform all the official builds directly.
As the release manager, I really only have two requirements:
- I want to ssh in there from my Ubuntu
- I want to automate the whole process
For Mac, linux and Wine I can do that. So I have just spend few hours browsing the net and it looks like that the combination of Windows PowerShell 2.0:
http://en.wikipedia.org/wiki/Windows_PowerShell
and some SSH server, there are quite a few, one commercial but free for one user one connection (perfect for me!):
http://www.powershellinside.com/powershell/ssh/
So if I understand the pages correctly, I can login there from linux, and then I use the PowerShell commands to script anything. It looks like I can even use my Fabric fabfiles with powershell:
https://gist.github.com/diyan/2850866
I can also use git with PowerShell:
http://windows.github.com/ http://haacked.com/archive/2011/12/13/better-git-with-powershell.aspx
Ondrej, you may be interested in some hack I've done to use winrm from fabric: https://github.com/fabric/fabric/pull/872
It gives a new winrm_run function where you can put any batch command. While the code is a hack, it works pretty well in practice. This works from mac os x and linux, without the need for wine, or ssh on windows.
David
I'm trying to weed out the issues here:
1) what should the binary installer for Windows look like: * bdist_wininst is the obvious choice but apparently has issues with the newer Windows security stuff -- the real solution is for distutils to be fixed/use something else/ etc -- but is there anyone here that has expertise, interest or time to get in and add-to or fix distutils?
Do binary eggs solve any of this ? maybe, but pip doesn't support that last I saw, and setuptools easy-install is kind-of sort-of broken.
But coming up with the best binary installer tool is orthogonal to the other questions:
2) Should we distribute binaries at all? - No: most people need a whole stack anyway, so they should get all that from the same place: - Enthought - Python(x.y) - Chris Gohlke's repository... - Yes: Some folks just want numpy damn it! Those big ol' distributions are a mess that don't work for everyone. (I'm in that camp.....)
But if we distribute binaries, we really should: - distribute them for 32 & 64 bit - Clearly define what the "official" binaries are, so that third-party packagers can build against that.
Historically, this has been a huge mess on the Mac, as there are many options for Python itself: Apple's builds, Python,org builds, macports, fink, homebrew, ...... Over the years, we in teh MacPython community have tried hard to establish that the python.org builds are the "official" builds that folks should support with binaries -- this has kind-of, sort-of worked, but I still don't see a better solution. (macports et al aren't really the issue, they aren't based on binaries anyway)
On Windows, this has been pretty much a non-issue: MS doesn't provide a build, and the pyton.org builds are well accepted as the default binaries. AFAIU, third party distributions are compatible as well (Active State, Enthought)
The parallel here is that we can establish in the scientific python community (that is, folks building packages based on numpy) that the binaries distributed by numpy are the "official" ones that should be supported by third part binaries. If we succeed in that, then folks can get the rest of the stack from any number of sources.
the python.org python builds are built with the MS compilers, is there any reason we HAVE to build with a open-source stack? Can you build third party packages against an MS-built binary numpy with open-source compilers?
-Chris
Christoph,
On Tue, Feb 5, 2013 at 3:04 PM, Christoph Gohlke cgohlke@uci.edu wrote: [...]
In order not to leave this discussion without a resolution:
Christophe - would you allow us to distribute your numpy binaries for 1.7 from the numpy sourceforge page?
Cheers,
Matthew
I am OK with providing 64 bit "numpy-MKL" binaries (that is numpy compiled with MSVC compilers and linked to Intel's MKL) for official numpy releases.
However:
- There seems to be no real consensus and urge for doing this. Using a
free toolchain capable of building the whole scipy-stack would be much preferred. Several 64 bit Python distributions containing numpy-MKL are already available, some for free.
- Releasing 64 bit numpy without matching scipy binaries would make
little sense to me.
- Please do not just redistribute the binaries from my website and
declare them official. They might contain unreleased fixes from git master and pull requests that are needed for my work and other packages.
- Numpy-MKL requires the Intel runtime DLLs (MKL is linked statically
btw). I ship those with the installers and append the directory containing the DLLs to os.environ['PATH'] in numpy/__init__.py. This is a big no-no according to numpy developers. I don't agree. Anyway, those changes are not in the numpy source repositories.
- My numpy-MKL installers are Python distutils bdist_wininst
installers. That means if Python was installed for all users, installing numpy-MKL on Windows >6.0 will prompt for UAC elevation. Another no-no?
I think that all these things should be possible to fix so that the binary is acceptable for the official NumPy binary.
How exactly do you build the binaries? I wasn't able to find the info at:
http://www.lfd.uci.edu/~gohlke/pythonlibs/
Do you have some scripts to do that? Do you use PowerShell? Or you do it by hand by mouse and clicks in Visual Studio somehow? If I can figure out how to do these builds, I'll be happy to figure out how to automate it and then we can try to figure out a solution that works for NumPy.
Ondrej
On 2/6/2013 10:35 PM, Ondřej Čertík wrote:
Christoph,
On Tue, Feb 5, 2013 at 3:04 PM, Christoph Gohlke cgohlke@uci.edu wrote: [...]
In order not to leave this discussion without a resolution:
Christophe - would you allow us to distribute your numpy binaries for 1.7 from the numpy sourceforge page?
Cheers,
Matthew
I am OK with providing 64 bit "numpy-MKL" binaries (that is numpy compiled with MSVC compilers and linked to Intel's MKL) for official numpy releases.
However:
- There seems to be no real consensus and urge for doing this. Using a
free toolchain capable of building the whole scipy-stack would be much preferred. Several 64 bit Python distributions containing numpy-MKL are already available, some for free.
- Releasing 64 bit numpy without matching scipy binaries would make
little sense to me.
- Please do not just redistribute the binaries from my website and
declare them official. They might contain unreleased fixes from git master and pull requests that are needed for my work and other packages.
- Numpy-MKL requires the Intel runtime DLLs (MKL is linked statically
btw). I ship those with the installers and append the directory containing the DLLs to os.environ['PATH'] in numpy/__init__.py. This is a big no-no according to numpy developers. I don't agree. Anyway, those changes are not in the numpy source repositories.
- My numpy-MKL installers are Python distutils bdist_wininst
installers. That means if Python was installed for all users, installing numpy-MKL on Windows >6.0 will prompt for UAC elevation. Another no-no?
I think that all these things should be possible to fix so that the binary is acceptable for the official NumPy binary.
How exactly do you build the binaries? I wasn't able to find the info at:
http://www.lfd.uci.edu/~gohlke/pythonlibs/
Do you have some scripts to do that? Do you use PowerShell? Or you do it by hand by mouse and clicks in Visual Studio somehow? If I can figure out how to do these builds, I'll be happy to figure out how to automate it and then we can try to figure out a solution that works for NumPy.
Ondrej
My development/build environment is listed at http://www.lfd.uci.edu/~gohlke/pythonlibs/#buildenv. Not that it helps much...
Assuming that Windows 7|8 Pro 64 bit, Visual Studio 2008 Pro SP1 (with 64 bit compiler option), Visual Studio 2010 Pro, Intel Composer XE 2013, 64 bit CPython 2.6, 2.7, 3.2 and 3.3 are installed, the following batch script (no need for PowerShell or an IDE) should build 64 bit numpy-MKL installers when run from within the numpy source directory. I do not really use this script but the "secrets" are there. It can be extended for building eggs and MSIs, 32 bit, and automated testing. Probably not all the libraries listed in site.cfg are needed but this works for me also with scipy and other packages.
@echo off setlocal set ICDIR=C:/Program Files (x86)/Intel/Composer XE rem Work around a bug in numpy distutils. Requires admin privileges fsutil hardlink create "%ICDIR%/mkl/lib/intel64/libiomp5md.lib" "%ICDIR%/compiler/lib/intel64/libiomp5md.lib" fsutil hardlink create "%ICDIR%/mkl/lib/intel64/libifportmd.lib" "%ICDIR%/compiler/lib/intel64/libifportmd.lib" rem Create site.cfg for static linking to 64 bit MKL echo [mkl] > site.cfg echo include_dirs = %ICDIR%/mkl/include >> site.cfg echo library_dirs = %ICDIR%/mkl/lib/intel64;%ICDIR%/compiler/lib/intel64
site.cfg
echo mkl_libs = mkl_lapack95_lp64,mkl_blas95_lp64,mkl_intel_lp64,mkl_intel_thread,mkl_core,libiomp5md,libifportmd
site.cfg
echo lapack_libs = mkl_lapack95_lp64,mkl_blas95_lp64,mkl_intel_lp64,mkl_intel_thread,mkl_core,libiomp5md,libifportmd
site.cfg
rem Build installers using distutils rd /q /s build call C:\Python26\python.exe setup.py bdist_wininst --user-access-control=auto rd /q /s build call C:\Python27\python.exe setup.py bdist_wininst --user-access-control=auto rd /q /s build call C:\Python32\python.exe setup.py bdist_wininst --user-access-control=auto copy /Y build\py3k\dist*.exe dist\ rd /q /s build call C:\Python33\python.exe setup.py bdist_wininst --user-access-control=auto copy /Y build\py3k\dist*.exe dist\ rd /q /s build endlocal
-- Christoph
On Mon, Feb 4, 2013 at 12:36 PM, Matthew Brett matthew.brett@gmail.com wrote:
Hi,
On Mon, Feb 4, 2013 at 12:27 PM, Ondřej Čertík ondrej.certik@gmail.com wrote:
On Sun, Feb 3, 2013 at 2:57 AM, David Cournapeau cournape@gmail.com wrote:
On Sun, Feb 3, 2013 at 12:28 AM, josef.pktd@gmail.com wrote:
On Sat, Feb 2, 2013 at 6:14 PM, Matthew Brett matthew.brett@gmail.com wrote:
Hi,
I see there is no Windows 64 bit installer for the 1.7 rc1.
related: Is there any chance to get newer mingw or mingw-w64 support "soonish"?
The problem has no solution until we can restrict support to windows 7 and above. Otherwise, any acceptable solution would require user to be an admin.
The installer is built with this VM/scripts:
https://github.com/certik/numpy-vendor
currently the VM itself is 32 bit. I think that might be upgraded to 64bit, and maybe it's possible to use 64 bit Wine:
but then we would need to figure out how to use Mingw with 64 bits.
I would be very happy to accept patches to the above repository.
Alternatively, if the actual Windows 64bit machine would have to be used, is there any way to automate the process? Would you compile it from command line (cmd.exe), just like I do in Wine? I would much prefer if we can figure out how to do this in Wine, so that the process can be automated and other people can easily reproduce it.
I wonder whether getting ming64 to work on 64 bit Wine is too hard to get working before the release? I often can't get 32-bit Wine working,
Yep, it took me over a week of work to figure out exactly how to get 32-bit Wine working with mingw and numpy. However, the work is done and you can either run my scripts in the VM, or you can easily reproduce it yourself by consulting the scripts:
https://github.com/certik/numpy-vendor/blob/master/setup-wine.sh https://github.com/certik/numpy-vendor/blob/master/fabfile.py
There are a lot of little tricks involved.
However, as Ralf says, apparently we would need to use MSVC + MKL anyway on 64bits.
Ondrej
On Mon, Feb 4, 2013 at 8:27 PM, Ondřej Čertík ondrej.certik@gmail.com wrote:
On Sun, Feb 3, 2013 at 2:57 AM, David Cournapeau cournape@gmail.com wrote:
On Sun, Feb 3, 2013 at 12:28 AM, josef.pktd@gmail.com wrote:
On Sat, Feb 2, 2013 at 6:14 PM, Matthew Brett matthew.brett@gmail.com wrote:
Hi,
I see there is no Windows 64 bit installer for the 1.7 rc1.
related: Is there any chance to get newer mingw or mingw-w64 support "soonish"?
The problem has no solution until we can restrict support to windows 7 and above. Otherwise, any acceptable solution would require user to be an admin.
The installer is built with this VM/scripts:
I am not sure whether you're replying to my observation or just giving a status update: with mingw-w64 (or recent mingw), the built installer will depend on several .dll (libgcc_s_sjil.dll) that we can't easily distribute. The only place we can realistically put them is in C:\Python$VERSION (or wherever python happens to be installed), and I think it is a very bad idea to install dll from NumPy there. In Windows 2008 and above, one can refer in .pyd where to look for dlls in another directory which is private to numpy.
David
On 2/4/2013 12:59 PM, David Cournapeau wrote:
On Mon, Feb 4, 2013 at 8:27 PM, Ondřej Čertík ondrej.certik@gmail.com wrote:
On Sun, Feb 3, 2013 at 2:57 AM, David Cournapeau cournape@gmail.com wrote:
On Sun, Feb 3, 2013 at 12:28 AM, josef.pktd@gmail.com wrote:
On Sat, Feb 2, 2013 at 6:14 PM, Matthew Brett matthew.brett@gmail.com wrote:
Hi,
I see there is no Windows 64 bit installer for the 1.7 rc1.
related: Is there any chance to get newer mingw or mingw-w64 support "soonish"?
The problem has no solution until we can restrict support to windows 7 and above. Otherwise, any acceptable solution would require user to be an admin.
The installer is built with this VM/scripts:
I am not sure whether you're replying to my observation or just giving a status update: with mingw-w64 (or recent mingw), the built installer will depend on several .dll (libgcc_s_sjil.dll) that we can't easily distribute. The only place we can realistically put them is in C:\Python$VERSION (or wherever python happens to be installed), and I think it is a very bad idea to install dll from NumPy there. In Windows 2008 and above, one can refer in .pyd where to look for dlls in another directory which is private to numpy.
David
If I understand correctly the problem is distributing dependency/runtime DLLs with a package and ensuring the DLLs are found by Windows when the pyd extensions are imported? For numpy-MKL and other packages I include/install the extra DLLs in the package directories and, if necessary, (i) append the package directory to os.environ['PATH'] or (ii) "pre-load" the DLLs into the process using Ctypes, both early in the package's main __init__.py. No admin rights are required.
Christoph
On Mon, Feb 4, 2013 at 3:49 PM, Christoph Gohlke cgohlke@uci.edu wrote:
On 2/4/2013 12:59 PM, David Cournapeau wrote:
On Mon, Feb 4, 2013 at 8:27 PM, Ondřej Čertík ondrej.certik@gmail.com wrote:
On Sun, Feb 3, 2013 at 2:57 AM, David Cournapeau cournape@gmail.com wrote:
On Sun, Feb 3, 2013 at 12:28 AM, josef.pktd@gmail.com wrote:
On Sat, Feb 2, 2013 at 6:14 PM, Matthew Brett matthew.brett@gmail.com wrote:
Hi,
I see there is no Windows 64 bit installer for the 1.7 rc1.
related: Is there any chance to get newer mingw or mingw-w64 support "soonish"?
The problem has no solution until we can restrict support to windows 7 and above. Otherwise, any acceptable solution would require user to be an admin.
The installer is built with this VM/scripts:
I am not sure whether you're replying to my observation or just giving a status update: with mingw-w64 (or recent mingw), the built installer
I was just giving a general status, sorry about not being clear.
will depend on several .dll (libgcc_s_sjil.dll) that we can't easily distribute. The only place we can realistically put them is in C:\Python$VERSION (or wherever python happens to be installed), and I think it is a very bad idea to install dll from NumPy there. In Windows 2008 and above, one can refer in .pyd where to look for dlls in another directory which is private to numpy.
Yes.
David
If I understand correctly the problem is distributing dependency/runtime DLLs with a package and ensuring the DLLs are found by Windows when the pyd extensions are imported? For numpy-MKL and other packages I include/install the extra DLLs in the package directories and, if necessary, (i) append the package directory to os.environ['PATH'] or (ii) "pre-load" the DLLs into the process using Ctypes, both early in the package's main __init__.py. No admin rights are required.
So that seems to be the only option. Is there any other solution?
Ondrej
On Tue, Feb 5, 2013 at 12:27 AM, Ondřej Čertík ondrej.certik@gmail.com wrote:
On Mon, Feb 4, 2013 at 3:49 PM, Christoph Gohlke cgohlke@uci.edu wrote:
On 2/4/2013 12:59 PM, David Cournapeau wrote:
On Mon, Feb 4, 2013 at 8:27 PM, Ondřej Čertík ondrej.certik@gmail.com wrote:
On Sun, Feb 3, 2013 at 2:57 AM, David Cournapeau cournape@gmail.com wrote:
On Sun, Feb 3, 2013 at 12:28 AM, josef.pktd@gmail.com wrote:
On Sat, Feb 2, 2013 at 6:14 PM, Matthew Brett matthew.brett@gmail.com wrote: > Hi, > > I see there is no Windows 64 bit installer for the 1.7 rc1.
related: Is there any chance to get newer mingw or mingw-w64 support "soonish"?
The problem has no solution until we can restrict support to windows 7 and above. Otherwise, any acceptable solution would require user to be an admin.
The installer is built with this VM/scripts:
I am not sure whether you're replying to my observation or just giving a status update: with mingw-w64 (or recent mingw), the built installer
I was just giving a general status, sorry about not being clear.
will depend on several .dll (libgcc_s_sjil.dll) that we can't easily distribute. The only place we can realistically put them is in C:\Python$VERSION (or wherever python happens to be installed), and I think it is a very bad idea to install dll from NumPy there. In Windows 2008 and above, one can refer in .pyd where to look for dlls in another directory which is private to numpy.
Yes.
David
If I understand correctly the problem is distributing dependency/runtime DLLs with a package and ensuring the DLLs are found by Windows when the pyd extensions are imported? For numpy-MKL and other packages I include/install the extra DLLs in the package directories and, if necessary, (i) append the package directory to os.environ['PATH'] or (ii) "pre-load" the DLLs into the process using Ctypes, both early in the package's main __init__.py. No admin rights are required.
So that seems to be the only option. Is there any other solution?
I don't think it is an acceptable solution in general: modifying the PATH in a package is a big no-no, even worse than adding the dll in $prefix. I have not thought about pre-loading, but if it works, that may be a workaround. That's a very ugly workaround, though...
David
participants (13)
-
Andrea Gavana -
Charles R Harris -
Chris Barker - NOAA Federal -
Christoph Gohlke -
Dag Sverre Seljebotn -
David Cournapeau -
Jonathan T. Niehof -
josef.pktd＠gmail.com -
Matthew Brett -
Nathaniel Smith -
Ondřej Čertík -
Ralf Gommers -
Robert Kern