Following on from recent discussion here and on the numfocus list, I'm trying to work out the set of packages that should make up a standardised 'scipy stack'. We've determined that Python, numpy, scipy, matplotlib and IPython are to be included. Then there's a list that have got a 'maybe': pandas, statsmodels, sympy, scikits-learn, scikits-image, PyTables, h5py, NetworkX, nose, basemap & netCDF4. My aim is to have a general set of packages that you can do useful work with, and will stand up to the competition (particularly Matlab & R), but without gaining too many subject-specific packages. But I don't know what's generally useful and what's subject specific. Vote at: http://www.doodle.com/ma6rnpnbfc6wivu9 It's set up so you can vote for or against a package, or abstain if you're not sure - I've abstained on most of them myself. Thanks, Thomas
On Wed, Oct 3, 2012 at 12:06 PM, Thomas Kluyver <takowl@gmail.com> wrote:
Following on from recent discussion here and on the numfocus list, I'm trying to work out the set of packages that should make up a standardised 'scipy stack'. We've determined that Python, numpy, scipy, matplotlib and IPython are to be included. Then there's a list that have got a 'maybe': pandas, statsmodels, sympy, scikits-learn, scikits-image, PyTables, h5py, NetworkX, nose, basemap & netCDF4.
My aim is to have a general set of packages that you can do useful work with, and will stand up to the competition (particularly Matlab & R), but without gaining too many subject-specific packages. But I don't know what's generally useful and what's subject specific.
Vote at: http://www.doodle.com/ma6rnpnbfc6wivu9
It's set up so you can vote for or against a package, or abstain if you're not sure - I've abstained on most of them myself.
Why is the default no, instead of abstain (Yes)? I had to go back to fix where I didn't vote. Josef
Thanks, Thomas _______________________________________________ SciPy-User mailing list SciPy-User@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-user
On 3 October 2012 17:52, <josef.pktd@gmail.com> wrote:
Why is the default no, instead of abstain (Yes)?
Because this isn't exactly the use case Doodle is designed for. Sorry about that, and thanks for checking your answer. Anyone else who did the same, please take a moment to edit your response. Early results suggest pandas, sympy, h5py and nose are the most popular. Thanks, Thomas
Just a thought, although late in the process; Is there in the default stack any toolkit to help create simple interactive GUIs, like e.g. Traits(ui)? Nothing overly complicated, but simple dialogues etc. would be great for creating simple apps for e.g. teaching. I know IDL has it and it is used quite frequently (yes, I'm an astronomer). Cheers Emil On 10/03/2012 06:52 PM, josef.pktd@gmail.com wrote:
On Wed, Oct 3, 2012 at 12:06 PM, Thomas Kluyver <takowl@gmail.com> wrote:
Following on from recent discussion here and on the numfocus list, I'm trying to work out the set of packages that should make up a standardised 'scipy stack'. We've determined that Python, numpy, scipy, matplotlib and IPython are to be included. Then there's a list that have got a 'maybe': pandas, statsmodels, sympy, scikits-learn, scikits-image, PyTables, h5py, NetworkX, nose, basemap & netCDF4.
My aim is to have a general set of packages that you can do useful work with, and will stand up to the competition (particularly Matlab & R), but without gaining too many subject-specific packages. But I don't know what's generally useful and what's subject specific.
Vote at: http://www.doodle.com/ma6rnpnbfc6wivu9
It's set up so you can vote for or against a package, or abstain if you're not sure - I've abstained on most of them myself. Why is the default no, instead of abstain (Yes)?
I had to go back to fix where I didn't vote.
Josef
Thanks, Thomas _______________________________________________ SciPy-User mailing list SciPy-User@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-user
SciPy-User mailing list SciPy-User@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-user
On 3 October 2012 21:41, Thøger Rivera-Thorsen <trive@astro.su.se> wrote:
Is there in the default stack any toolkit to help create simple interactive GUIs, like e.g. Traits(ui)? Nothing overly complicated, but simple dialogues etc. would be great for creating simple apps for e.g. teaching. I know IDL has it and it is used quite frequently (yes, I'm an astronomer).
Tkinter is included as part of the Python standard library, so you can build simple GUIs. For quickly presenting dialogs, you could easily install easygui (http://easygui.sourceforge.net/ ), which builds on Tkinter, but I don't think it should be part of the standard. I don't know how either compare to TraitsUI, which I haven't used. Thomas
On 10/3/2012 9:06 AM, Thomas Kluyver wrote:
Following on from recent discussion here and on the numfocus list, I'm trying to work out the set of packages that should make up a standardised 'scipy stack'. We've determined that Python, numpy, scipy, matplotlib and IPython are to be included. Then there's a list that have got a 'maybe': pandas, statsmodels, sympy, scikits-learn, scikits-image, PyTables, h5py, NetworkX, nose, basemap & netCDF4.
My aim is to have a general set of packages that you can do useful work with, and will stand up to the competition (particularly Matlab & R), but without gaining too many subject-specific packages. But I don't know what's generally useful and what's subject specific.
Vote at: http://www.doodle.com/ma6rnpnbfc6wivu9
It's set up so you can vote for or against a package, or abstain if you're not sure - I've abstained on most of them myself.
Thanks, Thomas
Hi, it was mentioned before: none of the suggested packages can read or write image files on their own, except for matplotlib's built-in PNG support. Matplotlib, Scipy and skimage depend on other, optional packages or binaries for image I/O: PIL, FreeImage, GDAL, PyQt. Christoph
On 3 October 2012 22:06, Christoph Gohlke <cgohlke@uci.edu> wrote:
it was mentioned before: none of the suggested packages can read or write image files on their own, except for matplotlib's built-in PNG support. Matplotlib, Scipy and skimage depend on other, optional packages or binaries for image I/O: PIL, FreeImage, GDAL, PyQt.
If we include scikits-image (which looks unlikely based on the current poll results), we had agreed to specify FreeImage, or possibly one of FreeImage and PIL. Matplotlib will need at least one backend installed, and the documentation says "Most backends support png, pdf, ps, eps and svg." That seems adequate. For saving images, there's less need to require a range of formats than if loading them is a key feature. Thomas
On 10/3/2012 3:09 PM, Thomas Kluyver wrote:
On 3 October 2012 22:06, Christoph Gohlke <cgohlke@uci.edu> wrote:
it was mentioned before: none of the suggested packages can read or write image files on their own, except for matplotlib's built-in PNG support. Matplotlib, Scipy and skimage depend on other, optional packages or binaries for image I/O: PIL, FreeImage, GDAL, PyQt.
If we include scikits-image (which looks unlikely based on the current poll results), we had agreed to specify FreeImage, or possibly one of FreeImage and PIL.
Matplotlib will need at least one backend installed, and the documentation says "Most backends support png, pdf, ps, eps and svg." That seems adequate. For saving images, there's less need to require a range of formats than if loading them is a key feature.
Thomas
I thought PIL was out of question because it's abandonware. Did anyone check if the triple-licensing option of FreeImage (GPLv2, GPLv3, or FIPL) is compatible with the Scipy stack? Also, FreeImage is not a Python package. Pdf, ps, eps and svg are vector graphics formats, not adequate for image IO. Christoph
On Wed, Oct 3, 2012 at 11:27 PM, Christoph Gohlke <cgohlke@uci.edu> wrote:
On 10/3/2012 3:09 PM, Thomas Kluyver wrote:
On 3 October 2012 22:06, Christoph Gohlke <cgohlke@uci.edu> wrote:
it was mentioned before: none of the suggested packages can read or write image files on their own, except for matplotlib's built-in PNG support. Matplotlib, Scipy and skimage depend on other, optional packages or binaries for image I/O: PIL, FreeImage, GDAL, PyQt.
If we include scikits-image (which looks unlikely based on the current poll results), we had agreed to specify FreeImage, or possibly one of FreeImage and PIL.
Matplotlib will need at least one backend installed, and the documentation says "Most backends support png, pdf, ps, eps and svg." That seems adequate. For saving images, there's less need to require a range of formats than if loading them is a key feature.
Thomas
I thought PIL was out of question because it's abandonware.
Pillow is a maintained, drop-in fork: http://pypi.python.org/pypi/Pillow/ -- Robert Kern
On 10/3/2012 3:34 PM, Robert Kern wrote:
On Wed, Oct 3, 2012 at 11:27 PM, Christoph Gohlke <cgohlke@uci.edu> wrote:
On 10/3/2012 3:09 PM, Thomas Kluyver wrote:
On 3 October 2012 22:06, Christoph Gohlke <cgohlke@uci.edu> wrote:
it was mentioned before: none of the suggested packages can read or write image files on their own, except for matplotlib's built-in PNG support. Matplotlib, Scipy and skimage depend on other, optional packages or binaries for image I/O: PIL, FreeImage, GDAL, PyQt.
If we include scikits-image (which looks unlikely based on the current poll results), we had agreed to specify FreeImage, or possibly one of FreeImage and PIL.
Matplotlib will need at least one backend installed, and the documentation says "Most backends support png, pdf, ps, eps and svg." That seems adequate. For saving images, there's less need to require a range of formats than if loading them is a key feature.
Thomas
I thought PIL was out of question because it's abandonware.
Pillow is a maintained, drop-in fork:
Seriously, only few of PIL's bugs have been fixed in Pillow (it's a fork to "foster packaging improvements"), there's no support for Python 3, no new features are planned, and the test suite was removed. Christoph
On 3 October 2012 23:27, Christoph Gohlke <cgohlke@uci.edu> wrote:
Did anyone check if the triple-licensing option of FreeImage (GPLv2, GPLv3, or FIPL) is compatible with the Scipy stack? Also, FreeImage is not a Python package.
IANAL, but I think the FIPL is acceptable. It looks roughly equivalent to LGPL. http://freeimage.sourceforge.net/freeimage-license.txt
Pdf, ps, eps and svg are vector graphics formats, not adequate for image IO.
For saving plots, vector formats + png seems adequate to me. PNG is lossless, so it can be converted to other raster formats if there's a specific need. And the standard is a minimum: distributions are free to support other image formats beyond these. For loading images, I agree that these options would not be adequate - at least JPEG support is important. But if scikits-image is not included, loading image files is not a key concern, so I don't think we need to specify it. Thanks, Thomas
On Wed, Oct 3, 2012 at 12:06 PM, Thomas Kluyver <takowl@gmail.com> wrote:
Following on from recent discussion here and on the numfocus list, I'm trying to work out the set of packages that should make up a standardised 'scipy stack'. We've determined that Python, numpy, scipy, matplotlib and IPython are to be included. Then there's a list that have got a 'maybe': pandas, statsmodels, sympy, scikits-learn, scikits-image, PyTables, h5py, NetworkX, nose, basemap & netCDF4.
My aim is to have a general set of packages that you can do useful work with, and will stand up to the competition (particularly Matlab & R), but without gaining too many subject-specific packages. But I don't know what's generally useful and what's subject specific.
Vote at: http://www.doodle.com/ma6rnpnbfc6wivu9
It's set up so you can vote for or against a package, or abstain if you're not sure - I've abstained on most of them myself.
Why I'm in favor of a "Big Scipy": Using Travis's popularity criterion: google has for "from scipy import stats" "About 104,000 results" scipy.stats is a bit of an outlier among the scipy subpackages in that it is more application oriented. I uses many tools from other scipy.subpackages. scipy.stats is in turn used by many application packages, if they don't want to bother coding a version of the statistics themselves. If you are in a field with a strong python background, then there are field specific packages available, cars, sherpa in the recent spectra discussion, nipy/pymvpa, pysal, ... If you are not in one of those python fields (or want to try something non-standard), then you have to use a general purpose library, or code it yourself. scikit-learn, statsmodels and scikit-image try to be the general purpose extension of scipy (the package), and there is a lot of useful and reusable code. for example, clustering with sklearn http://spikesort.org/docs/intro.html#installation a linear regression, or a polyfit if you have outliers use statsmodels that's not field specific. (I'm not using scikits-image, but I assume there are similar features, given the mailing list) (I would also like to use a scikits-signal, but it's still is vapor-ware.) As a user I don't care (much) about a new meta-package, python-xy and Gohlke have (almost) all I need an easy_install away, and a lot more than is under discussion here. Where I do see a potentially big advantage as a maintainer of statsmodels is in code sharing and being able to rely on more consistent package versions by users. Currently we are reluctant to add any additional dependencies to statsmodels not only because it requires more work by users, but also because it requires work for us to keep track of changes across versions of the different packages. We currently maintain compatibility modules for python between 2.5 and 3.2, and for numpy >= 1.4, scipy >= 0.7 and pandas > 0.7.1. Increasing the number of dependencies increases the number of version combinations that need to be tested. That's also a good reason for me not to split up scipy, keeping track of the versions of 8 (linalg, optimize, signal, sparse, stats, fftpack, integrate, interpolate, special and maybe some others) packages sounds like a lot of fun. (I wouldn't mind splitting off scipy.stats.) I would prefer to go the other way, and have a "scipy-big", where I can use any functions from any of the packages without having to worry too much about whether they are available on a users machine or about version compatibilities across packages. As a statsmodels developer I would be glad about the additional advertising and the hopefully faster development of or convergence to a standard through the scipy-stack discussed here, but, at least in the "data-analysis" area, I think we are well on our way to get to the "big-scipy" and fill in the major gaps compared to other languages or data analysis packages. Josef
Thanks, Thomas _______________________________________________ SciPy-User mailing list SciPy-User@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-user
On 4 October 2012 02:00, <josef.pktd@gmail.com> wrote:
Where I do see a potentially big advantage as a maintainer of statsmodels is in code sharing and being able to rely on more consistent package versions by users.
That's a good point: one of my other aims is that packages can more comfortably rely on things in the specification - similar to relying on the Python standard library. For example, I recall statsmodels was looking at adding formula support: I imagine there are tools in Sympy that you could use in this. It looks likely that Sympy will be part of the specification, so maybe there's less need to provide fallback functionality for when it's not installed. Thomas
On Thu, Oct 4, 2012 at 10:38 AM, Thomas Kluyver <takowl@gmail.com> wrote:
On 4 October 2012 02:00, <josef.pktd@gmail.com> wrote:
Where I do see a potentially big advantage as a maintainer of statsmodels is in code sharing and being able to rely on more consistent package versions by users.
That's a good point: one of my other aims is that packages can more comfortably rely on things in the specification - similar to relying on the Python standard library. For example, I recall statsmodels was looking at adding formula support: I imagine there are tools in Sympy that you could use in this. It looks likely that Sympy will be part of the specification, so maybe there's less need to provide fallback functionality for when it's not installed.
Those formulae have very different semantics. Sympy would probably not have saved much, if any, code. http://patsy.readthedocs.org/en/latest/formulas.html -- Robert Kern
On 4 October 2012 10:43, Robert Kern <robert.kern@gmail.com> wrote:
Those formulae have very different semantics. Sympy would probably not have saved much, if any, code.
OK, I guess that was a poor example. But the larger point is being able to depend on a larger set of packages, rather than reimplementing bits of those packages to make those dependencies optional. Thomas
On Thu, Oct 4, 2012 at 10:38 AM, Thomas Kluyver <takowl@gmail.com> wrote:
On 4 October 2012 02:00, <josef.pktd@gmail.com> wrote:
Where I do see a potentially big advantage as a maintainer of statsmodels is in code sharing and being able to rely on more consistent package versions by users.
That's a good point: one of my other aims is that packages can more comfortably rely on things in the specification - similar to relying on the Python standard library.
This suggests another possible way of coming up with the base package list... if a package is already included in all of Python(x,y), EPD, Anaconda, Debian, Redhat, <whatever other relevant distros I'm missing> then practically speaking it sticking it in the first version of the spec won't cause any problems for anybody, because everyone's already distributing it. But it will document that everyone is distributing it, which is useful for tutorials, making decisions about dependencies, etc. (Python: batteries included!) -n
On 4 October 2012 13:07, Nathaniel Smith <njs@pobox.com> wrote:
This suggests another possible way of coming up with the base package list... if a package is already included in all of Python(x,y), EPD, Anaconda, Debian, Redhat, <whatever other relevant distros I'm missing>
The the question becomes one of which distros are relevant. If we count EPD Free, for example, only nose (of the packages in the poll) is common to all the distributions at present. For Linux distributions, it's trickier: I have a wealth of packages available from the Ubuntu repositories, but they're mostly not installed by default - I'm not sure if even numpy is in a default installation. The intention is to make a metapackage called something like scipy-stack, which will pull in all the relevant packages. But for now, there's no set of packages you can assume will be installed together. Thomas
On Thu, Oct 4, 2012 at 1:19 PM, Thomas Kluyver <takowl@gmail.com> wrote:
On 4 October 2012 13:07, Nathaniel Smith <njs@pobox.com> wrote:
This suggests another possible way of coming up with the base package list... if a package is already included in all of Python(x,y), EPD, Anaconda, Debian, Redhat, <whatever other relevant distros I'm missing>
The the question becomes one of which distros are relevant. If we count EPD Free, for example, only nose (of the packages in the poll) is common to all the distributions at present.
I think Nathaniel meant included in the official repos, not in the single cdrom distribution (otherwise, you would indeed get an near-empty set because of Ubuntu) David
On 4 October 2012 13:38, David Cournapeau <cournape@gmail.com> wrote:
I think Nathaniel meant included in the official repos, not in the single cdrom distribution (otherwise, you would indeed get an near-empty set because of Ubuntu)
But if the criterion is 'available from repositories for all relevant distributions', then there's a very large set of packages we could specify. Thomas
On Thu, Oct 4, 2012 at 2:03 PM, Thomas Kluyver <takowl@gmail.com> wrote:
On 4 October 2012 13:38, David Cournapeau <cournape@gmail.com> wrote:
I think Nathaniel meant included in the official repos, not in the single cdrom distribution (otherwise, you would indeed get an near-empty set because of Ubuntu)
But if the criterion is 'available from repositories for all relevant distributions', then there's a very large set of packages we could specify.
I thought the idea was closer to take the intersection of all the distros (rh, ubuntu, epd free, anaconda, etc...) as a working basis. David
OK, based on the responses so far from the poll, here's a new draft of the standard. It's rather smaller than the previous draft (when we were using the name Pylab), but not completely minimalist. I'm fairly happy with the general shape of it. https://gist.github.com/3833499 The biggest remaining question (as I see it) is the hdf5 libraries. Both have got a somewhat mixed response on the poll, although h5py has a bit more support than PyTables. This did come up before, but let's hear more voices on the question. Should we specify neither, one, or both? Thanks,
On 4 October 2012 14:27, Thomas Kluyver <takowl@gmail.com> wrote:
For reference, a few notes on how it matches up to existing distributions. - Anaconda, EPD full & WinPython already meet that list - EPD Free does not currently include pandas or sympy. - Python(x,y) has older versions of pandas & IPython, but a new release is coming soon. - Ubuntu has older versions of the scipy library, pandas & IPython. The new release later this month will have the requisite versions of all three. Thanks, Thomas
On 4 October 2012 14:27, Thomas Kluyver <takowl@gmail.com> wrote:
The biggest remaining question (as I see it) is the hdf5 libraries. Both have got a somewhat mixed response on the poll, although h5py has a bit more support than PyTables. This did come up before, but let's hear more voices on the question. Should we specify neither, one, or both?
Discussion on the numfocus list has come to the conclusion that we should either specify both h5py and PyTables, or neither. Please register your opinion on this new poll: http://www.misterpoll.com/polls/568484 To be clear, I'm using all these polls to gauge what a larger number of people think. It's like Wikipedia's "!voting" model - the option with the most votes doesn't automatically win, but it's used to form a consensus. Thanks, Thomas
Hi, On Thu, Oct 4, 2012 at 9:39 PM, Thomas Kluyver <takowl@gmail.com> wrote:
On 4 October 2012 14:27, Thomas Kluyver <takowl@gmail.com> wrote:
The biggest remaining question (as I see it) is the hdf5 libraries. Both have got a somewhat mixed response on the poll, although h5py has a bit more support than PyTables. This did come up before, but let's hear more voices on the question. Should we specify neither, one, or both?
Discussion on the numfocus list has come to the conclusion that we should either specify both h5py and PyTables, or neither. Please register your opinion on this new poll: http://www.misterpoll.com/polls/568484
Why do you need to use a polling service that has this potentially malicious requirement that "You must disable safe mode to view this content." Regards, -eat
To be clear, I'm using all these polls to gauge what a larger number of people think. It's like Wikipedia's "!voting" model - the option with the most votes doesn't automatically win, but it's used to form a consensus.
Thanks, Thomas _______________________________________________ SciPy-User mailing list SciPy-User@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-user
The latest poll shows slightly more support for not specifying the HDF5 libraries in the standard at the moment (15 for specifying both, 19 for specifying neither). This is also the option I think is best. If there are any points about this or about the rest of the standard that you think we haven't already discussed, then please raise them now. As it stands, the draft standard we've worked out includes numpy, scipy, matplotlib, ipython, pandas, sympy and nose (plus a few dependencies). I think that's quite a good starting point, so this is kind of a last call for comments before we declare the standard done. Thanks, Thomas
On Sun, Oct 7, 2012 at 3:50 PM, Thomas Kluyver <takowl@gmail.com> wrote:
If there are any points about this or about the rest of the standard that you think we haven't already discussed, then please raise them now. As it stands, the draft standard we've worked out includes numpy, scipy, matplotlib, ipython, pandas, sympy and nose (plus a few dependencies). I think that's quite a good starting point, so this is kind of a last call for comments before we declare the standard done.
+1 for moving on with this fairly conservative but solid base. Once we sort out the kinks with this more targeted core, we can revisit this with an eye towards a more expanded definition of the spec. Kudos to you for hitting a good balance of discussion and action! f
Hello Thomas, I know that pandas can use HDF5 as a persistence backend. How optional is this? If this is completely optional than I would say that we should move ahead with what you recommend w/ pandas sans hdf5. If this is not optional than I would either suggest dropping pandas or including PyTables and h5py as well. Be Well Anthony On Sun, Oct 7, 2012 at 6:40 PM, Fernando Perez <fperez.net@gmail.com> wrote:
On Sun, Oct 7, 2012 at 3:50 PM, Thomas Kluyver <takowl@gmail.com> wrote:
If there are any points about this or about the rest of the standard that you think we haven't already discussed, then please raise them now. As it stands, the draft standard we've worked out includes numpy, scipy, matplotlib, ipython, pandas, sympy and nose (plus a few dependencies). I think that's quite a good starting point, so this is kind of a last call for comments before we declare the standard done.
+1 for moving on with this fairly conservative but solid base. Once we sort out the kinks with this more targeted core, we can revisit this with an eye towards a more expanded definition of the spec.
Kudos to you for hitting a good balance of discussion and action!
f
--
Hi Anthony, On Monday, 8 October 2012 02:02:09 UTC+1, Anthony Scopatz wrote:
I know that pandas can use HDF5 as a persistence backend. How optional is this? If this is completely optional than I would say that we should move ahead with what you recommend w/ pandas sans hdf5. If this is not optional than I would either suggest dropping pandas or including PyTables and h5py as well.
As Robert says, it's completely optional. If PyTables is installed, pandas can store objects in HDF5, but if not, the rest of pandas still works perfectly. That also allows pandas to support Python 3, while PyTables doesn't yet. Thanks all, Thomas
On 8 October 2012 00:40, Fernando Perez <fperez.net@gmail.com> wrote:
+1 for moving on with this fairly conservative but solid base. Once we sort out the kinks with this more targeted core, we can revisit this with an eye towards a more expanded definition of the spec.
Kudos to you for hitting a good balance of discussion and action!
Thanks, Fernando. I think the next step is to start reworking the 'new' scipy.org website (http://scipy.github.com/ ) to focus on the stack, rather than scipy-the-package. I've just kicked that off with a pull request replacing the 'download' page with an 'install' page: https://github.com/scipy/scipy.org-new/pull/3 I'd like to invite anyone with an interest in this (and I know there are a lot of you) to get involved with the website. A few of the things we'll need: - Update the front page to promote the Scipy stack we've agreed on: adding Pandas & Sympy, rearranging the current distinction of 'core projects' vs. 'related projects'. - A page describing the Scipy stack specification. - A new separate page (or pages) about scipy-the-package. - Some general design work wouldn't go amiss - do we need the breadcrumb bar? Can we improve top level navigation? Best wishes, Thomas
Hey Thomas, On Mon, Oct 8, 2012 at 7:21 AM, Thomas Kluyver <takowl@gmail.com> wrote:
Thanks, Fernando. I think the next step is to start reworking the 'new' scipy.org website (http://scipy.github.com/ ) to focus on the stack, rather than scipy-the-package. I've just kicked that off with a pull request replacing the 'download' page with an 'install' page:
Kyle Mandli had already been pushing hard on this front, it would be great if the two efforts could play off each other, as he'd spent a good amount of time already on this idea... f
On Mon, Oct 8, 2012 at 8:25 PM, Fernando Perez <fperez.net@gmail.com> wrote:
Hey Thomas,
On Mon, Oct 8, 2012 at 7:21 AM, Thomas Kluyver <takowl@gmail.com> wrote:
Thanks, Fernando. I think the next step is to start reworking the 'new' scipy.org website (http://scipy.github.com/ ) to focus on the stack, rather than scipy-the-package. I've just kicked that off with a pull request replacing the 'download' page with an 'install' page:
Kyle Mandli had already been pushing hard on this front, it would be great if the two efforts could play off each other, as he'd spent a good amount of time already on this idea...
Is his work somewhere public? He hasn't made any PRs yet. Ralf
On Mon, Oct 8, 2012 at 11:30 AM, Ralf Gommers <ralf.gommers@gmail.com> wrote:
Is his work somewhere public? He hasn't made any PRs yet.
Dunno, we had long discussions at SciPy'12 about planning and then there were a few threads on the lists after that. But I got swamped and tuned out, we'll have to wait for him to pitch in with info.
I'd personally make it the lowest common denominator of Python(xy), Anaconda, EPD-full (rather than free - I think free has too little to be useful and we're not mandating that any particular scipy-stack implementation ought to be free - but that's probably somewhat contentious), Sage etc. As far as linux distros go I'd consider the Debian/Ubuntu repositories (as being the most comprehensive linux distros) but would stop short of requiring packages to be available on RH or other linuxes. If you mandate pip / easy-install you could probably have the metapackage easy-install anything that wasn't available in the distro. Any linux worth it's salt should be build- capable. To ease installation of my python-microscopy package I wrote an Ubuntu based install script for a 'scipy-stack' like environment which does this. It can be seen at http://code.google.com/p/python-microscopy/source/browse/PYME/install_depend... Even though it's been dismissed as 'too hard' I still think there is a strong case for specifying Scipy-stack to include a compiler - EPD manages to do this well on both x32 and x64 using mingw so it's definitely technically possible. More importantly, retro-fitting mingw & msys to an existing distro can be quite painful (the last time I tried I needed to edit the source of distutils to make it invoke mingw by default before it would work in complex build situations or with easy-install etc). In my opinion the omission of a compiler stops the distribution from being easily extendible if the user wants to experiment with other packages (or mandates that someone maintain a scipy-multiverse with compiled versions of all the possible packages and writes a suitable search and install interface). I'd love to be able to specify the 'Scipy stack' as a broader alternative to EPD for people installing my packages under windows, but without build capability it's not going to happen. cheers, David ________________________________ From: Thomas Kluyver <takowl@gmail.com> To: SciPy Users List <scipy-user@scipy.org> Sent: Friday, 5 October 2012 1:19 AM Subject: Re: [SciPy-User] Scipy stack: standard packages (poll) On 4 October 2012 13:07, Nathaniel Smith <njs@pobox.com> wrote:
This suggests another possible way of coming up with the base package list... if a package is already included in all of Python(x,y), EPD, Anaconda, Debian, Redhat, <whatever other relevant distros I'm missing>
The the question becomes one of which distros are relevant. If we count EPD Free, for example, only nose (of the packages in the poll) is common to all the distributions at present. For Linux distributions, it's trickier: I have a wealth of packages available from the Ubuntu repositories, but they're mostly not installed by default - I'm not sure if even numpy is in a default installation. The intention is to make a metapackage called something like scipy-stack, which will pull in all the relevant packages. But for now, there's no set of packages you can assume will be installed together. Thomas _______________________________________________ SciPy-User mailing list SciPy-User@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-user
On 5 October 2012 00:23, David Baddeley <david_baddeley@yahoo.com.au> wrote:
Even though it's been dismissed as 'too hard' I still think there is a strong case for specifying Scipy-stack to include a compiler - EPD manages to do this well on both x32 and x64 using mingw so it's definitely technically possible.
It's certainly technically possible, but it's nonetheless a major requirement for anyone trying to make a compliant distribution. And you can do a lot of useful stuff without needing a compiler. As for extending the environment, many packages are already available in various compiled forms (.exe installers, debian packages, pypm, etc.). We may well try to standardise a larger environment with a compiler later, but I don't want to get into that at the moment. For now, let's focus on a stack that can be used without needing a compiler. I appreciate that doesn't resolve your build case, but we can't solve every problem at once. Thanks, Thomas
participants (13)
-
Anthony Scopatz -
Christoph Gohlke -
David Baddeley -
David Cournapeau -
eat -
Fernando Perez -
josef.pktd＠gmail.com -
Nathaniel Smith -
Ralf Gommers -
Robert Kern -
Robert Kern -
Thomas Kluyver -
Thøger Rivera-Thorsen