On Wed, Jan 4, 2012 at 06:37, Travis Oliphant <travis@continuum.io> wrote:
So, my (off the top of my head) take on what should be core scipy is:
fftpack stats io special optimize] linalg lib.blas lib.lapack misc
I think the other packages should be maintained, built and distributed as
scipy-constants scipy-integrate scipy-cluster scipy-ndimage scipy-spatial scipy-odr scipy-sparse scipy-maxentropy scipy-signal scipy-weave (actually I think weave should be installed separately and/or merged with other foreign code integration tools like fwrap, f2py, etc.)
Then, we could create a scipy superpack to install it all together. What issues do people see with a plan like this?
The main technical issue/decision is how to split up the "physical" packages themselves. Do we use namespace packages, such that scipy.signal will still be imported as "from scipy import signal", or do we rename the packages such that each one is its own top-level package? It's important to specify this when making a proposal because each imposes different costs that we may want to factor into how we divide up the packages. I think the lesson we've learned from scikits (and ETS, for that matter) is that this community at least does not want to use namespace packages. Some of this derives from a distate of setuptools, which is used in the implementation, but a lot of it derives from the very concept of namespace packages independent of any implementation. Monitoring the scikit-learn and pystatsmodels mailing lists, I noticed that a number of installation problems stemmed just from having the top-level package being "scikits" and shared between several packages. This is something that can only be avoided by not using namespace packages altogether. There are also technical issues that cut across implementations. Namely, the scipy/__init__.py files need to be identical between all of the packages. Maintaining non-empty identical __init__.py files is not feasible. We don't make many changes to it these days, but we won't be able to make *any* changes ever again. We could empty it out, if we are willing to make this break with backwards compatibility once. Going with unique top-level packages, do we use a convention like "scipy_signal", at least for the packages being broken out from the current monolithic scipy? Do we provide a proxy package hierarchy for backwards compatibility (e.g. having proxy modules like scipy/signal/signaltools.py that just import everything from scipy_signal/signaltools.py) like Enthought does with etsproxy after we split up ETS? -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco