Note: this is a reply to the thread, and not specifically to Ralf's comments (but those are included).

After reading all the replies, the first question that comes to mind is: should SciPy have *any* datasets?

I think this question has already been answered: we have had functions that return images in scipy.misc for a long time, and I don't recall anyone ever suggesting that these be removed.  (Well, there was lena(), but I don't think anyone had a problem with adding a replacement image.)  And the pull request for the ECG dataset has been merged (added to scipy.misc), so there is current support among the developers for providing datasets.

So the remaining questions are:
   (1) Where do the datasets reside?
   (2) What are the criteria for adding a new datasets?

Here's my 2¢:

(1) Where do the datasets reside?

My preference is to keep all the datasets in the top-level module scipy.datasets. Robert preferred this module for discoverability, and I agree.  By having all the datasets in one place, anyone can easily see what is available.  Teachers and others developing educational material know where to find source material for examples.  Developers, too, can easily look for examples to use in our docstrings or tutorials. (By the way, adding examples to the docstrings of all functions is an ongoing effort: https://github.com/scipy/scipy/issues/7168.)

Also, there are many well-known datasets that could be used as examples for multiple scipy packages.  For a concrete example, a dataset that I could see adding to scipy is the Hald cement dataset.  SciPy should eventually have an implementation of the PCA decomposition, and it could conceivably live in scipy.linalg.  It would be reasonable to use the Hald data in the docstrings of the new PCA function(s) (cf. https://www.mathworks.com/help/stats/pca.html).  At the same time, the Hald data could enrich the docstrings of some functions in scipy.stats.

Similarly, Fisher's iris dataset provides a well-known example that could be used in docstrings in both scipy.cluster and scipy.stats.


(2) What are the criteria for adding a new datasets?

So far, the only compelling reason I can see to even have datasets is to have interesting examples in the docstrings (or at least in our tutorials).  For example, the docstring for scipy.ndimage.gaussian_filter and several other transformations in ndimage use the image returned by scipy.misc.ascent():

    https://docs.scipy.org/doc/scipy/reference/generated/scipy.ndimage.gaussian_filter.html

I could see the benefit of having well-known datasets such as Fisher's iris data, the Hald cement data, and some version of a sunspot activity time series, to be used in the docstrings in scipy.stats, scipy.signal, scipy.cluster, scipy.linalg, and elsewhere.

Stéfan expressed regret about including datasets in sciki-image.  The main issue seems to be "bloat".  Scikit-image is an image processing library, so the datasets used there are likely all images, and there is a minimum size for a sample image to be useful as an example.  For scipy, we already have two images, and I don't think we'll need more.  The newly added ECG dataset is 116K (which is less than the existing image datasets: "ascent.dat" is 515K and "face.dat" is 1.5M).  The potential datasets that I mentioned above (Hald, iris, sunspots) are all very small.   If we are conservative about what we include, and focus on datasets chosen specifically to demonstrate scipy functionality, we should be able to avoid dataset bloat.

This leads to my proposal for the criteria for adding a dataset:

(a) Not too big.  The size of a dataset should not exceed $MAX (but I don't have a good suggestion for what $MAX should be at the moment).
(b) The dataset should be well-known, where "well-known" means that the dataset is one that is already widely used as an example and many people will know it by name (e.g. the iris dataset), or the dataset is a sample of a common signal type or format (e.g an ECG signal, or an image such as misc.ascent).
(c) We actually *use* the dataset in one of *our* docstrings or tutorials.  I don't think our datasets package should become a repository of interesting scientific data with no connection to the scipy code.  Its purpose should be to enrich our documentation.  (Note that by this criterion, the recently added ECG signal would not qualify!)

To summarize: I'm in favor scipy.datasets, a conservatively curated subpackage containing well-known datasets.


Warren


P.S. I should add that I'm not in favor putting code in scipy that fetches data from the web.  That type of data retrieval could be useful, but it seems more appropriate for a package that is independent of scipy.