[SciPy-Dev] New subpackage: scipy.data

[josef.pktd at gmail.com]

On Thu, Mar 29, 2018 at 7:54 PM, Ilhan Polat <ilhanpolat at gmail.com> wrote:
> Would a separate repo scipy-datasets help ? Then something like
>
> try:
>      importing
> except:
>     warn('I'm off to interwebz')
>     download from the repo
>
> might be feasible. The download part can either be that particular dataset
> or the whole scipy-datasets clone.
>

IMO:

It depends on the scale where this should go.
I don't think it's worth it (maintaining and installing another
package or repo) for scipy
given that scipy is mostly a basic numerical library and not driven by specific
applications.

For most areas there should be already some online repos or packages and
it would be enough to have the accessing functions in scipy.datasets.
The only area that I can think of where there might not be some readily
available online source for datasets is signal.

Josef


>
>
>
> On Fri, Mar 30, 2018 at 1:16 AM, Stefan van der Walt <stefanv at berkeley.edu>
> wrote:
>>
>> On Thu, 29 Mar 2018 18:54:52 -0400, Warren Weckesser wrote:
>> > Can you summarize the problems that make you regret including the
>> > data?
>>
>> - The size of the repository (extra time on each clone, and that for
>>   data that isn't necessary in most use cases)
>>
>> - Artificial limit on data sizes: we now have a default place to store
>>   data, but we still need an additional mechanism for larger datasets.
>>   How do you choose the threshold for what goes in, what is too big?
>>
>> - Because these tiny embedded datasets are easily available, they become
>>   the default for demos.  If data is stored externally, realistic
>>   examples become more feasible and likely.
>>
>> Best regards
>> Stéfan
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>