Re: [Numpy-discussion] Adding weights to cov and corrcoef
Date: Thu, 06 Mar 2014 13:40:40 +0100
From: Sebastian Berg <sebastian@sipsolutions.net> Subject: Re: [Numpy-discussion] Adding weights to cov and corrcoef (Sebastian Berg) To: numpy-discussion@scipy.org Message-ID: <1394109640.9122.13.camel@sebastian-t440> Content-Type: text/plain; charset="UTF-8"
On Mi, 2014-03-05 at 10:21 -0800, David Goldsmith wrote:
+1 for it being "too baroque" for NumPy--should go in SciPy (if it isn't already there): IMHO, NumPy should be kept as "lean and mean" as possible, embellishments are what SciPy is for. (Again, IMO.)
Well, on the other hand, scipy does not actually have a `std` function of its own, I think.
Oh, well, in that case forget I said anything. (Though I think it's "interesting" that no one else has chimed in: if you're the only one that needs it (at this time), perhaps it would be best to "roll your own" and then offer to "pass it around." :-)) DG
So if it is quite useful I think this may be an option (I don't think I ever used weights with std, so I can't argue strongly for inclusion myself). Unless adding new functions to `scipy.stats` (or just statsmodels) which implement different types of weights is the longer term plan, then things might bite...
DG _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
------------------------------
Message: 5 Date: Thu, 6 Mar 2014 13:45:36 +0000 From: Nathaniel Smith <njs@pobox.com> Subject: Re: [Numpy-discussion] numpy gsoc ideas (was: numpy gsoc topic idea: configurable algorithm precision and vector math library integration) To: Discussion of Numerical Python <numpy-discussion@scipy.org> Message-ID: <CAPJVwBm=no71WvC9Zjh7DXNaGn0jpAGmvrOyoVvmHK-MW= R3Fw@mail.gmail.com> Content-Type: text/plain; charset=UTF-8
On Thu, Mar 6, 2014 at 5:17 AM, Sturla Molden <sturla.molden@gmail.com> wrote:
Nathaniel Smith <njs@pobox.com> wrote:
3. Using Cython in the numpy core
The numpy core contains tons of complicated C code implementing elaborate operations like indexing, casting, ufunc dispatch, etc. It would be really nice if we could use Cython to write some of these things.
So the idea of having a NumPy as a pure C library in the core is abandoned?
This question doesn't make sense to me so I think I must be missing some context.
Nothing is abandoned: This is one email by one person on one mailing list suggesting a project to the explore the feasibility of something. And anyway, Cython is just a C code generator, similar in principle to (though vastly more sophisticated than) the ones we already use. It's not like we've ever promised our users we'll keep stable which kind of code generators we use internally.
However, there is a practical problem: Cython assumes that each .pyx file generates a single compiled module with its own Cython-defined API. Numpy, however, contains a large number of .c files which are all compiled together into a single module, with its own home-brewed system for defining the public API. And we can't rewrite the whole thing. So for this to be viable, we would need some way to compile a bunch of .c *and .pyx* files together into a single module, and allow the .c and .pyx files to call each other.
Cython takes care of that already.
http://docs.cython.org/src/userguide/sharing_declarations.html#cimport
http://docs.cython.org/src/userguide/external_C_code.html#using-cython-decla...
Linking multiple .c and .pyx files together into a single .so/.dll is much more complicated than just using 'cimport'. Try it if you don't believe me :-).
-n
-- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org
------------------------------
Message: 6 Date: Thu, 6 Mar 2014 13:59:30 +0000 From: Nathaniel Smith <njs@pobox.com> Subject: Re: [Numpy-discussion] numpy gsoc ideas (was: numpy gsoc topic idea: configurable algorithm precision and vector math library integration) To: Discussion of Numerical Python <numpy-discussion@scipy.org> Message-ID: <CAPJVwB=PmquKm5j4-oquCkLkcK8G1pipB8XLbq5= 26izjbJPjg@mail.gmail.com> Content-Type: text/plain; charset=UTF-8
On Thu, Mar 6, 2014 at 9:11 AM, David Cournapeau <cournape@gmail.com> wrote:
On Wed, Mar 5, 2014 at 9:11 PM, Nathaniel Smith <njs@pobox.com> wrote:
So this project would have the following goals, depending on how practical this turns out to be: (1) produce a hacky proof-of-concept system for doing the above, (2) turn the hacky proof-of-concept into something actually viable for use in real life (possibly this would require getting changes upstream into Cython, etc.), (3) use this system to actually port some interesting numpy code into cython.
Having to synchronise two projects may be hard for a GSoC, no ?
Yeah, if someone is interested in this it would be nice to get someone from Cython involved too. But that's why the primary goal is to produce a proof-of-concept -- even if all that comes out is that we learn that this cannot be done in an acceptable manner, then that's still a succesful (albeit disappointing) result.
Otherwise, I am a bit worried about cython being used on the current C code as is, because core and python C API are so interwined (especially multiarray).
I don't understand this objection. The whole advantage of Cython is that it makes it much, much easier to write code that involves intertwining complex algorithms and heavy use of the Python C API :-). There's tons of bug-prone spaghetti in numpy for doing boring things like refcounting, exception passing, and argument parsing.
-n
-- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org
------------------------------
Message: 7 Date: Thu, 6 Mar 2014 10:35:15 -0700 From: Charles R Harris <charlesr.harris@gmail.com> Subject: Re: [Numpy-discussion] 1.8.1rc1 on sourceforge. To: Discussion of Numerical Python <numpy-discussion@scipy.org> Message-ID: < CAB6mnx+btuF3vKxvebfBZKybggp+C6mtnb7zD1ck1bqk9VXV2w@mail.gmail.com> Content-Type: text/plain; charset="iso-8859-1"
On Wed, Mar 5, 2014 at 7:28 PM, Matthew Brett <matthew.brett@gmail.com
wrote:
Hi,
On Wed, Mar 5, 2014 at 3:29 PM, Matthew Brett <matthew.brett@gmail.com> wrote:
Hi,
I built (and tested) some numpy wheels for the rc1:
Now building, installing, testing, uploading wheels nightly on OSX 10.9:
http://nipy.bic.berkeley.edu/builders/numpy-bdist-whl-osx-2.7 http://nipy.bic.berkeley.edu/builders/numpy-bdist-whl-osx-3.3
and downloading, testing built wheels on OSX 10.6:
http://nipy.bic.berkeley.edu/builders/numpy-bdist-whl-osx-2.7-downloaded http://nipy.bic.berkeley.edu/builders/numpy-bdist-whl-osx-3.3-downloaded
Chuck - are you release manager for this cycle? Would you mind sending me your public ssh key so I can give you access to the buildbots for custom builds and so on?
Cheers,
Julian has done most of the work for 1.8.1. I did the 1.8.0 release because it needed doing, but building releases isn't my strong point and Ralf actually did the builds for that. So I'll happily send you my ssh, but either Ralph or Julian might be a better bet for getting the work done :)
Chuck
participants (1)
-
David Goldsmith