Mailman 3 January 2013 - SciPy-Dev

Re: [SciPy-Dev] docstring standard: parameter shape description
by Joe Harrington 29 Jan '13

29 Jan '13

On Sun, Jan 27, 2013 at 2:51 PM, Ralf Gommers <ralf.gommers(a)gmail.com> wrote: > Hi, > > When merging the doc wiki edits there were a large number of changes to the > shape description of parameters/returns. This is not yet described in the > docstring standard > (https://github.com/numpy/numpy/blob/master/doc/HOWTO_DOCUMENT.rst.txt), and > currently is done in various ways: > > param1 : ndarray, shape (N,) I think it should be consistent between all cases, start with the class and then the shape, and solve the general problem. Initially, I agreed with Josef about being terse, but it reads hard that way and if you're a newbie you might wonder what the numbers in parens are. The word "shape" does not add an extra line, and the comma makes sense as an appositive in English. So, I prefer: param1 : ndarray, shape XXXXX For XXXXX, we need to specify: ranges of allowed numbers of dimensions ranges of allowed sizes within each dimension low- or high-side unconstrained sizes in either case We should accept the output of .shape, and define some range conventions. Of course, there will be pathological cases, particularly in specialist packages that adopt the numpy doc standard, where nothing but text will adequately describe the allowed dimensions ("If there are three dimensions, then the second dimension must..."). A "(see text)" should be allowed after the shape spec. So, this is my counterproposal for inclusion in the standard: ------------------------------------------------------------------------------- param1 : ndarray, shape <shapespec> [(see text)] as in param1 : ndarray, shape (2, 2+, dim(any), 4-, 4-6, any) (see text) in <shapespec>: the spec reads from the slowest-varying to the fastest-varying dimension a number means exactly that number of items on that axis a number followed by a "+" ("-") means that number or more (fewer) items a-b means between a and b items, INCLUSIVE "any" means any number of items on that axis dim(dimspec) means the conventions above apply for dimensions instead of items The example would mean an array with dimensions, from slowest to fastest-varying, of size: 2 2 or more (0 or more axes can be inserted here) 0 to 4 4 to 6 any size, including absent (use 1+ to require a dimension) ------------------------------------------------------------------------------- I thought of basing the ranges off the Python indexing spec, but I find it potentially confusing. Is there a reason to propagate the Python weirdness that the ending index is 1 more than the final item? The latter behavior of Python is useful in programming (you don't have to write "-1" all the time), but it is error-inducing to many, even to non-beginners. However, if we did this, the example would look like: (2, :3, dim(:), 4:, 4:7, :) I don't think it wise to use the indexing spec with a changed meaning for the ending index. Either we should adopt the indexing spec or we should adopt another spec that looks different enough from the indexing spec not to be confusing. Remember that the docs need to be clear to beginners and help them not make errors. Thoughts? --jh--

5 7

docstring standard: parameter shape description
by Ralf Gommers 27 Jan '13

27 Jan '13

Hi, When merging the doc wiki edits there were a large number of changes to the shape description of parameters/returns. This is not yet described in the docstring standard ( https://github.com/numpy/numpy/blob/master/doc/HOWTO_DOCUMENT.rst.txt), and currently is done in various ways: param1 : ndarray, shape (N,) param1 : ndarray of shape (N,) param1 : (N,) ndarray param1 : 1-D ndarray param1 : ndarray A 1-D array .... To keep this consistent I'd like to add it to the standard. My proposal would be: 1. If the actual shape is used in the description (for example to say "return size is N+1), then use: param1 : ndarray, shape (N,) 2. If it's not used but has to be 1-D (or 2-D, ...), then use: param1 : 1-D ndarray This post was triggered by doc wiki edits and a review comment on those at https://github.com/scipy/scipy/pull/405 by the way. Ralf

2 1

segfault on python 3 - Gohlke binaries
by josef.pktd＠gmail.com 27 Jan '13

27 Jan '13

I just tried to install a new python 3.3 on Windows 7 python-3.3.0.amd64.msi numpy-MKL-1.7.0rc1.win-amd64-py3.3.exe scipy-0.12.0.dev.win-amd64-py3.3.exe installers ran without problems (aside: I also have the 32bit version of python installed in a different directory from the 64 bit version) numpy.test() ends without errors and failures scipy.test(verbose=3) ends with a segfault at Compare dsbevx eigenvalues and eigenvectors ... <segfault> I'm just a "consumer" of binaries, and have no idea. Josef

3 5

Re: [SciPy-Dev] Expanding Scipy's KDE functionality
by Daniel Smith 25 Jan '13

25 Jan '13

Ok, let's see if I can respond to everyone's comments. >From Jake: > 2) The algorithm seems limited to one or maybe two dimensions. > scipy.stats.gaussian_kde is designed for N dimensions, so it might be > difficult to find a fit for this bandwidth selection method. One option > might be to allow this bandwidth selection method via a flag in > scipy.stats.gaussian_kde, and raise an error if the dimensionality is > too high. To do that, your code would need to be reworked fairly > extensively to fit in the gaussian_kde class. In principal, this method can be applied in N dimensions. However, I think it would be unwise to do so. The method requires that you simultaneously estimate the density and the bandwidth. Because of that, you have to implement the method on some mesh, and mesh size grows exponentially with the number of dimensions. The code certainly could be reworked to work in N dimensions, but I don't think it would be effective enough to be worth the effort. The results are also primarily used for visualization, which is useless beyond 2-d. >From Ralf: > My impression is that this can't be integrated with gaussian_kde - it's not a bandwidth estimation method but an adaptive density estimator. It's both. The bandwidth estimate falls out of the density estimate. That bandwidth estimate could be easily used to generate an estimate on a different mesh. > My suggestion would be to implement the density estimator and do a good amount of performance testing, at least show that the performance is as good as described in table 1 of the paper. I can certainly do that. I will post here when the tests are up and running. >From Barbier de Reuille Pierre: > It should be easy to separate them and use the estimation of the bandwidth without the density estimation. Unfortunately, that is not the case. The bandwidth estimate is generated from a fixed point calculation based on the norm of a derivative of the estimated density. Unless I am missing something, it would not be possible to estimate that derivative without an explicit density estimate. Fourier coordinates are used because the derivative estimate is simpler in those coordinates. > For example, as stated in the paper, the method is equivalent to a reflexion method with a gaussian kernel. But the renormalisation method gives very similar results too, without enforcing > that f'(0) = 0 (i.e. first derivative is always 0 on the boundaries). I have not currently implemented any boundary corrections, but it would not be difficult to implement the renormalization method using the bandwidth estimate from this method. It would require a second density estimate, but the estimate would be much, much better than the current code. > Also, can you generalise the bandwidth calculation to unbounded domains? or at least half-domains (i.e. [a; oo[ or ]-oo; a])? It seems that it all depends on the domain being of finite size. In fact, the method currently only works on unbounded domains. The exact domain you calculate the density on is an optional parameter to the density estimator function. The actual domain you calculate on has to be finite because a finite mesh is needed. > I have a different concern though: is it normal that the density returned by the method is not normalized? (i.e. the integral of the density is far from being one). That's a bug. I can fix that with one line of code. I have always just used the density estimate without units, because they aren't particularly informative. However, the output should be normalized, or at least a flag included to make it so. It seems like the next step is to set up a testing regime for comparison to the two existing methods to compare speed and reproduce the data from Table 1 in the paper. Also, it seems likely that statsmodels is the more appropriate setting for this project. In particular, I want to generalize the method to periodic domains, which appears to be a novel implementation so more intensive testing will likely be needed. Thanks, Daniel

4 13

Getting IO Error while running Scipy Central on local server
by Surya Kasturi 24 Jan '13

24 Jan '13

I successfully installed the scipy central site on my computer. I just tried to submit a code snipped for testing.. and ran into the below error. IOError at /item/new/ [Errno 2] No such file or directory: 'compile\\conf.py' Please look into the error: http://dpaste.com/890230/

4 6

Updating SciPy-Central Website
by Surya Kasturi 24 Jan '13

24 Jan '13

After discussing some ideas, I consolidated few ideas on SciPy Central 1. We could improve the design of the site 1.a) Using Twitter Bootstrap 1.b) OR, we can take up scipy.org design template and implement 2. We could add OpenID feature for the site. So, what do you say?

3 2

Like to participate in scipy.signals
by Surya Kasturi 24 Jan '13

24 Jan '13

Hi David, I am Surya, Electronics Engineering Student with CS background. I see that you are one of the maintainers<https://github.com/scipy/scipy/blob/master/doc/MAINTAINERS.rst.txt> of Scipy.Signals for which I am thinking to contribute some code (in Python)! As I am a newcomer to Opensource development and writing production level code, I am wondering if you could help me a bit in starting things out initially. Currently I am not really having any new ideas to pick up and deliver modules.. So, I am very open to pick up any assigned tasks or start with small stuff! Also, I have gone through some bugs in Signals module (#980 #928). However, as I am fairly new to the whole system, it would be great if you can provide some tips/ pointers etc for starting off. (Please let me know if there is any other interesting stuff I can do) Regarding my skills and interests: I know Python, Django, C. Interested in Signal Processing - Image Processing , Web dev. -- I was asked by couple of guys to check up respective leads for help Regards Surya

3 6

weave for Python3.3 ?
by Helmut Jarausch 24 Jan '13

24 Jan '13

Hi, has anybody started to update weave for Python3.3? It's not a simple 2to3 call. Helmut.

4 6

Re: [SciPy-Dev] Electronics student with programming background - Like to participate in SciPy - Write some code and learn!
by Daniel Smith 23 Jan '13

23 Jan '13

Hello, I am also looking for ways to contribute to Scipy. I have experience with Python, C/C++ and limited experience with the Numpy C API. In particular, I have some code implementing the kernel density estimator bandwidth selection algorithm from the following paper: Z. I. Botev, J. F. Grotowski, and D. P. Kroese. Kernel density estimation via diffusion. The Annals of Statistics, 38(5):2916–2957, 2010. That method is more resilient to multi-modal data than the standard plug-in estimators. I would love to add that method to the current SciPy stats package if there is interest. Thanks, Daniel > Hi, > > I am Surya, studying Junior Year - Electronics & Communication Engineering > with Computer Science/ Programming background. I have looked into SciPy and > its really amazing! > > In this regard, I would like to explore the possibility of contributing to > this project by writing code and simultaneously learn the real engineering > stuff. My skills lie in Python, Django, C - and little Facebook API, Cloud > platforms (Openshift), Git. > > Also, I wrote some fun-stuff projects during week ends which you might like > to take a look. > > 1. Https://apps.facebook.com/pingmee -- Lets people ping their friends > using cartoons (Python, Django -- PIL) > 2. Https://apps.facebook.com/suryaphotography -- social reader framework > for my photography blog; Not yet finished (Python, Django -- Google Feed > API) - Got to finish if time permits > 3. Https://github.com/ksurya -- Github handle > > So, I am ready to take up any work and get along with it that involves > Python! > > Regarding my scientific skills, I studied Engineering Mathematics, Digital > Signal Processing (now studying), Signals & Systems etc. [ More on signals ] > > > Thanks for reading! waiting for your reply > > -- Surya

2 3

Electronics student with programming background - Like to participate in SciPy - Write some code and learn!
by Surya Kasturi 23 Jan '13

23 Jan '13

Hi, I am Surya, studying Junior Year - Electronics & Communication Engineering with Computer Science/ Programming background. I have looked into SciPy and its really amazing! In this regard, I would like to explore the possibility of contributing to this project by writing code and simultaneously learn the real engineering stuff. My skills lie in Python, Django, C - and little Facebook API, Cloud platforms (Openshift), Git. Also, I wrote some fun-stuff projects during week ends which you might like to take a look. 1. Https://apps.facebook.com/pingmee -- Lets people ping their friends using cartoons (Python, Django -- PIL) 2. Https://apps.facebook.com/suryaphotography -- social reader framework for my photography blog; Not yet finished (Python, Django -- Google Feed API) - Got to finish if time permits 3. Https://github.com/ksurya -- Github handle So, I am ready to take up any work and get along with it that involves Python! Regarding my scientific skills, I studied Engineering Mathematics, Digital Signal Processing (now studying), Signals & Systems etc. [ More on signals ] Thanks for reading! waiting for your reply -- Surya

4 4