Mailman 3 October 2012 - SciPy-User

Re: [SciPy-User] Complex error function
by The Helmbolds Oct. 13, 2012

Oct. 13, 2012

To all who contributed to the recent discussions, Thanks for setting me straight! This happens often enough I need the acronymn: URRIAWA !! (U R Right ! I Am Wrong -- Again !!) Bob H

1 0

"small data" statistics
by josef.pktd＠gmail.com Oct. 13, 2012

Oct. 13, 2012

Most statistical tests and statistical inference in scipy.stats and statsmodels relies on large number assumptions. Everyone is talking about "Big data", but is anyone still interested in doing small sample statistics in python. I'd like to know whether it's worth spending any time on general purpose small sample statistics. for example: http://facultyweb.berry.edu/vbissonnette/statshw/doc/perm_2bs.html ``` Example homework problem: Twenty participants were given a list of 20 words to process. The 20 participants were randomly assigned to one of two treatment conditions. Half were instructed to count the number of vowels in each word (shallow processing). Half were instructed to judge whether the object described by each word would be useful if one were stranded on a desert island (deep processing). After a brief distractor task, all subjects were given a surprise free recall task. The number of words correctly recalled was recorded for each subject. Here are the data: Shallow Processing: 13 12 11 9 11 13 14 14 14 15 Deep Processing: 12 15 14 14 13 12 15 14 16 17 ``` Josef

7 13

Re: [SciPy-User] SciPy erf function
by The Helmbolds Oct. 13, 2012

Oct. 13, 2012

> On 12/10/12 12:48, The Helmbolds wrote: >>> On 09/10/12 19:12, Pauli Virtanen wrote: >>>> 09.10.2012 19:28, "Claas H. K?hler" kirjoitti: >>>>> I have a question regarding the error function > scipy.special.erf: >>>>> >>>>> Is it intended, that the erf of an imaginary argument yields > a >>> non-vanishing real-part? >>>>> >>>>> I get e.g. >>>>> erf(1j)= 1.6504257587975431j >>>>> erf(5j)= (1+8298273879.8992386j) <snip, snip> >> On my system, I get the correct answers if I'm careful about the call > to erf. >> If I call erf with a single real value, I get the ordinary (not the > complex) error function value. >> If I call erf with a NumPy array or a Python sequence, I get the complex er > ror function returned. >> I do not think SciPy's erf is supposed to be called with a complex > number. <Claas responds> > According to the docs it is. Otherwise I would expect to see a domain error, > similar to erfc. > > Regards > Claas <snip, snip> OK. Maybe we can agree on the following: 1. The documentation is wrong. 2. Supplying bogus argument should trigger a warning and a refusal to use it. IMHO, these are both bugs. And AFAIK they have nothing to do with the formulas in either the "Handbook of Mathematical Functions" or its descendant, the "Digital Library of Mathematical Functions".

1 0

Re: [SciPy-User] SciPy-User Digest, Vol 110, Issue 21
by The Helmbolds Oct. 12, 2012

Oct. 12, 2012

> On 09/10/12 19:12, Pauli Virtanen wrote: >> 09.10.2012 19:28, "Claas H. K?hler" kirjoitti: >>> I have a question regarding the error function scipy.special.erf: >>> >>> Is it intended, that the erf of an imaginary argument yields a > non-vanishing real-part? >>> >>> I get e.g. >>> erf(1j)= 1.6504257587975431j >>> erf(5j)= (1+8298273879.8992386j) >>> >>> The first result is what I would expect in accordance with Wolfram > alpha. The second result, however, >>> has a real part of unity. As far as I know, the real part of erf should > always vanish for purely >>> imaginary numbers. >>> >>> Any support would be appreciated. >> >> The reason here is that the ye olde complex erf Fortran implementation >> that Scipy has uses the asymptotic expansion (Abramowitz & Stegun >> 7.1.23) to compute large-argument values. The asymptotic series is for >> erfc, and one always gets Re erf = 1 along the imaginary axis. >> >> Of course, this is somewhat naive. While it does produce reasonable >> relative accuracy as a complex number, the accuracy of the real and >> imaginary parts separately is not necessarily OK near the imaginary axis. >> >> The issue with Scipy here is twofold -- first, there are no better >> existing special function libraries we could use, or at least I'm not >> aware of them. Second, writing these from scratch takes time and >> expertise and nobody has so far volunteered to do any work in this >> direction. >> > Thanks for the quick response! > > The bottom line is that erf is actually not (correctly) implemented for complex > arguments, if I > understand you correctly. > > I suspect there are good reasons to provide a function which is known to yield > incorrect results, so > that throwing a type error is not an option? (This is what erfc does on my > machine) > > However, adding a warning when called with complex arguments could be helpful to > prevent naiive use > as in my case. Adding this important piece of information to the docs would not > harm either, from my > point of view. > > In any case, thanks for the quick support. > > Regards > Claas On my system, I get the correct answers if I'm careful about the call to erf. If I call erf with a single real value, I get the ordinary (not the complex) error function value. If I call erf with a NumPy array or a Python sequence, I get the complex error function returned. I do not think SciPy's erf is supposed to be called with a complex number. For example: >>> special.erf(1j) 1.6504257587975431j # Wrong answer! >>> special.erf((0,1)) array([ 0. , 0.84270079]) # Right answer. Two more examples: >>> for y in range(-10, 11): temp = special.erf((0,y)) print y, temp # Calling with a sequence, returns a NumPy array -10 [ 0. -1.] -9 [ 0. -1.] -8 [ 0. -1.] -7 [ 0. -1.] -6 [ 0. -1.] -5 [ 0. -1.] -4 [ 0. -0.99999998] -3 [ 0. -0.99997791] -2 [ 0. -0.99532227] -1 [ 0. -0.84270079] 0 [ 0. 0.] 1 [ 0. 0.84270079] 2 [ 0. 0.99532227] 3 [ 0. 0.99997791] 4 [ 0. 0.99999998] 5 [ 0. 1.] 6 [ 0. 1.] 7 [ 0. 1.] 8 [ 0. 1.] 9 [ 0. 1.] 10 [ 0. 1.] OTOH-------------------------------------------------------------------------------------------- >>> for y in range(-10, 11): temp = special.erf(y) print y, temp # Calling with a (scalar) real value returns a (scalar) real value. -10 -1.0 -9 -1.0 -8 -1.0 -7 -1.0 -6 -1.0 -5 -0.999999999998 -4 -0.999999984583 -3 -0.999977909503 -2 -0.995322265019 -1 -0.84270079295 0 0.0 1 0.84270079295 2 0.995322265019 3 0.999977909503 4 0.999999984583 5 0.999999999998 6 1.0 7 1.0 8 1.0 9 1.0 10 1.0 Bob and Paula H

3 2

[SciPy-user] Problems Importing
by Overtim3 Oct. 12, 2012

Oct. 12, 2012

I have already installed scipy and cant seem to get anything to import from there. Im running windows 7 64 bit. Any ideas? -- View this message in context: http://old.nabble.com/Problems-Importing-tp34543713p34543713.html Sent from the Scipy-User mailing list archive at Nabble.com.

2 1

[SciPy-user] Help with (Cumulative density function) CDF
by Overtim3 Oct. 12, 2012

Oct. 12, 2012

So I'm having a hard time using this function cdf(x, a, b, loc=0, scale=1) Cumulative density function. I don't fully understand where the a, b come into play I'm trying to plot this and make mine look similar to this http://en.wikipedia.org/w/index.php?title=File:Binomial_distribution_cdf.sv… but cant seem to quite get it. Any help would be appreciated. -- View this message in context: http://old.nabble.com/Help-with-%28Cumulative-density-function%29-CDF-tp345… Sent from the Scipy-User mailing list archive at Nabble.com.

2 1

Reminder: Last day of early registration for PyData NYC conference!
by Peter Wang Oct. 12, 2012

Oct. 12, 2012

Hi everyone, Just a friendly reminder that today is the final day of early registration for the PyData NYC conference later this month! We have a fantastic lineup of talks and workshops on a variety of topics related to Python for data analysis, including topics that are hard to find at other conferences (e.g. practical perspectives on Python and Hadoop, using Python and R, etc.). http://nyc2012.pydata.org/ Use the discount code "numpy" for a 20% discount off of registration! We are also looking for sponsors. We are proud to feature D. E. Shaw, JP Morgan, and Appnexus as gold sponsors. If your company or organization would like some visibility in front of a few hundred Python data hackers, please visit our sponsor information page: http://nyc2012.pydata.org/sponsors/becoming/ Thanks, Peter

1 0

ANN: WinPython v2.7.3.0
by Pierre Raybaut Oct. 11, 2012

Oct. 11, 2012

Hi all, I'm pleased to introduce my new contribution to the Python community: WinPython. WinPython v2.7.3.0 has been released and is available for 32-bit and 64-bit Windows platforms: http://code.google.com/p/winpython/ WinPython is a free open-source portable distribution of Python for Windows, designed for scientists. It is a full-featured (see http://code.google.com/p/winpython/wiki/PackageIndex) Python-based scientific environment: * Designed for scientists (thanks to the integrated libraries NumPy, SciPy, Matplotlib, guiqwt, etc.: * Regular *scientific users*: interactive data processing and visualization using Python with Spyder * *Advanced scientific users and software developers*: Python applications development with Spyder, version control with Mercurial and other development tools (like gettext) * *Portable*: preconfigured, it should run out of the box on any machine under Windows (without any installation requirements) and the folder containing WinPython can be moved to any location (local, network or removable drive) * *Flexible*: one can install (or should I write "use" as it's portable) as many WinPython versions as necessary (like isolated and self-consistent environments), even if those versions are running different versions of Python (2.7, 3.x in the near future) or different architectures (32bit or 64bit) on the same machine * *Customizable*: using the integrated package manager (wppm, as WinPython Package Manager), it's possible to install, uninstall or upgrade Python packages (see http://code.google.com/p/winpython/wiki/WPPM for more details on supported package formats). *WinPython is not an attempt to replace Python(x,y)*, this is just something different (see http://code.google.com/p/winpython/wiki/Roadmap): more flexible, easier to maintain, movable and less invasive for the OS, but certainly less user-friendly, with less packages/contents and without any integration to Windows explorer [*]. [*] Actually there is an optional integration into Windows explorer, providing the same features as the official Python installer regarding file associations and context menu entry (this option may be activated through the WinPython Control Panel). Enjoy! -Pierre

3 2

Thresholding in sparse matrices
by Jaidev Deshpande Oct. 11, 2012

Oct. 11, 2012

Hi, When constructing a sparse matrix, (for instance, using scipy.sparse.coo_matrix) does the function take into account any tolerance? In other words, does an element have to be exactly zero to be casted as a zero in the sparse matrix? Can a tolerance value be specified below which every element would be casted as zero? Suppose I have a matrix x: >>> x = np.array([1e-3, 1e-5, 1e-10]) >>> coo_matrix(x) <1x3 sparse matrix of type '<type 'numpy.float64'>' with 3 stored elements in COOrdinate format> Now, I want x[1] and x[2] to be zeros, because they are less than 0.001 (say). This thresholding can be done on the array x itself, but I wonder if there is a way to do this through the sparse constructors. Thanks

1 0

NumPy Binomial BTPE method Problem
by Josh Lawrence Oct. 11, 2012

Oct. 11, 2012

Hello all, I am implementing a binomial random variable in MATLAB. The default method in the statistics toolbox is extremely slow for large population/trial size. I am needing to do trials for n as large as 2**28. I found in NumPy some code that implements a binomial random draw in numpy/random/mtrand/distributions.c. I was trying to convert the code to MATLAB and the BTPE method seems to have an error in lines 337-341 of distributions.c. The if ... else if ... else statement I think is incorrect. I think it should be an if ... else ... statement followed by the contents of the original else which starts on line 337. The if ... else if ... else block is as follows: #### begin code snippet #### if (m < y) { for (i=m; i<=y; i++) { F *= (a/i - s); } } else if (m > y) { for (i=y; i<=m; i++) { F /= (a/i - s); } } else { if (v > F) goto Step10; goto Step60; } #### end code snippet #### >From what I can tell, the variable F is only used in the comparison within the else{} statment (i.e. the if(v > F) statement) and nowhere else within the scope of the function. I also found a fortran implementation here: http://wstein.org/home/wstein/www/home/mhansen/spkgs_in_progress/octave-3.2… and it appears this is from where the code was originally adapted as the variable names are the same. My parsing of fortran GOTOs is a bit rusty, but I think the contents of the else block in above snippet should be not be conditional. I don't understand the underlying algorithm very well and don't have access the the BTPE paper, so I can't comment on the validity of the fortran code. There just seems to be an error in logic in the above code. So please have someone who understands it look at it. It appears Robert Kern wrote the function a decent portion of the file at some point in the past. I hope this helps. Cheers, -- Josh Lawrence P.S. I apologize if my email is inconvenient, but I could not figure out how to tell gmail to set the reply-to field to be scipy-user(a)scipy.org.

3 12