Mailman 3 March 2009 - NumPy-Discussion

Overlapping ranges
by Peter Saffrey March 16, 2009

March 16, 2009

I'm trying to file a set of data points, defined by genome coordinates, into bins, also based on genome coordinates. Each data point is (chromosome, start, end, point) and each bin is (chromosome, start, end). I have about 140 million points to file into around 100,000 bins. Both are (roughly) evenly distributed over the 24 chromosomes (1-22, X and Y). Genome coordinates are integers and my data points are floats. For each data point, (end - start) is roughly 1000, but the bins are are of uneven widths. Bins might have also overlap - in that case, I need to know all the bins that a point overlaps. By overlap, I mean the start or end of the data point (or both) is inside the bin or that the point entirely covers the bin. At the moment, I'm using a fairly naive approach that finds roughly in the genome (which gene) each point might be and then checking it against the bins in that gene. If I split the problem into chromosomes, I feel sure there must be some super-fast matrix approach I can apply using numpy, but I'm struggling a bit. Can anybody suggest something? Peter

3 2

1.3.x branch created - trunk now opened for 1.4
by David Cournapeau March 16, 2009

March 16, 2009

Hi, I have just started the 1.3.x branch - as such, any change done to the trunk will not end up in the 1.3 release. I will announce the 1.3 beta release within the day, hopefully, cheers, David

1 0

svn and tickets email status
by Ryan May March 16, 2009

March 16, 2009

Hi, What's the status on SVN and ticket email notifications? The only messages I'm seeing since the switch is the occasional spam. Should I try re-subscribing? Ryan -- Ryan May Graduate Research Assistant School of Meteorology University of Oklahoma Sent from: Norman Oklahoma United States.

2 1

Superfluous array transpose (cf. ticket #1054)
by Sturla Molden March 16, 2009

March 16, 2009

Regarding ticket #1054. What is the reason for this strange behaviour? >>> a = np.zeros((10,10),order='F') >>> a.flags C_CONTIGUOUS : False F_CONTIGUOUS : True OWNDATA : True WRITEABLE : True ALIGNED : True UPDATEIFCOPY : False >>> (a+1).flags C_CONTIGUOUS : True F_CONTIGUOUS : False OWNDATA : True WRITEABLE : True ALIGNED : True UPDATEIFCOPY : False Sturla Molden

3 4

inplace dot products
by Olivier Grisel March 16, 2009

March 16, 2009

Hi numpist people, I discovered the ufunc and there ability to compute the results on preallocated arrays: >>> a = arange(10, dtype=float32) >>> b = arange(10, dtype=float32) + 1 >>> c = add(a, b, a) >>> c is a True >>> a array([ 1., 3., 5., 7., 9., 11., 13., 15., 17., 19.], dtype=float32) My questions is : is there a way to have an equivalent for the dot product operation: I want atlas to build my dot products without allocating a temporary array and reuse a preallocated array of results. Suppose I have: >>> results = array((10, 3), dtype=float32) >>> W = arange(6, dtype=float32).reshape((2, 3)) >>> x = arange(20, dtype=float32).reshape((10, 2)) What I want is the equivalent of the following without the intermediate call to malloc: >>> results[:] = dot(x, W) Any idea? I tried to introspect the various docstring of numpy core modules but I could not get any lead. -- Olivier

4 9

Has IPython been useful to you? Please let me know...
by Fernando Perez March 16, 2009

March 16, 2009

Hi all, [ apologies for the semi-spam, I'll keep this brief and expect all replies off-list ] IPython is a project that many of you on this list are likely to use in your daily work, either directly or indirectly (if you've embedded it or used it as a component of some other system). I would simply like to ask you, if IPython has been significantly useful for a project you use, lead, develop, etc., to let me know. For legal/professional reasons, I need to gather information about who has found IPython to be of value. I started IPython as a toy 'afternoon hack' in late 2001, and today it continues to grow, as the nicely summarized Ohloh stats show: https://www.ohloh.net/p/ipython (obviously, this is now the result of the work of many, not just myself, as is true of any healthy open source project as it grows). But I have never systematically tracked its impact, and now I need to do so. So, if you have used IPython and it has made a significant contribution to your project, work, research, company, whatever, I'd be very grateful if you let me know. A short paragraph on what this benefit has been is all I ask. Once I gather any information I get, I would contact directly some of the responders to ask for your authorization before quoting you. I should stress that any information you give me will only go in a documentation packet in support of my legal/residency process here in the USA (think of it as an oversized, obnoxiously detailed CV that goes beyond just publications and regular academic information). To keep traffic off this list, please send your replies directly to me, either at this address or my regular work one: Fernando.Perez(a)berkeley.edu In advance, many thanks to anyone willing to reply. I've never asked for anything in return for working on IPython and the ecosystem of scientific Python tools, but this is actually very important, so any information you can provide me will be very useful. Best regards, Fernando Perez.

1 0

Py_ssize_t
by Charles R Harris March 15, 2009

March 15, 2009

Some of the calls to the python c-api have been changed to use Py_ssize_t. As Py_ssize_t was not available in Python 2.4 I wonder if we check if it is defined and set it to int if not. Also, are we running tests on Python 2.4 for the release? Chuck

2 1

Build Failure on IA64
by Patrick Marsh March 14, 2009

March 14, 2009

Hi, I'm trying to build numpy from SVN and ran across this error: numpy/core/include/numpy/npy_cpu.h:44:10: error: #error Unknown CPU, please report this to numpy maintainers with information about your platform (OS, CPU and compiler) This is on a linux machine using gcc. Here is the processor information: processor : 0 vendor : GenuineIntel arch : IA-64 family : Itanium 2 model : 2 revision : 1 archrev : 0 features : branchlong cpu number : 0 cpu regs : 4 cpu MHz : 1500.000000 itc MHz : 1500.000000 BogoMIPS : 2244.60 siblings : 1 Patrick -- Patrick Marsh Graduate Research Assistant School of Meteorology University of Oklahoma http://www.patricktmarsh.com

2 3

code performanceon windows (32 and/or 64 bit) using SWIG: C++ compiler MS vs.cygwin
by Sebastian Haase March 13, 2009

March 13, 2009

Hi, I was wondering if people could comment on which compiler produces faster code, MS-VS2003 or cygwin g++ ? I use Python 2.5 and SWIG. I have C/C++ routines for large (maybe 10MB, 100MB or even >1GB (on XP 64bit)) data processing. I'm not talking about BLAS or anything like that .... just for-loops mostly on contiguous memory. Or should the speed / memory performance of the resulting code be the same ? Thanks, Sebastian Haase

7 17

random number generator--problems?
by JJ March 13, 2009

March 13, 2009

Hello: I just ran across this article saying that the random number generator in Linux is broken. http://www-cdf.fnal.gov/publications/cdf6850_badrand.pdf The article starts: <<The main purpose of this note is to exhibit a defect in the Linux C and C++ standard library pseudo-random number generator rand and the Linux/UNIX system-library generator random. Since Linux rand and Linux/UNIX random use identical algorithms, we will use “random” to refer to both. The defect is uncovered when random fails a simple empirical test.>> As I understand it, Numpy uses a different random number generator (Mersenne Twister), but I just wanted to verify that any problems with the Linux random number generators do not apply to Numpy. Can someone please verify this? Thanks John

2 1