Mailman 3 March 2011 - NumPy-Discussion

bug with numpy 2 ** N
by Dmitrey March 23, 2011

March 23, 2011

>>> 2**64 18446744073709551616L >>> 2**array(64) -9223372036854775808 >>> 2**100 1267650600228229401496703205376L >>> 2**array(100) -9223372036854775808

2 1

Is numpy/scipy linux apt or PYPI installation linked with ACML?
by Dmitrey March 23, 2011

March 23, 2011

Hi all, I have AMD processor and I would like to get to know what's the easiest way to install numpy/scipy linked with ACML. Is it possible to link linux apt or PYPI installation linked with ACML? Answer for the same question about MKL also would be useful, however, AFAIK it has commercial license and thus can't be handled in the ways. Thank you in advance, D.

5 5

Numpy: controlling ATLAS multithreading
by Mark Dixon March 23, 2011

March 23, 2011

Hi, I've built Numpy 1.5.1 against a copy of ATLAS 3.8.3 and LAPACK 3.3.0, it's passing all tests and is looking good. Great. I note that Numpy preferentially links against the multi-threaded ATLAS libraries; however, we sometimes want to constrain a process to a single thread, and sometimes not. I see that ATLAS doesn't have the capability of setting the maximum number of threads used with, say, an environment variable. So, here comes the long-shot question: Does Numpy have the capability to choose between the single-threaded and multi-threaded ATLAS libraries at runtime? Or do I need to make this decision at build time? Thanks, Mark -- ----------------------------------------------------------------- Mark Dixon Email : m.c.dixon(a)leeds.ac.uk HPC/Grid Systems Support Tel (int): 35429 Information Systems Services Tel (ext): +44(0)113 343 5429 University of Leeds, LS2 9JT, UK -----------------------------------------------------------------

2 2

SciPy 2011 Call for Papers
by Amenity Applewhite March 23, 2011

March 23, 2011

Hello, SciPy 2011 <http://conference.scipy.org/scipy2011/index.php>, the 10th Python in Science conference, will be held July 11 - 16, 2011, in Austin, TX. At this conference, novel applications and breakthroughs made in the pursuit of science using Python are presented. Attended by leading figures from both academia and industry, it is an excellent opportunity to experience the cutting edge of scientific software development. The conference is preceded by two days of tutorials, during which community experts provide training on several scientific Python packages. *We'd like to invite you to consider presenting at SciPy 2011.* The list of topics that are appropriate for the conference includes (but is not limited to): * new Python libraries for science and engineering; * applications of Python to the solution of scientific or computational problems; * high performance, parallel and GPU computing with Python; * use of Python in science education. *Specialized Tracks* This year we also have two specialized tracks. They will be run concurrent to the main conference. *Python in Data Science Chair: Peter Wang, Streamitive, Inc.* This track focuses on the advantages and challenges of applying Python in the emerging field of "data science". This includes a breadth of technologies, from wrangling realtime data streams from the social web, to machine learning and semantic analysis, to workflow and repository management for large datasets. *Python and Core Technologies Chair: Anthony Scopatz, Enthought, Inc.* In an effort to broaden the scope of SciPy and to engage the larger community of software developers, we are pleased to introduce the _Python & Core Technologies_ track. Talks will cover subjects that are not directly related to science and engineering, yet nonetheless affect scientific computing. Proposals on the Python language, visualization toolkits, web frameworks, education, and other topics are appropriate for this session. *Talk/Paper Submission* We invite you to take part by submitting a talk abstract on the conference website at: http://conference.scipy.org/scipy2011/papers.php Papers are included in the peer-reviewed conference proceedings, to be published online. *Important dates for authors:* Friday, April 15: Tutorial proposals due (remember: stipends will be provided for Tutorial instructors) http://conference.scipy.org/scipy2011/tutorials.php Sunday, April 24: Paper abstracts due Sunday, May 8: Student sponsorship request due http://conference.scipy.org/scipy2011/student.php Tuesday, May 10: Accepted talks announced Monday, May 16: Student sponsorships announced Monday, May 23: Early Registration ends Sunday, June 20: Papers due Monday-Tuesday, July 11 - 12: Tutorials Wednesday-Thursday, July 13 - July 14: Conference Friday-Saturday, July 15 - July 16: Sprints The SciPy 2011 Team @SciPy2011 http://twitter.com/SciPy2011 _________________________ Amenity Applewhite Enthought, Inc. <http://www.enthought.com> Scientific Computing Solutions

1 0

Appending a numpy array to binary file
by Alessandro March 23, 2011

March 23, 2011

Hi everybody, I'm trying to append, inside a loop, into a binary file some arrays using numpy.save(file, arr) but wvhen I open it there is only the last array I saved. If I use savetxt it works perfectly. Do you know how can I append data to a binary file inside a loop? Example: x = arange(10) fp = open("TempFile","ab") for i in range(3): save(fp,x) fp.close() Data saved into the file is: [0,1,2,3,4,5,6,7,8,9] but I would like: [0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9 0,1,2,3,4,5,6,7,8,9] Thank you. -- ------------------------------------------------------------------------------ Alessandro Sanginario Politecnico di Torino Department of Electronics C.so Duca degli Abruzzi, 24 10129 Torino - Italy Tel. +39 0115644151 Fax. +39 0115644099 Skype: sangiox ------------------------------------------------------------------------------

3 2

Fortran was dead ... [was Re: rewriting NumPy code in C or C++ or similar]
by Sebastian Haase March 23, 2011

March 23, 2011

On Mon, Mar 14, 2011 at 9:24 PM, Ondrej Certik <ondrej(a)certik.cz> wrote: > Hi Sturla, > > On Tue, Mar 8, 2011 at 6:25 AM, Sturla Molden <sturla(a)molden.no> wrote: >> Den 08.03.2011 05:05, skrev Dan Halbert: >>> Thanks, that's a good suggestion. I have not written Fortran since 1971, >>> but it's come a long way. I was a little worried about the row-major vs >>> column-major issue, but perhaps that can be handled just by remembering >>> to reverse the subscript order between C and Fortran. >> >> In practice this is not a problem. Most numerical libraries for C assume >> Fortran-ordering, even OpenGL assumes Fortran-ordering. People program >> MEX files for Matlab in C all the time. Fortran-ordering is assumed in >> MEX files too. >> >> In ANSI C, array bounds must be known at compile time, so a Fortran >> routine with the interface >> >> subroutine foobar( lda, A ) >> integer lda >> double precision A(lda,*) >> end subroutine >> >> will usually be written like >> >> void foobar( int lda, double A[]); >> >> in C, ignoring different calling convention for lda. >> >> Now we would index A(row,col) in Fortran and A[row + col*lda] in C. Is >> that too difficult to remember? >> >> In ANSI C the issue actually only arises with small "array of arrays" >> having static shape, or convoluted contructs like "pointer to an array >> of pointers to arrays". Just avoid those and stay with 1D arrays in C -- >> do the 1D to 2D mapping in you mind. >> >> In C99 arrays are allowed to have dynamic size, which mean we can do >> >> void foobar( int lda, double *pA ) >> { >> typedef double row_t [lda]; >> vector_t *A = (vector_t*)((void*)&pA[0]); >> >> Here we must index A[k][i] to match A(i,k) in Fortran. I still have not >> seen anyone use C99 like this, so I think it is merely theoretical. >> >> Chances are if you know how to do this with C99, you also know how to >> get the subscripts right. If you are afraid to forget "to reverse the >> subscript order between C and Fortran", it just tells me you don't >> really know what you are doing when using C, and should probably use >> something else. >> >> Why not Cython? It has "native support" for NumPy arrays. >> >> Personally I prefer Fortran 95, but that is merely a matter of taste. > > +1 to all that you wrote about Fortran. I am pretty much switching to > it from C/C++ for all my numerical work, and then I use Cython to call > it from Python, as well as cmake for the build system. > > Ondrej Hi, this is quite amazing... Sturla has been writing so much about Fortran recently, and Ondrej now says he has done the move from C/C++ to Fortran -- I thought Fortran was dead ... !? ;-) What am I missing here ? Apparently (from what I was able to read-up so far) there is a BIG difference between FORTRAN 77 and F95. But isn't gcc or gfortran still only supporting F77 ? How about IntelCC's Fortran ? Is that superior? Do you guys have any info / blogs / docs where one could get an up-to-date picture? Like: 1. How about debugging - does gdb work or is there somthing better ? 2. How is the move of the F77 community to F95 in general ? How many people / projects are switching. 3. Or is the move rather going like Fortran 77 -> C -> Python -> Fortran 95 !? ;-) Thanks, Sebastian Haase

12 27

git tag question/warning
by Ralf Gommers March 23, 2011

March 23, 2011

Hi, After tagging 1.6.0b1 and pushing it I found some issues that only showed up with released=True in setup.py. According to the docs at http://www.kernel.org/pub/software/scm/git/docs/git-tag.html there is a sane (just make a new tag) and an insane (delete tag, reuse same tag) way to deal with this. Since the tagged version will actually not compile due to a version string issue in setup.py I want to go the insane way. Apologies. Please correct me if there's a better option. If you pulled from the github repo in the last few hours, please check if you have a new local tag and delete it. Changes to go in https://github.com/rgommers/numpy/tree/fixes-1.6.0b1 Ralf

3 3

fast robus implementation of float()
by Christian K. March 23, 2011

March 23, 2011

Hi, I wonder if someone has a good solution for a fast conversion of gridded ascii data to ndarray. It should manage ',' as decimal point (on demand) and special windows numbers as 1.#INF. Of course, this is easy to wrap in a small function but I expect it to be slow when the input size is in the Mb range. Regards, Christian

2 1

Suggestion about reading text file and extracting data LAS format (newbie questions)
by Sachin Kumar Sharma March 22, 2011

March 22, 2011

BB, I have picked python recently and use it for simple day to day task. I am looking a routine for loading Well Log Las format and storing information as arrays or List. Las format looks like following # LAS format # Project units are specified as depth units #================================================================== ~Version information VERS. 2.0: WRAP. NO: #================================================================== ~Well STRT .ft 1900.0000000 : STOP .ft 6725.0000000 : STEP .ft 0.50000000 : NULL . -999.250000 : COMP. : COMPANY WELL. : WELL FLD. : FIELD LOC. : LOCATION SRVC. : SERVICE COMPANY DATE. Tuesday, March 22 2011 08:31:52 : DATE PROV. : PROVINCE UWI. : UNIQUE WELL ID API. : API NUMBER #================================================================== ~Curve DEPT .ft : DEPTH PHIE .ft3/ft3 : POROSITY SW . : WATER SATURATION VCLfin . : VCL FACIES_NN . : FACIES KairHU .mD : PERM ~Parameter #================================================================== 2711.0000000 0.2031999975 1.0000000000 0.1447000057 5.0000000000 171.17599487 2711.5000000 0.2098000050 1.0000000000 0.1421000063 5.0000000000 199.55209351 2712.0000000 0.2072000057 1.0000000000 0.1570000052 5.0000000000 200.47390747 2712.5000000 0.2063000053 1.0000000000 0.1530999988 5.0000000000 192.77430725 2713.0000000 0.2161999941 1.0000000000 0.1184000000 5.0000000000 208.76330566 2713.5000000 0.2213999927 1.0000000000 0.0970999971 5.0000000000 214.45840454 2714.0000000 0.2188999951 1.0000000000 0.0939999968 5.0000000000 198.89270020 2714.5000000 0.2120999992 1.0000000000 0.1231999993 5.0000000000 192.96949768 2715.0000000 0.2094999999 1.0000000000 0.1480000019 5.0000000000 203.21719360 2715.5000000 0.2054000050 1.0000000000 0.1673000008 6.0000000000 201.36019897 2716.0000000 0.2071000040 1.0000000000 0.1544999927 5.0000000000 197.63549805 2716.5000000 0.2006999999 1.0000000000 0.1601999998 5.0000000000 173.38989258 2717.0000000 0.2045000046 1.0000000000 0.1494999975 5.0000000000 181.11289978 2717.5000000 0.2016000003 1.0000000000 0.1485999972 5.0000000000 167.39660645 2718.0000000 0.2023999989 1.0000000000 0.1332000047 5.0000000000 158.64739990 2718.5000000 0.1915999949 1.0000000000 0.1460999995 5.0000000000 127.54989624 2719.0000000 0.1824000031 1.0000000000 0.1656000018 5.0000000000 110.23919678 2719.5000000 0.1812999994 1.0000000000 0.1741999984 5.0000000000 111.93460083 2720.0000000 0.1774999946 1.0000000000 0.1779000014 5.0000000000 102.91439819 2720.5000000 0.1650000066 1.0000000000 0.1888999939 5.0000000000 76.584999084 2721.0000000 0.1314000040 1.0000000000 0.2365999967 5.0000000000 35.996101379 2721.5000000 0.1238999963 1.0000000000 0.2381000072 5.0000000000 28.215700150 2722.0000000 0.1242000014 1.0000000000 0.2396000028 5.0000000000 28.818500519 2722.5000000 0.1412999928 1.0000000000 0.2210000008 5.0000000000 45.044498444 2723.0000000 0.1543000042 1.0000000000 0.2046999931 5.0000000000 61.145401001 2723.5000000 0.1601999998 1.0000000000 0.2020999938 5.0000000000 71.653701782 2724.0000000 0.1479000002 1.0000000000 0.2287999988 5.0000000000 57.839698792 2724.5000000 0.1274999976 1.0000000000 0.2714999914 7.0000000000 39.379501343 2725.0000000 0.1230000034 1.0000000000 0.2777000070 7.0000000000 35.316699982 2725.5000000 0.1237000003 1.0000000000 0.2709000111 7.0000000000 34.647201538 Blue shows the header precedes the data (shown in green). The file normally contains between 3000-10000 data rows. If someone can share a routine existing for the same or Can advise what is the best way to read these lines. Currently, I edit these files by hand to remove the headers and use something like this to plot X = numpy.loadtxt('ASCII.txt') # X is an array t = X [: ,0] # extract the first column v = X [: ,1] # extract the second column len (t) len (v) plot (t,v) # plot the data Is there a way, I can use readline option to direct to read from where the data starts and pick the name of columns from header like porosity, permeability. Cheers Sachin ************************************************************************ Sachin Kumar Sharma Senior Geomodeler

2 1

argmin and datetimes
by Benjamin Root March 21, 2011

March 21, 2011

Came across an odd behavior with a numpy array of datetimes: >>> import numpy as np >>> from datetime import datetime >>> a = np.empty((9, 20), dtype=datetime) >>> a.fill(datetime.now()) This works: >>> a.min() datetime.datetime(2011, 3, 21, 15, 5, 10, 990733) but, this fails: >>> np.argmin(a) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/bvr/Programs/numpy/numpy/core/fromnumeric.py", line 754, in argmin return argmin(axis) TypeError: unsupported operand type(s) for -: 'int' and 'datetime.datetime' Is this a bug, or am I just doing datetimes wrong? For reference, I am at commit aada93306acfb4e2eb816faf32652edf8825cf45 Thanks, Ben

2 1