Mailman 3 July 2007 - NumPy-Discussion

Undefined symbol "fpsetsticky"
by Luis N 17 Jul '07

17 Jul '07

I just built numpy from svn checkout: Python 2.4.3 (#2, Nov 8 2006, 23:56:15) [GCC 3.4.6 [FreeBSD] 20060305] on freebsd6 Type "help", "copyright", "credits" or "license" for more information. >>> import numpy Traceback (most recent call last): File "<stdin>", line 1, in ? File "/usr/local/lib/python2.4/site-packages/numpy/__init__.py", line 39, in ? import core File "/usr/local/lib/python2.4/site-packages/numpy/core/__init__.py", line 6, in ? import umath ImportError: /usr/local/lib/python2.4/site-packages/numpy/core/umath.so: Undefined symbol "fpsetsticky" What might this suggest?

1 0

NumPy/SciPy LAPACK version
by Kevin Jacobs <jacobs＠bioinformed.com> 16 Jul '07

16 Jul '07

Hi all, This is a bit of a SciPy question, but I thought I'd ask here since I'm already subscribed. I'd like to add some new LAPACK bindings to SciPy and was wondering if there was a minimum version requirement for LAPACK, since it would be ideal if I could use some of the newer 3.0 features. In addition to using some block methods only added in 3.0, it is very convenient to use the WORK=-1 for space queries instead of reimplementing the underlying logic in the calc_work module. The routines of most interest to me are: DGELSD DGGGLM DGGLSE I've also found that SciPy's sqrtm is broken: >>> a=matrix([[59, 64, 69], [64, 72, 80], [69, 80, 91]]) >>> sqrtm(a) array([[ 5.0084801 +1.03420519e-08j, 4.40747825 -2.06841037e-08j, 3.8064764 +1.03420519e-08j], [ 4.40747825 -2.06841037e-08j, 4.88353492 +4.13682075e-08j, 5.3595916 -2.06841037e-08j], [ 3.8064764 +1.03420519e-08j, 5.3595916 -2.06841037e-08j, 6.9127068 +1.03420519e-08j]]) >>> sqrtm(a)*sqrtm(a) array([[ 25.08487289 +1.03595922e-07j, 19.42586452 -1.82329475e-07j, 14.48926259 +7.87335527e-08j], [ 19.42586452 -1.82329475e-07j, 23.84891336 +4.04046172e-07j, 28.72522212 -2.21716697e-07j], [ 14.48926259 +7.87335527e-08j, 28.72522212 -2.21716697e-07j, 47.78551529 +1.42983144e-07j]]) So not even close... (and yes, it does deserve a bug report if one doesn't already exist) -Kevin

5 12

Porting "IDL Astronomy User's Library" to numpy
by Yannick Copin 16 Jul '07

16 Jul '07

Hi, I'd be interested in some astronomical utilities from the IDL Astronomy User's Library (http://idlastro.gsfc.nasa.gov/contents.html) converted to python/numpy. I had a look to idl2python (http://software.pseudogreen.org/i2py/), but the automatic translation fails, mostly because (I think) the conversion is Numeric-oriented, and because of the intrinsic differences in the function argument management between IDL and python. So, before pursuing in this direction, I'd like to know if this exercice has already been done, at least partially. Cheers. -- .~. Yannick COPIN (o:>* Doctus cum libro /V\ Institut de physique nucleaire de Lyon (IN2P3 - France) // \\ Tel: (33/0) 472 431 968 AIM: YnCopin ICQ: 236931013 /( )\ http://snovae.in2p3.fr/ycopin/ ^`~'^

4 4

Problems building SVN on Windows
by numpy-discussion＠robince.ftml.net 16 Jul '07

16 Jul '07

Hi, I am trying to build numpy on Windows from the current SVN (rev. 3884 I think) and I'm having some problems. I've successfully built ATLAS and LAPACK. I've installed Python 2.5.1 from the standard windows installer. I also have a fresh install of Cygwin with all mingw development tools etc. I created a site.cfg file from the template with the following entries: ----------- [blas_opt] libraries = f77blas, cblas, atlas library_dirs = C:\cygwin\home\mqbxfri2\ATLAS\atlas_win32\lib include_dirs = C:\cygwin\home\mqbxfri2\ATLAS\atlas_win32\include # [lapack_opt] libraries = lapack, f77blas, cblas, atlas library_dirs = C:\cygwin\home\mqbxfri2\ATLAS\atlas_win32\lib include_dirs = C:\cygwin\home\mqbxfri2\ATLAS\atlas_win32\include [fftw] libraries = fftw3 library_dirs = "C:\fftw" ----------- So the first problem is this file doesn't seem to be recognised (and these library paths aren't picked up). I've tried from the cygwin shell and windows cmd and with the file next to setup.py and also in numpy/distutils. However the only paths searched are C:\,C:\Python25\Libs,C:\Python25\lib. I got around this by copying the .a files to C:\. This is a bit messy but at least it finds ATLAS and LAPACK. Then when I try to run config/build as described on the wiki I get an error about no valid fortran compiler being defined. The full output of python setup.py -v build --compiler=mingw32 is attached. I would be greatful if anyone could help me to get this working. Also if someone could provide more details on how to ensure fftw is used. The precompiled fftw library I downloaded provides libfftw3-3.dll as well as couple of others. Will this be picked up correctly by my fftw entry in site.cfg (if site.cfg was working) or should it be "libraries=fftw3-3"? Is it OK to use this DLL or do I have to build a static library myself from source. Finally, I gather the very nice scipy website/wiki is relatively new... I was wondering if you had considered a phpbb type forum for support issues such as this. I think this can be a lot more accessible for new users (such as myself)... Thanks in advance, Robin

6 8

ANN: SciPy 2007 Conference Updates
by Travis Vaught 15 Jul '07

15 Jul '07

Greetings, We're excited to have *Ivan Krstić*, the director of security architecture for the One Laptop Per Child project as our Keynote Speaker this year. The planning for the SciPy 2007 Conference is moving along. Please see below for some important updates. Schedule Available ------------------ The full schedule of talks has been posted here: http://www.scipy.org/SciPy2007/ConferenceSchedule Early Registration Extended --------------------------- If you haven't yet registered for the conference, the early registration deadline has been extended to Wednesday, July 18th, 2007. For more information on the conference see: http://www.scipy.org/SciPy2007 Student Sponsorship ------------------- Enthought, Inc. (http://www.enthought.com) is sponsoring the registration fees for up to 5 college or graduate students to attend the conference. To apply, please send a short description of what you are studying and why you'd like to attend to info(a)enthought.com. Please include telephone contact information. BOFs & Sprints -------------- If you're planning to attend and are interested in selecting BOF or Sprint Session topics, please weigh in at: BOFs: http://www.scipy.org/SciPy2007/BoFs Sprints: http://www.scipy.org/SciPy2007/Sprints We're looking forward to a great conference this year! Best, Travis

1 0

optimization of arr**-2
by Sebastian Haase 15 Jul '07

15 Jul '07

Hi, I compared for a 256x256 float32 normal-noise (x0=100,sigma=1) array the times to do 1./ (a*a) vs. a**-2 >>> U.timeIt('1./(a*a)', 1000) (0.00090877471871, 0.00939644563778, 0.00120674694689, 0.000687777554628) >>> U.timeIt('a**-2', 1000) (0.00876591857354, 0.0263829620803, 0.00952076311375, 0.00173311803255) The numbers are min,max, mean, stddev over thousand runs. N.__version == 1.0.1 The slowdown is almost 10 fold. Similar tests for **-1, and **2 show that the corresponding times are identical - i.e. those cases are optimized to not call the pow routine. Can this be fixed for the **-2 case ? Thanks, Sebastian Haase

2 1

numpy.where
by Sebastian Haase 15 Jul '07

15 Jul '07

Hi. Two things. 1) The doc-string of numpy.where() states that transpose(where(cond, x,y)) whould always return a 2d-array. How can this be true?? It also says (before) that if x,y are given where(cond,x,y) always returns an array of the same shape as cond .... 2) Could we have another optional argument "dtype" in numpy.where()? Otherwise I would have to always write code like this: a = N.where( arr>x, 1.0, 0.0) a = a.astype(N.float32) I use N.__version__ == '1.0.1' Thanks, Sebastian Haase

3 6

Should bool_ subclass int?
by Timothy Hochberg 14 Jul '07

14 Jul '07

I'm working on getting some old code working with numpy and I noticed that bool_ is not a subclass of int. Given that python's bool subclasses into and that the other scalar types are subclasses of their respective counterparts it seems at first glance that numpy.bool_ should subclass python's bool, which in turn subclasses int. Or am I missing something here? -- . __ . |-\ . . tim.hochberg(a)ieee.org

11 29

[ANN] PyTables & PyTables Pro 2.0 released
by Ivan Vilata i Balaguer 13 Jul '07

13 Jul '07

======================================== Announcing PyTables & PyTables Pro 2.0 ======================================== PyTables is a library for managing hierarchical datasets and designed to efficiently cope with extremely large amounts of data with support for full 64-bit file addressing. PyTables runs on top of the HDF5 library and NumPy package for achieving maximum throughput and convenient use. After more than one year of continuous development and about five months of alpha, beta and release candidates, we are very happy to announce that the PyTables and PyTables Pro 2.0 are here. We are pretty confident that the 2.0 versions are ready to be used in production scenarios, bringing higher performance, better portability (specially in 64-bit environments) and more stability than the 1.x series. You can download a source package of the PyTables 2.0 with generated PDF and HTML docs and binaries for Windows from http://www.pytables.org/download/stable/ For an on-line version of the manual, visit: http://www.pytables.org/docs/manual-2.0 In case you want to know more in detail what has changed in this version, have a look at ``RELEASE_NOTES.txt``. Find the HTML version for this document at: http://www.pytables.org/moin/ReleaseNotes/Release_2.0 If you are a user of PyTables 1.x, probably it is worth for you to look at ``MIGRATING_TO_2.x.txt`` file where you will find directions on how to migrate your existing PyTables 1.x apps to the 2.0 version. You can find an HTML version of this document at http://www.pytables.org/moin/ReleaseNotes/Migrating_To_2.x Introducing PyTables Pro 2.0 ============================ The main difference between PyTables Pro and regular PyTables is that the Pro version includes OPSI, a new indexing technology, allowing to perform data lookups in tables exceeding 10 gigarows (10**10 rows) in less than 1 tenth of a second. Wearing more than 15000 tests and having passed the complete test suite in the most common platforms (Windows, Mac OS X, Linux 32-bit and Linux 64-bit), we are pretty confident that PyTables Pro 2.0 is ready to be used in production scenarios, bringing maximum stability and top performance to those users who need it. For more info about PyTables Pro, see: http://www.carabos.com/products/pytables-pro For the operational details and benchmarks see the OPSI white paper: http://www.carabos.com/docs/OPSI-indexes.pdf Coinciding with the publication of PyTables Pro we are introducing an innovative liberation process that will allow to ultimate release the PyTables Pro 2.x series as open source. You may want to know that, by buying a PyTables Pro license, you are contributing to this process. For details, see: http://www.carabos.com/liberation New features of PyTables 2.0 series =================================== - A complete refactoring of many, many modules in PyTables. With this, the different parts of the code are much better integrated and code redundancy is kept under a minimum. A lot of new optimizations have been included as well, making working with it a smoother experience than ever before. - NumPy is finally at the core! That means that PyTables no longer needs numarray in order to operate, although it continues to be supported (as well as Numeric). This also means that you should be able to run PyTables in scenarios combining Python 2.5 and 64-bit platforms (these are a source of problems with numarray/Numeric because they don't support this combination as of this writing). - Most of the operations in PyTables have experimented noticeable speed-ups (sometimes up to 2x, like in regular Python table selections). This is a consequence of both using NumPy internally and a considerable effort in terms of refactorization and optimization of the new code. - Combined conditions are finally supported for in-kernel selections. So, now it is possible to perform complex selections like:: result = [ row['var3'] for row in table.where('(var2 < 20) | (var1 == "sas")') ] or:: complex_cond = '((%s <= col5) & (col2 <= %s)) ' \ '| (sqrt(col1 + 3.1*col2 + col3*col4) > 3)' result = [ row['var3'] for row in table.where(complex_cond % (inf, sup)) ] and run them at full C-speed (or perhaps more, due to the cache-tuned computing kernel of Numexpr, which has been integrated into PyTables). - Now, it is possible to get fields of the ``Row`` iterator by specifying their position, or even ranges of positions (extended slicing is supported). For example, you can do:: result = [ row[4] for row in table # fetch field #4 if row[1] < 20 ] result = [ row[:] for row in table # fetch all fields if row['var2'] < 20 ] result = [ row[1::2] for row in # fetch odd fields table.iterrows(2, 3000, 3) ] in addition to the classical:: result = [row['var3'] for row in table.where('var2 < 20')] - ``Row`` has received a new method called ``fetch_all_fields()`` in order to easily retrieve all the fields of a row in situations like:: [row.fetch_all_fields() for row in table.where('column1 < 0.3')] The difference between ``row[:]`` and ``row.fetch_all_fields()`` is that the former will return all the fields as a tuple, while the latter will return the fields in a NumPy void type and should be faster. Choose whatever fits better to your needs. - Now, all data that is read from disk is converted, if necessary, to the native byteorder of the hosting machine (before, this only happened with ``Table`` objects). This should help to accelerate applications that have to do computations with data generated in platforms with a byteorder different than the user machine. - The modification of values in ``*Array`` objects (through __setitem__) now doesn't make a copy of the value in the case that the shape of the value passed is the same as the slice to be overwritten. This results in considerable memory savings when you are modifying disk objects with big array values. - All leaf constructors (except for ``Array``) have received a new ``chunkshape`` argument that lets the user explicitly select the chunksizes for the underlying HDF5 datasets (only for advanced users). - All leaf constructors have received a new parameter called ``byteorder`` that lets the user specify the byteorder of their data *on disk*. This effectively allows to create datasets in other byteorders than the native platform. - Native HDF5 datasets with ``H5T_ARRAY`` datatypes are fully supported for reading now. - The test suites for the different packages are installed now, so you don't need a copy of the PyTables sources to run the tests. Besides, you can run the test suite from the Python console by using:: >>> tables.tests() Resources ========= Go to the PyTables web site for more details: http://www.pytables.org/ Go to the PyTables Pro web page for more details: http://www.carabos.com/products/pytables-pro About the HDF5 library: http://hdfgroup.org/HDF5/ About NumPy: http://numpy.scipy.org/ To know more about the company behind the development of PyTables, see: http://www.carabos.com/ Acknowledgments =============== Thanks to many users who provided feature improvements, patches, bug reports, support and suggestions. See the ``THANKS`` file in the distribution package for a (incomplete) list of contributors. Many thanks also to SourceForge who have helped to make and distribute this package! And last, but not least thanks a lot to the HDF5 and NumPy (and numarray!) makers. Without them PyTables simply would not exist. Share your experience ===================== Let us know of any bugs, suggestions, gripes, kudos, etc. you may have. ---- **Enjoy data!** -- The PyTables Team

1 0

Conversion from numarray to numpy
by Michał Szpadzik 12 Jul '07

12 Jul '07

Some time ago I wrote small lib for signal processing. Now I'm trying to convert it from numarray to numpy, and I've problem: Code in numarray: for i in range(len(myinput)-m+1): cin=tempmatr[:] ctmp=tempmatr[i] xtmp=(numarray.abs(cin-ctmp))<=r x2tmp=numarray.sum(numarray.transpose(xtmp)) mcount=numarray.sum(x2tmp==m) allcount=allcount+mcount Code in numpy: for i in range(len(myinput)-m+1): cin=tempmatr[:] ctmp=tempmatr[i] xtmp=(numpy.abs(cin-ctmp))<=r x2tmp=numpy.sum(numpy.transpose(xtmp)) mcount=numpy.sum(x2tmp==m) allcount=allcount+mcount In numarray line: xtmp=(numarray.abs(cin-ctmp))<=r returns array of ones and zeros, but in numpy xtmp=(numpy.abs(cin-ctmp))<=r returns True/False I would like to know how change this True/False to 1/0 THX -- Mike Szpadzik

3 4