NumPy-Discussion November 2011

numpy-discussion@python.org

64 participants
66 discussions

BSD C port of FFTPACK incl. bluestein algorithm
by Dag Sverre Seljebotn 18 Nov '11

18 Nov '11

I've been in touch with Martin Reinecke, author of the libpsht code for spherical harmonic transforms, about licensing issues. libpsht itself will remain under the GPL, but he is likely to release his C port of FFTPACK under BSD in the near future, as it is based on the public domain FFTPACK. I'm grateful for this change for my own purposes (allows releasing my own competing SHT library under the BSD) -- but it could perhaps be useful for NumPy or SciPy as well, depending on how complete the port is? E.g., perhaps make numpy.fft more complete (is the numpy.fft/scipy.fftpack split simply because of the Fortran dependency?). Not sure about whether all of FFTPACK or a subset is included, but it does include a Bluestein implementation in addition. http://libpsht.svn.sourceforge.net/viewvc/libpsht/libfftpack/ Dag Sverre

4 6

Re: [Numpy-discussion] how to use the name of a ndarray as a string
by Chao YUE 18 Nov '11

18 Nov '11

Thanks to all people for this very nice discussions. the solutions are more that what I want!! and help me to clarify some concepts, and really begin to use class as a beginner :) I would say either Olivier or denis's solution can solve my problem completely. cheers, Chao 2011/11/18 denis <denis-bz-gg(a)t-online.de> > as Olivier and Chris say, use a dict from the start. > To make this a bit easier / more readable, > use this tiny class Dotdict: > > class Dotdict( dict ): > """ d = dotdict(), d.key same as d["key"] """ > # aka Bag, Bunch > # cPickle.dumps( d, -1 ): PicklingError in 2.6.4 > def __init__(self, **kwargs): > dict.__init__(self, kwargs) > self.__dict__ = self > > d = Dotdict() > d.a = np.arange(10) > d.b = np.ones((3,4)) > d.b[1,2] += 1 > print d.b > > > One can also save all the d.* arrays like this: > > def savedotdict( d, prefix="d.", save=np.savetxt, **kwargs ): > """ save all d.* to files """ > for name, val in d.items(): > out = prefix + name > if isinstance( val, basestring ): > with open( out, "w" ) as f: > f.write( val.rstrip() ) > f.write( "\n" ) > else: > print "saving %s to %s" % (val.shape, out) > save( out, val, **kwargs ) > > d.info = "2011-11-18 nov kilroy" # From() > savedotdict( d, "d.", fmt="%.3g" ) > > cheers > -- denis > > (If you use this, could you post it to the numpy-discussion group > please ? > It rejects me, grr.) > > > On Nov 10, 11:17 am, Chao YUE <chaoyue...(a)gmail.com> wrote: > > Hi, > > > > Does anyone know how I can quickly use the name of a ndarray as a string? > -- *********************************************************************************** Chao YUE Laboratoire des Sciences du Climat et de l'Environnement (LSCE-IPSL) UMR 1572 CEA-CNRS-UVSQ Batiment 712 - Pe 119 91191 GIF Sur YVETTE Cedex Tel: (33) 01 69 08 29 02; Fax:01.69.08.77.16 ************************************************************************************

2 1

Subclassing ndarray
by Fabrice Silva 18 Nov '11

18 Nov '11

Hi, I wish I could subclass ndarray so that I can use the ufuncs, but that I could ensure the array remains Fortran contiguous, and that the number of dimension is always 3 (even if the dtype can be more complex than a simple numerical type). My intention is to deal with fields that can be scalar, vectorial or even tensorial, but it must stay compatible with a library we are using that assumes 3D problems with column-major order, and some more attributes concerning spatial steps, strides and comments... I have been looking at __array_{wrap, prepare, finalize}__ methods. I would believe __array_prepare__ would be the good starting point to impose the 'F'-ordered array and atleast_3d, but the point is that I don't know at that time if the ufunc will transform the datatype (from vector to scalar, from int to float, etc...) Some thought? Regards -- Fabrice Silva

1 0

ANN: Spyder v2.1.2
by Pierre Raybaut 16 Nov '11

16 Nov '11

Hi all, On the behalf of Spyder's development team (http://code.google.com/p/spyderlib/people/list), I'm pleased to announce that Spyder v2.1.2 has been released and is available for Windows XP/Vista/7, GNU/Linux and MacOS X: http://code.google.com/p/spyderlib/ As this is mostly a maintenance release, a lot of bugs were fixed and some minor features were added: http://code.google.com/p/spyderlib/wiki/ChangeLog Spyder is a free, open-source (MIT license) interactive development environment for the Python language with advanced editing, interactive testing, debugging and introspection features. Originally designed to provide MATLAB-like features (integrated help, interactive console, variable explorer with GUI-based editors for dictionaries, NumPy arrays, ...), it is strongly oriented towards scientific computing and software development. Thanks to the `spyderlib` library, Spyder also provides powerful ready-to-use widgets: embedded Python console (example: http://packages.python.org/guiqwt/_images/sift3.png), NumPy array editor (example: http://packages.python.org/guiqwt/_images/sift2.png), dictionary editor, source code editor, etc. Description of key features with tasty screenshots can be found at: http://code.google.com/p/spyderlib/wiki/Features On Windows platforms, Spyder is also available as a stand-alone executable (don't forget to disable UAC on Vista/7). This all-in-one portable version is still experimental (for example, it does not embed sphinx -- meaning no rich text mode for the object inspector) but it should provide a working version of Spyder for Windows platforms without having to install anything else (except Python 2.x itself, of course). Don't forget to follow Spyder updates/news: * on the project website: http://code.google.com/p/spyderlib/ * and on our official blog: http://spyder-ide.blogspot.com/ Last, but not least, we welcome any contribution that helps making Spyder an efficient scientific development/computing environment. Join us to help creating your favourite environment! (http://code.google.com/p/spyderlib/wiki/NoteForContributors) Enjoy! -Pierre

1 0

Using dtype=object
by Nathan Faggian 16 Nov '11

16 Nov '11

Hi, I am interested in the use of numpy with native python objects, like so: In [91]: import collections In [92]: testContainer = collections.namedtuple('testContainer', 'att1 att2 att3') In [93]: test1 = testContainer(1, 2, 3) In [94]: test2 = testContainer(4, 5, 6) In [95]: test1 Out[95]: testContainer(att1=1, att2=2, att3=3) In [96]: test2 Out[96]: testContainer(att1=4, att2=5, att3=6) In [97]: x = np.empty((2,2), dtype=object) In [98]: x[0,0] = test1 In [99]: x[1,1] = test2 In [100]: x Out[100]: array([[testContainer(att1=1, att2=2, att3=3), None], [None, testContainer(att1=4, att2=5, att3=6)]], dtype=object) Does anyone know if it possible to form a mask using the attributes of the objects stored in the ndarray? After a few failed attempts I am left wondering if I should use a prepared dtype instead. Cheers, Nathan.

2 1

Memory hungry reduce ops in Numpy
by Andreas Müller 15 Nov '11

15 Nov '11

Hi everybody. When I did some normalization using numpy, I noticed that numpy.std uses more ram than I was expecting. A quick google search gave me this: http://luispedro.org/software/ncreduce The site claims that std and other reduce operations are implemented naively with many temporaries. Is that true? And if so, is there a particular reason for that? This issues seems quite easy to fix. In particular the link I gave above provides code. Cheers, Andy

6 12

dedicated function for resize with averaging or rebin 2d arrays?
by Andrea Zonca 15 Nov '11

15 Nov '11

hi, I work in astrophysics where the most common programming language is currently IDL. A common request of people switching from IDL to python is the implementation of the REBIN function, which either downsizes a 2d array by averaging or increases its dimension by repeating its elements. In both cases the new shape must be an integer factor of the old shape. I believe it is a very handy function for quick smoothing of 2 dimensional data. I found a discussion about this topic in the archives: http://thread.gmane.org/gmane.comp.python.numeric.general/885/focus=894 Do you think it would be useful to add such function to numpy? I created a simple implementation to help in the discussion: https://gist.github.com/1348792 thanks, Andrea Zonca

4 6

numpy.int32 is not subclass of int, but numpy.int64 is
by MACKEITH Andrew 15 Nov '11

15 Nov '11

Could someone explain this? An instance of numpy.int32 is not an instance of int or numpy.int. An instance of numpy.int64 is an instance of int and numpy.int. I don't know if it is a bug in my linux build. Andrew >python26 Python 2.6.2 (r262:71600, Jul 8 2010, 11:49:56) [GCC 4.1.2 20070115 (SUSE Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> >>> import numpy >>> a = numpy.array((1,2,3)) >>> type(a[0]) <type 'numpy.int64'> >>> isinstance(a[0], int) True >>> isinstance(a[0], numpy.int) True >>> seven = numpy.int64(7) >>> isinstance(seven, numpy.int) True >>> isinstance(seven, int) True >>> >>> five = numpy.int32(5) >>> isinstance(five, numpy.int) False >>> isinstance(five, int) False >>> five 5 >>> seven 7 >>> issubclass(numpy.int64, numpy.int) True >>> issubclass(numpy.int32, numpy.int) False >>> >>> import sys >>> sys.maxint 9223372036854775807 >>> import platform >>> print platform.platform() Linux-2.6.32.12-0.7-default-x86_64-with-SuSE-11-x86_64 >>> This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged. If you are not one of the named recipients or have received this email in error, (i) you should not read, disclose, or copy it, (ii) please notify sender of your receipt by reply email and delete this email and all attachments, (iii) Dassault Systemes does not accept or assume any liability or responsibility for any use of or reliance on this email. For other languages, go to http://www.3ds.com/terms/email-disclaimer

3 5

Odd-looking long double on windows 32 bit
by Matthew Brett 15 Nov '11

15 Nov '11

Hi, Sorry for my continued confusion here. This is numpy 1.6.1 on windows XP 32 bit. In [2]: np.finfo(np.float96).nmant Out[2]: 52 In [3]: np.finfo(np.float96).nexp Out[3]: 15 In [4]: np.finfo(np.float64).nmant Out[4]: 52 In [5]: np.finfo(np.float64).nexp Out[5]: 11 If there are 52 bits of precision, 2**53+1 should not be representable, and sure enough: In [6]: np.float96(2**53)+1 Out[6]: 9007199254740992.0 In [7]: np.float64(2**53)+1 Out[7]: 9007199254740992.0 If the nexp is right, the max should be higher for the float96 type: In [9]: np.finfo(np.float64).max Out[9]: 1.7976931348623157e+308 In [10]: np.finfo(np.float96).max Out[10]: 1.#INF I see that long double in C is 12 bytes wide, and double is the usual 8 bytes. So - now I am not sure what this float96 is. I was expecting 80 bit extended precision, but it doesn't look right for that... Does anyone know what representation this is? Thanks a lot, Matthew

3 11

ANN: SciPy 0.10.0 released
by Ralf Gommers 14 Nov '11

14 Nov '11

Hi all, I am pleased to announce the availability of SciPy 0.10.0. For this release over a 100 tickets and pull requests have been closed, and many new features have been added. Some of the highlights are: - support for Bento as a build system for scipy - generalized and shift-invert eigenvalue problems in sparse.linalg - addition of discrete-time linear systems in the signal module Sources and binaries can be found at , release notes are copied below. Enjoy, The SciPy developers ========================== SciPy 0.10.0 Release Notes ========================== .. contents:: SciPy 0.10.0 is the culmination of 8 months of hard work. It contains many new features, numerous bug-fixes, improved test coverage and better documentation. There have been a limited number of deprecations and backwards-incompatible changes in this release, which are documented below. All users are encouraged to upgrade to this release, as there are a large number of bug-fixes and optimizations. Moreover, our development attention will now shift to bug-fix releases on the 0.10.x branch, and on adding new features on the development master branch. Release highlights: - Support for Bento as optional build system. - Support for generalized eigenvalue problems, and all shift-invert modes available in ARPACK. This release requires Python 2.4-2.7 or 3.1- and NumPy 1.5 or greater. New features ============ Bento: new optional build system -------------------------------- Scipy can now be built with `Bento <http://cournape.github.com/Bento/>`_. Bento has some nice features like parallel builds and partial rebuilds, that are not possible with the default build system (distutils). For usage instructions see BENTO_BUILD.txt in the scipy top-level directory. Currently Scipy has three build systems, distutils, numscons and bento. Numscons is deprecated and is planned and will likely be removed in the next release. Generalized and shift-invert eigenvalue problems in ``scipy.sparse.linalg`` --------------------------------------------------------------------------- The sparse eigenvalue problem solver functions ``scipy.sparse.eigs/eigh`` now support generalized eigenvalue problems, and all shift-invert modes available in ARPACK. Discrete-Time Linear Systems (``scipy.signal``) ----------------------------------------------- Support for simulating discrete-time linear systems, including ``scipy.signal.dlsim``, ``scipy.signal.dimpulse``, and ``scipy.signal.dstep``, has been added to SciPy. Conversion of linear systems from continuous-time to discrete-time representations is also present via the ``scipy.signal.cont2discrete`` function. Enhancements to ``scipy.signal`` -------------------------------- A Lomb-Scargle periodogram can now be computed with the new function ``scipy.signal.lombscargle``. The forward-backward filter function ``scipy.signal.filtfilt`` can now filter the data in a given axis of an n-dimensional numpy array. (Previously it only handled a 1-dimensional array.) Options have been added to allow more control over how the data is extended before filtering. FIR filter design with ``scipy.signal.firwin2`` now has options to create filters of type III (zero at zero and Nyquist frequencies) and IV (zero at zero frequency). Additional decomposition options (``scipy.linalg``) --------------------------------------------------- A sort keyword has been added to the Schur decomposition routine (``scipy.linalg.schur``) to allow the sorting of eigenvalues in the resultant Schur form. Additional special matrices (``scipy.linalg``) ---------------------------------------------- The functions ``hilbert`` and ``invhilbert`` were added to ``scipy.linalg``. Enhancements to ``scipy.stats`` ------------------------------- * The *one-sided form* of Fisher's exact test is now also implemented in ``stats.fisher_exact``. * The function ``stats.chi2_contingency`` for computing the chi-square test of independence of factors in a contingency table has been added, along with the related utility functions ``stats.contingency.margins`` and ``stats.contingency.expected_freq``. Basic support for Harwell-Boeing file format for sparse matrices ---------------------------------------------------------------- Both read and write are support through a simple function-based API, as well as a more complete API to control number format. The functions may be found in scipy.sparse.io. The following features are supported: * Read and write sparse matrices in the CSC format * Only real, symmetric, assembled matrix are supported (RUA format) Deprecated features =================== ``scipy.maxentropy`` -------------------- The maxentropy module is unmaintained, rarely used and has not been functioning well for several releases. Therefore it has been deprecated for this release, and will be removed for scipy 0.11. Logistic regression in scikits.learn is a good alternative for this functionality. The ``scipy.maxentropy.logsumexp`` function has been moved to ``scipy.misc``. ``scipy.lib.blas`` ------------------ There are similar BLAS wrappers in ``scipy.linalg`` and ``scipy.lib``. These have now been consolidated as ``scipy.linalg.blas``, and ``scipy.lib.blas`` is deprecated. Numscons build system --------------------- The numscons build system is being replaced by Bento, and will be removed in one of the next scipy releases. Backwards-incompatible changes ============================== The deprecated name `invnorm` was removed from ``scipy.stats.distributions``, this distribution is available as `invgauss`. The following deprecated nonlinear solvers from ``scipy.optimize`` have been removed:: - ``broyden_modified`` (bad performance) - ``broyden1_modified`` (bad performance) - ``broyden_generalized`` (equivalent to ``anderson``) - ``anderson2`` (equivalent to ``anderson``) - ``broyden3`` (obsoleted by new limited-memory broyden methods) - ``vackar`` (renamed to ``diagbroyden``) Other changes ============= ``scipy.constants`` has been updated with the CODATA 2010 constants. ``__all__`` dicts have been added to all modules, which has cleaned up the namespaces (particularly useful for interactive work). An API section has been added to the documentation, giving recommended import guidelines and specifying which submodules are public and which aren't. Authors ======= This release contains work by the following people (contributed at least one patch to this release, names in alphabetical order): * Jeff Armstrong + * Matthew Brett * Lars Buitinck + * David Cournapeau * FI$H 2000 + * Michael McNeil Forbes + * Matty G + * Christoph Gohlke * Ralf Gommers * Yaroslav Halchenko * Charles Harris * Thouis (Ray) Jones + * Chris Jordan-Squire + * Robert Kern * Chris Lasher + * Wes McKinney + * Travis Oliphant * Fabian Pedregosa * Josef Perktold * Thomas Robitaille + * Pim Schellart + * Anthony Scopatz + * Skipper Seabold + * Fazlul Shahriar + * David Simcha + * Scott Sinclair + * Andrey Smirnov + * Collin RM Stocks + * Martin Teichmann + * Jake Vanderplas + * Gaël Varoquaux + * Pauli Virtanen * Stefan van der Walt * Warren Weckesser * Mark Wiebe + A total of 35 people contributed to this release. People with a "+" by their names contributed a patch for the first time.

4 4

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

NumPy-Discussion November 2011