Mailman 3 December 2004 - SciPy-User

io.loadmat: TypeError Array can not be safely cast to required type
by Yichun Wei Dec. 14, 2004

Dec. 14, 2004

I'm sorry. This is only for v6 .mat file. - yichun

1 0

io.loadmat: TypeError Array can not be safely cast to required type
by Yichun Wei Dec. 14, 2004

Dec. 14, 2004

Hi, Sorry if you received multiple messages about this. I do not know if this is a perticular case. I was fine with io.loadmat about 1 week ago. It persuaded me to make a switch to scipy. However, when I updated my python to active python 2.3.4, the following happened: >>> import scipy.io >>> scipy.io.loadmat("test") Traceback (most recent call last): File "<stdin>", line 1, in ? File "C:\Python23\Lib\site-packages\scipy\io\mio.py", line 692, in loadmat thisdict = _loadv5(fid,basename) File "C:\Python23\Lib\site-packages\scipy\io\mio.py", line 631, in _loadv5 el, varname = _get_element(fid) File "C:\Python23\Lib\site-packages\scipy\io\mio.py", line 619, in _get_element el, name = _parse_mimatrix(fid,numbytes) File "C:\Python23\Lib\site-packages\scipy\io\mio.py", line 510, in _parse_mimatrix result = squeeze(transpose(reshape(result,dims[::-1]))) TypeError: Array can not be safely cast to required type Scipy 0.3.2_266.4242 Numeric tried with 23.5 and 23.6, with the same problem Python 2.3.4 and 2.3.2 from activepython anyone can help me out? - yichun

1 0

Incorporating SUNDIALS in SciPy
by Jan-Jan van der Vyver Dec. 14, 2004

Dec. 14, 2004

Dear SciPy-community, The latest package of SUNDAILS (SUite of Nonlinear and DIfferential/ALgebraic equation Solvers) have just been released (BSD license). Would it make sense to integrate this in SciPy? If not, why not? (The major reason I'm still using Matlab is easy to use ode solvers.) Thanks, Jan-Jan -------- Forwarded Message -------- From: Radu Serban <serban1(a)llnl.gov> To: Radu Serban <serban1(a)llnl.gov> Subject: SUNDIALS 2.0 released Date: Thu, 09 Dec 2004 13:30:38 -0800 SUNDIALS (SUite of Nonlinear and DIfferential/ALgebraic equation Solvers) version 2.0 has been released under an open source BSD license. SUNDIALS 2.0 consists of the following four solvers: CVODE v.2.2.0 solves initial value problems for ordinary differential equation (ODE) systems. CVODES v.2.1.0 solves ODE systems and includes sensitivity analysis capabilities (both forward and adjoint). IDA v.2.2.0 solves initial value problems for differential-algebraic equation (DAE) systems. KINSOL v.2.2.0 solves nonlinear algebraic systems. SUNDIALS was implemented with the goal of providing robust time integrators and nonlinear solvers that can easily be incorporated into existing simulation codes. Basic operations on vectors are isolated to a separate module, for which the distributed package includes serial and parallel versions, but the user can supply his/her own version. The SUNDIALS solvers are written in ANSI C, with CVODE and KINSOL also providing a FORTRAN to C interface. All solvers can be built in single, double, or extended precision. More information (description, documentation, changes in the current versions) and download links can be found on the updated SUNDIALS web page: http://www.llnl.gov/CASC/sundials -- Radu Serban for the SUNDIALS team

5 11

Floating-point summation
by Nils Wagner Dec. 14, 2004

Dec. 14, 2004

Hi all, Afaik scipy's sum(x) relies on Numeric's sum implementation. However, the result of the actual implementation of sum(x) may be inaccurate as illustrated in the small example (sum.py). Any comment ? Nils

2 2

ANN: IPython 0.6.5 is out.
by Fernando Perez Dec. 13, 2004

Dec. 13, 2004

Hi all, [I'm taking the liberty to announce this here, as many scipy/matplotlib users are also ipython users, and this release includes changes coordinated with the new matplotlib release, coming very soon. Sorry for those getting duplicates if you are on all these lists.] I'm glad to announce the release of IPython 0.6.6. IPython's homepage is at: http://ipython.scipy.org and downloads are at: http://ipython.scipy.org/dist I've provided RPMs (Py2.2 and 2.3), plus source downloads (.tar.gz and .zip). Debian, Fink and BSD packages for this version should be coming soon, as the respective maintainers (many thanks to Jack Moffit, Andrea Riciputi and Dryice Liu) have the time to follow their packaging procedures. Many thanks to Enthought for their continued hosting support for IPython, and to all the users who contributed ideas, fixes and reports. Release notes ------------- This release was made to fix a few crashes recently found by users, and also to keep compatibility with matplotlib, whose internal namespace structure was recently changed. * Adapt to matplotlib's new name convention, where the matlab-compatible module is called pylab instead of matlab. The change should be transparent to all users, so ipython 0.6.6 will work both with existing matplotlib versions (which use the matlab name) and the new versions (which will use pylab instead). * Don't crash if pylab users have a non-threaded pygtk and they attempt to use the GTK backends. Instead, print a decent error message and suggest a few alternatives. * Improved printing of docstrings for classes and instances. Now, class, constructor and instance-specific docstrings are properly distinguished and all printed. This should provide better functionality for matplotlib.pylab users, since matplotlib relies heavily on class/instance docstrings for end-user information. * New timing functionality added to %run. '%run -t prog' will time the execution of prog.py. Not as fancy as python's timeit.py, but quick and easy to use. You can optionally ask for multiple runs. * Improved (and faster) verbose exeptions, with proper reporting of dotted variable names (this had been broken since ipython's beginnings). * The IPython.genutils.timing() interface changed, now the repetition number is not a parameter anymore, fixed to 1 (the most common case). timings() remains unchanged for multiple repetitions. * Added ipalias() similar to ipmagic(), and simplified their interface. They now take a single string argument, identical to what you'd type at the ipython command line. These provide access to aliases and magics through a python function call, for use in nested python code (the special alias/magic syntax only works on single lines of input). * Fix an obscure crash with recursively embedded ipythons at the command line. * Other minor fixes and cleanups, both to code and documentation. The NEWS file can be found at http://ipython.scipy.org/NEWS, and the full ChangeLog at http://ipython.scipy.org/ChangeLog. Enjoy, and as usual please report any problems. Regards, Fernando. _______________________________________________ IPython-dev mailing list IPython-dev(a)scipy.net http://scipy.net/mailman/listinfo/ipython-dev

1 0

eigenvalue
by Reik H. Börger Dec. 8, 2004

Dec. 8, 2004

Hi Tiziano, I looked at the symeig-routines and I am surprised - they are working better than scipy's. I don't know, which algorithms are used, but it doesn't look like cutting off the imaginary parts. It is good enough for me :-) thanks alot Reik > You may want to have a look at: > http://mdp-toolkit.sourceforge.net/symeig.html > > The symeig module contains a Python wrapper for the LAPACK functions > to solve the standard and generalized eigenvalue problems for > symmetric (hermitian) positive definite matrices. Those specialized > algorithms give an important speed-up with repect to the generic > LAPACK eigenvalue problem solver used by SciPy (scipy.linalg.eig). > > hth, > tiziano

4 4

Sparse matrix formats
by Nils Wagner Dec. 8, 2004

Dec. 8, 2004

Hi all, Where can I find some useful information (tutorial) concerning the different sparse matrix formats used in scipy ? Any pointer would be appreciated. Nils

1 0

linalg.eigenvalues
by Reik H. Börger Dec. 5, 2004

Dec. 5, 2004

Hi, Tiziano, thanks for the link, I will check it soon. Robert, thanks for the link about floating-point arithmetics. What you say about 1e-15 being zero for practical purposes, I am aware of that. My problem is, that other routines in python rely on linalg.eig or similar procedures, for example in generating mutlivariate normal random variates. When I give my covariance matrix to the routine and it comes back to me, saying that my matrix is not positive definite, though I can prove it, then it is a problem. There are some easy ways to check, if a matrix is positive without using a eigenvalue decomposition. I think, this should be included in either the linalg.eig-routine or the stats.rv.multinormal-routine. You say, that Maple is basically rounding the imaginary part. Are you sure or do you guess? How does Maple know, that this matrix has no complex eigenvalues? Does it perform the checks mentioned above? Thanks, Reik > You may want to have a look at: > http://mdp-toolkit.sourceforge.net/symeig.html > > The symeig module contains a Python wrapper for the LAPACK functions > to solve the standard and generalized eigenvalue problems for > symmetric (hermitian) positive definite matrices. Those specialized > algorithms give an important speed-up with repect to the generic > LAPACK eigenvalue problem solver used by SciPy (scipy.linalg.eig). > > hth, > tiziano > > You are running into the problems of finite-precision floating point > arithmetic. Values around 1e-15 *are* zero for practical purposes. > Since > you know that the output values should be real (up to numerical > precision), you can just take the real part. That is more or less what > Maple is doing. > > I also recommend reading "What Every Computer Scientist Should Know > About Floating-Point Arithmetic": > > http://docs.sun.com/source/806-3568/ncg_goldberg.html > > -- > Robert Kern > rkern(a)ucsd.edu > > "In the fields of hell where the grass grows high > Are the graves of dreams allowed to die." > -- Richard Harter >

1 0

ANN: PyTables 0.9.1 released
by Francesc Altet Dec. 4, 2004

Dec. 4, 2004

Announcing PyTables 0.9.1 ------------------------- This release is mainly a maintenance version. In it, some bugs has been fixed and a few improvements has been made. One important thing is that chunk sizes in EArrays has been re-tuned to get much better performance and compression rations. Besides, it has been tested against the latest Python 2.4 and all test units seems to pass fine. What it is ---------- PyTables is a solid hierarchical database package designed to efficiently manage extremely large amounts of data (with support for full 64-bit file addressing). It features an object-oriented interface that, combined with C extensions for the performance-critical parts of the code, makes it a very easy-to-use tool for high performance data storage and retrieval. It is built on top of the HDF5 library and the numarray package, and provides containers for both heterogeneous data (Tables) and homogeneous data (Array, EArray) as well as containers for keeping lists of objects of variable length (like Unicode strings or general Python objects) in a very efficient way (VLArray). It also sports a series of filters allowing you to compress your data on-the-fly by using different compressors and compression enablers. But perhaps the more interesting features are its powerful browsing and searching capabilities that allow you to perform data selections over heterogeneous datasets exceeding gigabytes of data in just tenths of second. Besides, all the PyTables I/O is buffered, implemented in C and carefully tuned so that you can reach much better performance with PyTables than with your own home-grown wrappings to the HDF5 library. Changes more in depth --------------------- Improvements: - The chunksize computation for EArrays has been re-tuned to allow better performance and *much* better compression rations. - New --unpackshort and --quantize flags has been added to nctoh5 script. --unpackshort unpack short integer variables to float variables using scale_factor and add_offset netCDF variable attributes. --quantize quantize data to improve compression using least_significant_digit netCDF variable attribute (not active by default). See http://www.cdc.noaa.gov/cdc/conventions/cdc_netcdf_standard.shtml for further explanation of what this attribute means. Thanks to Jeff Whitaker for providing this. - Table.itersequence has received a new parameter called "sort". This allows to disable the sorting of the sequence in case the user wants so. Backward-incompatible changes: - Now, the AttributeSet class throw an AttributeError on __getattr__ for nonexistent attributes in it. Formerly, the routine returned None, which is pretty much against convention in Python and breaks the built-in hasattr() function. Thanks to Norbert Nemec for noting this and offering a patch. - VLArray.read() has changed its behaviour. Now, it always returns a list, as stated in documentation, even when the number of elements to return is 0 or 1. This is much more consistent when representing the actual number of elements on a certain VLArray row. API additions: - A Row.getTable() has been added. It is an accessor for the associated Table object. - A File.copyAttrs() has been added. It allows copying attributes from one leaf to other. Properly speaking, this was already there, but not documented :-/ Bug fixes: - Now, the copy of hierarchies works even when there are scalar Arrays (i.e. Arrays which shape is ()) on it. Thanks to Norbert Nemec for providing a patch. - Solved a memory leak regarding the Filters instance associated with the File object, that was not released after closing the file. Now, there are no known leaks on PyTables itself. - Fixed a bug in Table.append() when the table was indexed. The problem was that if table was in auto-indexing mode, some rows were lost in the indexation process and hence, not indexed correctly. - Improved security of nodes name checking. Closes #1074335 Important note for Python 2.4 and Windows users ----------------------------------------------- If you are willing to use PyTables with Python 2.4 in Windows platforms, you will need to get the HDF5 library compiled for MSVC 7.1, aka .NET (and possible LZO and UCL as well, if you want support for LZO and UCL at all). It can be found at: ftp://ftp.ncsa.uiuc.edu/HDF/HDF5/current/bin/windows/5-163-winxp-net2003.zip Where can PyTables be applied? ------------------------------ PyTables is not designed to work as a relational database competitor, but rather as a teammate. If you want to work with large datasets of multidimensional data (for example, for multidimensional analysis), or just provide a categorized structure for some portions of your cluttered RDBS, then give PyTables a try. It works well for storing data from data acquisition systems (DAS), simulation software, network data monitoring systems (for example, traffic measurements of IP packets on routers), very large XML files, or for creating a centralized repository for system logs, to name only a few possible uses. What is a table? ---------------- A table is defined as a collection of records whose values are stored in fixed-length fields. All records have the same structure and all values in each field have the same data type. The terms "fixed-length" and "strict data types" seem to be quite a strange requirement for a language like Python that supports dynamic data types, but they serve a useful function if the goal is to save very large quantities of data (such as is generated by many scientific applications, for example) in an efficient manner that reduces demand on CPU time and I/O resources. What is HDF5? ------------- For those people who know nothing about HDF5, it is a general purpose library and file format for storing scientific data made at NCSA. HDF5 can store two primary objects: datasets and groups. A dataset is essentially a multidimensional array of data elements, and a group is a structure for organizing objects in an HDF5 file. Using these two basic constructs, one can create and store almost any kind of scientific data structure, such as images, arrays of vectors, and structured and unstructured grids. You can also mix and match them in HDF5 files according to your needs. Platforms --------- I'm using Linux (Intel 32-bit) as the main development platform, but PyTables should be easy to compile/install on many other UNIX machines. This package has also passed all the tests on a UltraSparc platform with Solaris 7 and Solaris 8. It also compiles and passes all the tests on a SGI Origin2000 with MIPS R12000 processors, with the MIPSPro compiler and running IRIX 6.5. It also runs fine on Linux 64-bit platforms, like AMD Opteron running GNU/Linux 2.4.21 Server, Intel Itanium (IA64) running GNU/Linux 2.4.21 or PowerPC G5 with Linux 2.6.x in 64bit mode. It has also been tested in MacOSX platforms (10.2 but should also work on newer versions). Regarding Windows platforms, PyTables has been tested with Windows 2000 and Windows XP (using the Microsoft Visual C compiler), but it should also work with other flavors as well. Web site -------- Go to the PyTables web site for more details: http://pytables.sourceforge.net/ To know more about the company behind the PyTables development, see: http://www.carabos.com/ Share your experience --------------------- Let me know of any bugs, suggestions, gripes, kudos, etc. you may have. Enjoy data! -- Francesc Altet Who's your data daddy? PyTables

1 0

ANN: IPython 0.6.5 is out
by Fernando Perez Dec. 1, 2004

Dec. 1, 2004

Hi all, [sorry for those who'll get this twice, but I figure that enough scipy users are also ipython users that at this point the cross-post is forgiven] [Note better emacs support at the end, which I forgot in the ipython list msg] I'm glad to announce that IPython 0.6.5 is finally out. IPython's homepage is at: http://ipython.scipy.org and downloads are at: http://ipython.scipy.org/dist I've provided RPMs for Python 2.2 and 2.3, plus source downloads (.tar.gz and .zip). Debian, Fink and BSD packages for this version should be coming soon, as the respective maintainers (many thanks to Jack Moffit, Andrea Riciputi and Dryice Liu) have the time to follow their packaging procedures. Many thanks to Enthought for their continued hosting support for IPython, and to all the users who contributed ideas, fixes and reports. I'd promised that 0.6.4 would be the last version before the cleanup, but Prabhu Ramachandran managed to resucitate the GUI threading support which I'd worked on recently, but disabled after thinking it could not work. It turns out we were very close, and Prabhu did fix the remaining problems. Since this is a fairly significant improvement, I decided to make a release for it. In the process I added a few minor other things. *** WHAT is IPython? IPython tries to: 1. Provide an interactive shell superior to Python's default. IPython has many features for object introspection, system shell access, and its own special command system for adding functionality when working interactively. 2. Serve as an embeddable, ready to use interpreter for your own programs. IPython can be started with a single call from inside another program, providing access to the current namespace. 3. Offer a flexible framework which can be used as the base environment for other systems with Python as the underlying language. *** NEW for this release: As always, the complete NEWS file can be found at http://ipython.scipy.org/NEWS, and the whole ChangeLog at http://ipython.scipy.org/ChangeLog. * Threading support for WXPython and pyGTK. It is now possible (with the -wthread and -gthread options) to control wx/gtk graphical interfaces from within an interactive ipython shell. Note that your wx/gtk libs need to be compiled with threading support for this to work. There is also experimental (but brittle) support for ALSO running Tkinter graphical interfaces alongside with wx or gtk ones. * New -d option to %run, for executing whole scripts with the interactive pdb debugger. This allows you to step, watch variables, set breakpoints, etc, without having to modify your scripts in any way. * Added filtering support for variable types to %who and %whos. You can now say 'whos function str' and whos will only list functions and strings, instead of all variables. Useful when working with crowded namespaces. (For some reason I forgot to document this in the ChangeLog). * Added ipython.el to the end-user distribution, for (X)Emacs support, since now the official python-mode.el from http://sourceforge.net/projects/python-mode has all the necessary fixes for ipython to work correctly (in CVS at this moment, it will go into the next release I suppose). * Other minor fixes and cleanups, both to code and documentation. Enjoy, and as usual please report any problems. Regards, Fernando.

1 0