Mailman 3 April 2000 - NumPy-Discussion

Gnu Scientic Library
by Vanroose Wim 30 Apr '00

30 Apr '00

Dear Madam, Sir, I recently started to use the GSLibrary from http://soureware.cygnus.com/gsl/ The have a interesting collection of special functions. And a started to wrap several of them into my python programs. Does anybody has experiences with GSL?? Would n't it be beautiful to produce a python module based on the GSL special functions. Did somebody already do it??? Wim Vanroose

3 2

NumPy, Python DB-API and MySQL
by Tim Churches 29 Apr '00

29 Apr '00

I've been experimenting with pulling quantitative data out of a MySQL table into NumPy arrays via Andy Dustman's excellent MySQLdb module and then calculating various statistics from the data using Gary Strangman's excellent stats.py functions, which when operating on NumPy arrays are lightning-fast. The problem is the speed with which data can be extracted from a column of a MySQL (or any other SQL database) query result set and stuffed into a NumPy array. This inevitably involves forming a Python list and then assigning that to a NumPy array. This is both slow and memory-hungry, especially with large datsets (I have een playing with a few million rows). I was wondering if it would be feasible to initially add a method to the _mysql class in the MySQLdb module which iterated through a result set using a C routine (rather than a Python routine) and stuffed the data directly into a NumPy array (or arrays - one for each column in the result set) in one fell swoop (or even iterating row-by-row but in C)? I suspect that such a facility would be much faster than having to move the data into NumPy via a standard Python list (or actually via tuples within a list, which i sthe way the Python DB-API returns results). If this direct MySQL-to-NumPy interface worked well, it might be desirable to add it to the Python DB-API specification for optional implementation in the other database modules which conform to the API. There are probably other extensions which would make the DB-API more useful for statistical applications, which tend to be set (column)-oriented rather than row-oriented - will post to the list as these occur to me. Cheers, Tim Churches PS I will be away for the next week so apologies in advance for not replying immediately to any follow-ups to this posting. TC

2 4

ScientificPython 2.1.0 with MPI interface
by Konrad Hinsen 21 Apr '00

21 Apr '00

I have just put ScientificPython 2.0.1 and 2.1.0 on my FTP server, ftp://dirac.cnrs-orleans.fr/pub/ScientificPython/ while Starship is recovering. Version 2.0.1 is mostly a bugfix release, with only minor additions. 2.1.0 is identical to 2.0.1 except for the addition of an MPI interface module. I have tested this on only one platform (Linux/Intel running MPICH), so I would be very interested in feedback from people running different MPI platforms. MPI support in ScientificPython is still very basic; there are probably more complete MPI interfaces out there. The strong point of ScientificPython's interface is the integration into Python: communicators are Python objects, all communication happens via methods defined on communicator objects, support is provided for sending and receiving both string and NumPy array objects. Moreover, Python scripts can rather easily be written in such a way that they work both with and without MPI support, of course using only a single processor when no MPI is available. Finally, there is a full C API as well, which means that other C modules can make use of MPI without having to link to the MPI library, which is particularly useful for dynamic library modules. It also facilitates packaging of MPI-based code, which doesn't need to know anything at all about the MPI library. Happy Easter, Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen(a)cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.55.69 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais -------------------------------------------------------------------------------

1 0

Re: Welcome To "Numpy-discussion"!
by Pedro Vale Lima 19 Apr '00

19 Apr '00

Hello, Can someone give me a hand? I'm porting some code and I need to do QR decomposition. I couldn't find a such a function in Numpy. As I remember Lapack has one, isn't it part of the python interface? thanks pedro vale lima -- University of Coimbra, Portugal eq3pvl(a)eq.uc.pt

3 3

Numeric Python release?
by Joe Van Andel 18 Apr '00

18 Apr '00

sourceforge.net shows the latest release of Numeric Python as v 15.2, dated 1/19/2000 Could I encourage the Numeric Python developers to release a more recent version of Numeric Python? I know the latest is always available from CVS, but I'm sure that there are people who aren't ready to deal with CVS, just to get a current version of Numeric. -- Joe VanAndel National Center for Atmospheric Research http://www.atd.ucar.edu/~vanandel/ Internet: vanandel(a)ucar.edu

1 0

multiprocessor machine
by Jean-Bernard Addor 15 Apr '00

15 Apr '00

Hey Numpy people! I just put a second processor in my computer and it seems Numpy don't use it. Is Numpy able to use 2 processors? From wich version does't work? Jean-Bernard

4 3

compiling extensions with cygwin???
by Les Schaffer 13 Apr '00

13 Apr '00

i need to compile a simple Python extension, which uses NumPy/C API, on WinXX. does anyone have (bad/good) experience compiling NumPy extension with cygwin or mingw32 compilers? i am trying to decide whether i need to purchase VC++ or not. many thanks les schaffer

2 1

Request for Datasets
by Robert Kern 12 Apr '00

12 Apr '00

Hello, I'm working on a Multipack module to use ODRPACK for non-linear regression problems. ODRPACK is available from NETLIB if you want information: http://www.netlib.org/odrpack/index.html I'm in the debugging phase right now, and I want to be able to test my interfaces to most if not all of ODRPACK's capabilities. Consequently, I need datasets with some of the following properties: * multiple inputs (or a vector of inputs) * multiple responses (or a vector of responses) * errors/weights on either the responses, inputs, or both * covariance matrices for the errors on responses/inputs/both (in the case of multiple inputs/responses) * any differentiable functional form that I can make a Python function compute using Numpy and SpecialFuncs ufuncs (and maybe a few others) * problems where it is sensible to fix particular parameters and estimate the others * problems where it is sensible to fix particular datapoints (e.g. boundary conditions) * problems where some datapoints should be removed I would be much obliged if any of you could send me datasets that have some of these characteristics. I would prefer them to be in something parsable by Python, either a simple plaintext format or even NetCDF to goad me into learning how to use Konrad's NetCDF interface. A description of the function to fit to is necessary, and a brief description of the problem and perhaps even the expected answers would be nice. *** Please e-mail these to me and not the list. If you would like more information or clarification, please e-mail me. Many thanks for your time and possible contribution. -- Robert Kern kern(a)caltech.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter

1 0

Re: NumPy and None (null, NaN, missing)
by Martin Maechler 11 Apr '00

11 Apr '00

>>>>> "TimC" == gestalt-system-discuss-admin <gestalt-system-discuss-admin(a)lists.sourceforge.net> writes: TimC> Date: Sun, 09 Apr 2000 01:07:13 +1000 TimC> From: Tim Churches <tchur(a)bigpond.com> TimC> Organization: Gestalt Institute TimC> To: strang(a)nmr.mgh.harvard.edu, strang(a)bucky.nmr.mgh.harvard.edu, TimC> gestalt-system-discuss(a)lists.sourceforge.net, TimC> numpy-discussion(a)lists.sourceforge.net TimC> I'm a new user of MumPy so forgive me if this is a FAQ. ...... TimC> I've been experimenting with using Gary Strangman's excellent stats.py TimC> functions. The spped of these functions when operating on NumPy arrays TimC> and the ability of NumPy to swallow very large arrays is remarkable. TimC> However, one deficiency I have noticed is the lack of the ability TimC> to represent nulls (i.e. missing values, None or NaN TimC> [Not-a-Number] in NumPy arrays. Missing values commonly occur in TimC> real-life statistical data and although they are usually excluded TimC> from most statistical calculations, it is important to be able to TimC> keep track of the number of missing data elements and report TimC> this. I'm just a recent "listener" on gestalt-system-discuss, and don't even have any python experience. I'm member of the R core team (www.r-project.org). In R (and even in S-plus, but almost invisibly there), we even do differentiate between "NA" (missing / not available) and "NaN" (IEEE result of 0/0, etc). I'd very much like to have these different as in R. I think our implementation of these is quite efficient, implementing NA as one particular bit pattern from the whole possible NaN set. We use code like the following (R source, src/main/arithmetic.c ) : static double R_ValueOfNA(void) { ieee_double x; x.word[hw] = 0x7ff00000; x.word[lw] = 1954; return x.value; } int R_IsNA(double x) { if (isnan(x)) { ieee_double y; y.value = x; return (y.word[lw] == 1954); } return 0; } Martin Maechler <maechler(a)stat.math.ethz.ch> http://stat.ethz.ch/~maechler/ TimC> Because NumPy arrays can't represent missing data via a TimC> special value, it is necessary to exclude missing data elements TimC> from NumPy arrays and keep track of them elsewhere (in standard TimC> Python lists). This is messy. Also, it is quite common to use TimC> various imputation techniques to estimate the values of missing TimC> data elements - the ability to represent missing data in a NumPy TimC> array and then change it to an imputed value would be a real TimC> boon.

2 1

__version__ hack in Matrix.py is busted
by Charles G Waldman 09 Apr '00

09 Apr '00

I can't import "Matrix" due to the following cruft: __id__ = """ $Id: Matrix.py,v 1.1.1.1 2000/01/13 21:23:06 dubois Exp $ """[1:-1] import string __version__ = int(__id__[string.index(__id__, '#')+1:-1]) You can't count on the CVS ID having a "#" character in it; each time the file is checked in and out of CVS the Id is rewritten. I don't think this trick can be made to work. I think what is needed is either a simpler more failsafe way of setting __version__, or simply to eliminate __version__ altogether.

1 0