From hinsen@cnrs-orleans.fr Tue Dec 1 19:02:52 1998
From: hinsen@cnrs-orleans.fr (Konrad Hinsen)
Date: Tue, 1 Dec 1998 20:02:52 +0100
Subject: [Matrix-SIG] re: advocacy
In-Reply-To: <199811302203.RAA23085@eclipse.stsci.edu> (perry@stsci.edu)
References: <199811302203.RAA23085@eclipse.stsci.edu>
Message-ID: <199812011902.UAA20320@dirac.cnrs-orleans.fr>
> a lot of users off. Emacs users (and I am one) around here are a
> distinct minority; we simply cannot tell people that they should use it
> if we want them to use our software.
What's the problem? If they are willing to use some new IDE, why shouldn't
they use Emacs? Just rename emacs to "pythonIDE" ;-)
> > Just out of curiosity - could someone give a short explanation
> > for non-astronomers what FITS really is?
> >
> FITS (Flexible Image Transport System) was a format originally designed
...
Thanks for the explanation!
Konrad.
--
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsen@cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.55.69
Rue Charles Sadron | Fax: +33-2.38.63.15.17
45071 Orleans Cedex 2 | Deutsch/Esperanto/English/
France | Nederlands/Francais
-------------------------------------------------------------------------------
From hauser@lisboa.ifm.uni-kiel.de Tue Dec 1 20:55:50 1998
From: hauser@lisboa.ifm.uni-kiel.de (Janko Hauser)
Date: Tue, 1 Dec 1998 21:55:50 +0100 (CET)
Subject: [Matrix-SIG] Interactive Shell (was re: advocacy)
In-Reply-To: <199811252250.RAA15587@eclipse.stsci.edu>
References: <199811252250.RAA15587@eclipse.stsci.edu>
Message-ID: <13924.22426.660243.174690@lisboa.ifm.uni-kiel.de>
Actually for personal use I have build an interactiv shell, which does
some of these things. If I think about the missing parts I see that it
needs a rewrite, but it is possible. Here an example of a
session.
@porto >psh
Python 1.5.2a2 (#2, Oct 26 1998, 22:21:28) [GCC 2.7.2.3] on linux2
Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
(interactiv_shell)
>>> len(dir()); dir()[:10]
276
['ArrayType', 'BuiltinFunctionType', 'BuiltinMethodType', 'ClassType',
'CodeType', 'Complex', 'Complex0', 'Complex16', 'Complex32',
'Complex64']
>>> who() # no locally defined variable
>>> a=a # here I hit TAB for expansion in local namespace
abs arange argsort arrayrange
absolute arccos array arraytype
add arcsin array2string asarray
alltrue arctan array_constructor assert
and arctan2 array_repr
animate argmax array_set
apply argmin array_str
>>> a=array([1,2,3])
>>> ?take
** doc for take **
take(a, indices, axis=0). Selects the elements in indices from array
a along the given axis.
>>> !ls
many_nice_files.py
>>> shell.log='log_of_session.log' # I start logging input
>>> a=span(-pi,pi,200)
>>> b=cos(a)
>>> !cat log_of_session.log
a=span(-pi,pi,200)
b=cos(a)
>>> shell.log='' # I end logging of input
>>> b[:5]
array([-1. , -0.99950159, -0.99800685, -0.99551728,
-0.99203536])
>>> !cat log_of_session.log
a=span(-pi,pi,200)
b=cos(a)
shell.log='' # this should not be there
>>> who() # look for local variables
a <type 'array'>
b <type 'array'>
>>>Ctrl-D # currently no quit but that's easy :-)
Together with commandline history from the readlinemodule this is
quite nice, but not perfect. No output logging for example.
But one should think about some design decisions. I also think emacs
is not a general answer. A GUI is not easily portable and the mainloop
interferes with most current plotting packages. I like the cmd
interface, which is the same everywhere. One can for some things popup
a window, like a variable browser, or a help screen. I think of this
syntax
! -> shell escape also a way to edit some methods
? -> help for object needs to look for different sources
@ -> interpreter relevant commands, possibly without parameters (no
braces)
For me it's important that all the logged lines are valid python, so
that one can right start to edit them, to have a script. So the output
if captured needs to go elsewhere. Also the interpreter relevant
commands are escaped by the @. So it should be @who. Some funtions are
needed for simple path, file manipulation and to change the current
directory. I don't want to introduce to much new commands with lots of
magic, the hole system should be close to Python scripts. If one wants
to use python she needs to learn it to some degree. There is a need
for a special import, so that the new objects are not visible by
who(). So in the end, there is work to be done, but it's also possible
to be more comfortable right now.
In the end I want to say, that I'm in favour for a scientific
workbench.
__Janko
Perry Greenfield writes:
> To which I would add:
>
> 6) Weak interactive environment: It is not always appreciated by
> non-scientists that interactive mode is where most time is spent
> looking at data. To that end, the interactive mode needs some conveniences
> that are currently lacking. Such as:
> - command (and output) logging.
> - easy system escapes (e.g., !ls in IRAF, $ls in IDL)
> - as trivial as it sounds, an intuitive exit command (like 'exit' or 'quit')
> - an easy way of re-executing moderately long command sequences or blocks
> without using an editor (e.g., you invoke a loop of several statements
> repeatedly because you need to tweak some parameters each time).
> - utility functions to start up help, or examine the namespace for
> existing data elements (much like IDL does)
> To these one can raise one more issue. I sense that Python gurus consider
> IDL users to be programming adverse, but in the spectrum of astronomical
> users, those that use IDL tend to be the ones that are most interested
> in programming. We (STScI) also must concern ourselves with the other
> half (or whatever) that find even IDL too intimidating and are much more
> comfortable with the more shell-like environment of IRAF. There are a
> lot of users that would not want to have to quote all strings or put
> parenthesis around all argument lists. This doesn't have a simple
> solution but I'm pretty sure that ignoring this issue will preclude
> use of Python by many astronomers (I won't speak for other fields).
> 7) Image display convenient for astronomers (either by saoimage or ximtool
> lookalikes or connections to either of those so that images could be
> displayed on those tools.
>
> After such a list some might wonder why we even consider using Python at
> all. Essentially the existing situation has even more serious drawbacks
> that we are trying to escape.
>
> Should our initial efforts succeed then we need solutions to the above
> issues. We make our software available to a wide community for a number
> of different platforms so it is simply not a matter of solving these
> problems for our own organization. If what we deliver is too hard to
> install or is to hard to use or understand in the wider astronomical
> community it will be a failure (I'm sure some would argue that existing
> IRAF-based software falls into that category). So it is essential, for
> example, that there is a standard distribution and installation mechanism.
> Fortunately, there appears to be serious interest in solving this for
> the whole Python community. For our initial plans, these issues do not
> all have to be solved immediately, but we have to see a way to deal with
> them eventually.
>
> Hopefully the FITS issue will be solved soon--I'm eagerly looking forward to
> Paul's implementation. (And make that ~5 FITS implementations; I did a
> primitive one for some test programs we were doing here). There is reason
> to believe that documentation will improve, and we can augment it where
> necessary. Numerical utilities can be developed as needed. It doesn't
> seem that one giant effort is needed there.
>
> The plotting situation seems to be somewhat muddled and one of my more
> serious concerns. I'd rather not spend the effort to develop a package
> from scratch. There a number of graphics packages out there but each
> has a shortcoming or two that makes us reluctant to use it. Our need
> is primarily for simple 2-d plotting tasks (x-y, contour, surface, etc.)
> but that has enough control to produce publication quality output.
> While we don't currently support Windows platforms, we don't want to
> rule that out. Nor can we distribute commercial software. We would
> prefer it to have an object oriented interface as well. This appears
> to rule out most of what is out there. PyGIST is not available on
> Windows, DISTLIN is commercial, GTK appears only to provide plotting
> primitives and doesn't run on Windows, VTK is volume rendering based
> and has little for 2-d plotting...about the only thing available now
> appears to be pgplot.) For the moment, I'm waiting to see what David
> Ascher produces; it sounds promising. Complicating our decisions is
> trying to decide how Java and JPython will fit into our future. We prefer
> a solution that will run in JPython as well (It almost sounds like we
> constraining ourselves out of a solution!)
>
> We are working on adding some conveniences to the interpreter by making
> our own wrapper. I'm not Guido's new IDLE environment will do it for us.
> I haven't had a chance to take a look at it yet.
>
> If we do go with Python it does mean that there will be some institutional
> support to help supply what is missing in the above areas.
>
> Perry Greenfield
> Space Telescope Science Institute
> Science Software Group
>
From jh@oobleck.tn.cornell.edu Thu Dec 3 19:29:43 1998
From: jh@oobleck.tn.cornell.edu (Joe Harrington)
Date: Thu, 3 Dec 1998 14:29:43 -0500
Subject: [Matrix-SIG] Re: advocacy
Message-ID: <199812031929.OAA11736@oobleck.tn.cornell.edu>
A week ago I posted a message asking who would be interested in
turning NumPy into a real data analysis package anyone could use. I
received 15 responses, many offering substantial time. Although I
mentionned a target of 20, this is a substantial number, and I think
it might grow once we got going. I'm off to Chile for some fieldwork
(site survey for a new 13-meter infrared telescope at 17000 feet!),
but I'll be back mid-month. For those who haven't seen it, Paul
Barrett and I wrote a summary of the session on data analysis
environments we held at ADASS VI. It includes some of our vision for
an ideal environment. It includes a separation of interpreter and
application code, so that the next generation of interpreter (a
fast-developing item) can use the last generation of application code
(once an FFT, always an FFT, for the most part). See:
http://oobleck.tn.cornell.edu/jh/ast.html
http://oobleck.tn.cornell.edu/jh/ast/papers/idae96.ps
Back in a few weeks!
--jh--
From cgw@pgt.com Sun Dec 6 02:04:17 1998
From: cgw@pgt.com (Charles G Waldman)
Date: Sat, 5 Dec 1998 21:04:17 -0500 (EST)
Subject: [Matrix-SIG] NumPy core dump
Message-ID: <13929.58913.15081.462442@sirius>
Using Numeric version 1.5, from the LLNLPython6 distribution, on top
of Python1.5.2b1, the following piece of code causes a core-dump:
from Numeric import *
AM = fromfunction(lambda x:sin(x*pi/512))
I know the code is incorrect (missing second arg. to fromfunction) but
shouldn't this raise an exception rather than dumping core?
From kernr@ncifcrf.gov Sun Dec 6 02:07:40 1998
From: kernr@ncifcrf.gov (Robert Kern)
Date: Sat, 05 Dec 1998 21:07:40 -0500
Subject: [Matrix-SIG] Eigenvalues return array
Message-ID: <3669E6EC.806D663C@mail.ncifcrf.gov>
This code on my Windows95 machine running Python 1.5.1 and Numerical
Python 1.6:
%---------------------------------------------------------------------
# eigenvalues return array test case
import Numeric
import LinearAlgebra
a = Numeric.array([[0.4, 0.7],
[0.6, 0.3]])
eigenvals = LinearAlgebra.eigenvalues(a)
print eigenvals # This prints 'array([ 1. , -0.3])'
print Numeric.equal(eigenvals, 1.0) # This prints 'array([0, 0])'
%---------------------------------------------------------------------
doesn't seem to be give the right results with the 'equals' comparison.
Note that 'Numeric.equal(Numeric.array([1., -0.3]), 1.)' returns the
correct 'array([1, 0])'. Is this a fixable bug?
--
Robert Kern |
----------------------|"In the fields of Hell where the grass grows high
This space | Are the graves of dreams allowed to die."
intentionally | - Richard Harter
left blank. |
From strang@nmr.mgh.harvard.edu Sun Dec 6 21:13:14 1998
From: strang@nmr.mgh.harvard.edu (Gary Strangman)
Date: Sun, 06 Dec 1998 16:13:14 -0500
Subject: [Matrix-SIG] Statistics functions for python
Message-ID: <366AF36A.539728BF@nmr.mgh.harvard.edu>
All,
I would like to announce the availability of my stats.py module.
The module computes various basic statistical functions (mean,
standard error, etc.) as well as some not-so-basic ones (non-
parametric t-tests, various correlations, within/between-subject
ANOVAs, etc.). One set of functions has been written for use
with list/tuple arguments while a second set has been written
for NumPy array arguments. (The latter set of functions are (of
course) faster, and usually offer more argument options.) The
module can be accessed at:
http://www.nmr.mgh.harvard.edu/Neural_Systems_Group/gary/python.html
Note that the module also requires my pstat.py module, which is
available at the same web address. Any additions to the collection
(or bug reports) are most welcome.
Happy stats-ing!
Gary
-----------------------------------------------------------------
Gary Strangman, PhD | Neural Systems Group
Office: 617-724-0662 | Massachusetts General Hospital
strang at nmr.mgh.harvard.edu | 13th Street, Bldg 149, Room 9103
| Charlestown, MA 02129
http://www.nmr.mgh.harvard.edu/Neural_Systems_Group/gary/
From hinsen@cnrs-orleans.fr Mon Dec 7 18:07:05 1998
From: hinsen@cnrs-orleans.fr (Konrad Hinsen)
Date: Mon, 7 Dec 1998 19:07:05 +0100
Subject: [Matrix-SIG] Eigenvalues return array
In-Reply-To: <3669E6EC.806D663C@mail.ncifcrf.gov> (message from Robert Kern on
Sat, 05 Dec 1998 21:07:40 -0500)
References: <3669E6EC.806D663C@mail.ncifcrf.gov>
Message-ID: <199812071807.TAA22672@dirac.cnrs-orleans.fr>
> eigenvals = LinearAlgebra.eigenvalues(a)
> print eigenvals # This prints 'array([ 1. , -0.3])'
>
> print Numeric.equal(eigenvals, 1.0) # This prints 'array([0, 0])'
That is probably an accuracy problem. Try
print eigenvals - 1.
I bet it won't print the first element as zero! In other words, the
first eigenvalue is one up to the precision used for printing, but not
exactly.
In practice, it is rarely useful in numerical algorithms to test floating
point numbers for equality. If you want to test for "very close to one",
you should use
Numeric.less(Numeric.fabs(eigenvals-1.), 1.e-10)
or whatever precision you want.
--
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsen@cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.55.69
Rue Charles Sadron | Fax: +33-2.38.63.15.17
45071 Orleans Cedex 2 | Deutsch/Esperanto/English/
France | Nederlands/Francais
-------------------------------------------------------------------------------
From phil@geog.ubc.ca Wed Dec 9 18:24:32 1998
From: phil@geog.ubc.ca (Phil Austin)
Date: 09 Dec 1998 10:24:32 -0800
Subject: [Matrix-SIG] Re: blitz++ vs. NumPy?
In-Reply-To: henk13@my-dejanews.com's message of Wed, 09 Dec 1998 15:24:48 GMT
References: <74m4ns$6ht$1@nnrp1.dejanews.com>
Message-ID: <m3hfv541i7.fsf@brant.geog.ubc.ca>
The following message is a courtesy copy of an article
that has been posted as well.
posting to comp.lang.python, henk13@my-dejanews.com writes:
>
> I just wondered what the advantages and disadvantages would be of doing
> Python numerical computiation in a Blitz++ (http://seurat.uwaterloo.ca/blitz)
> framework rather than NumPy's Lapack/Blas framework (this is _not_ meant to
> offend the people at LLNL). The inherint OO-design of Blitz nicely extends
> the mental concept in which Python programs are developed. For instance, the
> Array class could be straightforwardly mapped to a Python Array extension
> type.
>
> Is this some utopia or does it make sense?
>
> Comments are welcome!
>
Yes, the fit you describe is very close to completion thanks to Paul
Dubois,Todd Veldhuizen, Geoff Furnish and others. Here's an example
of a Python extension I wrote in about 20 minutes last night, which
calculates the 2nd order structure function for a 256 x 256 pixel
satellite image of cloud liquid water path. It uses Paul's CXX_Array.h
(see the source, http://xfiles.llnl.gov/CXX_Objects/cxx.htm and
particularly the last pages of
http://starship.skyport.net/~da/freebie). I've added a constructor
that creates an n-dimensional PyArray from dimensions specified by a
Blitz 1-d array.
Blitz arrays are reference-counted, and Todd's provided a constructor
that takes a raw C pointer (in this case to the PyArray data).
I can mirror the PyArrays with Blitz arrays of the
same dimensions, denoted by the "tied" prefix below and get the
template metaprograms, 2-dimensional subscripts etc., without paying
for a copy. I return the structure function and the pixel counts to
Python in a dictionary. The program strucfun.cxx below is compiled into
strucfun.so and then called from Python like this:
from Numeric import *
import strucfun
test=arange(4,typecode=Float32)
test.shape=(2,2)
print strucfun.strucout(test)
Some comments/questions:
1) From my perspective the combination of Blitz and CXX_Objects pushes
NumPy across a threshold: extension writing has become something that
Fortran77 and Matlab programming scientists (like me) can do.
2) Two things I wasn't able to figure out in 20 minutes are:
a) How to add my Blitz-based Py::Array constructor to Paul's code
using inheritence. It looks like:
explicit Array (blitz::Array<int,1> dimens, PyArray_TYPES t = PyArray_DOUBLE)
: Sequence(FromAPI(PyArray_FromDims(dimens.size(), dimens.data(), t))) {
validate();
}
b) Is there a general way to convert between a Blitz TinyVector of
length N (where N isn't known at compile time)
and a blitz::Array<int,1>? I'd like to avoid specifying the
dimensions to the Py::Array and the blitz::Array separately.
3) This worked the first time I tried it; everyone responsible for that
little miracle has my heartfelt thanks.
__________strucfun.cxx_____________
#include "Python.h"
#include "CXX_Objects.h"
#include "CXX_Extensions.h"
#include <blitz/array.h>
#include "CXX_Array.h"
#include <math.h>
USING(namespace Py)
USING(namespace std)
blitz::Array<int,1> convertit( blitz::TinyVector<int,2> tinyDimens)
{
blitz::Array<int,1> junk(2);
for(int j=0;j < tinyDimens.length();++j){
junk(j)=tinyDimens(j);
}
return junk;
}
static PyObject *
ex_strucout(PyObject* self, PyObject* args)
{
Py_Initialize();
import_array();
PyImport_ImportModule("Numeric");
Tuple t(args);
Array tau(t[0]);
//
// change these to exceptions later
//
assert(tau.rank()==2);
assert(tau.dimension(1)==tau.dimension(2));
assert(tau.species()==6);
int N=tau.dimension(1);
int Nbig=2*N-1;
blitz::TinyVector<int,2> tinyDimens(Nbig,Nbig);
Array sf(convertit(tinyDimens),PyArray_FLOAT);
Array count(convertit(tinyDimens),PyArray_FLOAT);
// make blitz arrays that point to the same data and have the
// same shape
blitz::Array<float,2> tiedTau((float*) sf.to_C(),tinyDimens);
blitz::Array<float,2> tiedSf((float*) sf.to_C(),tinyDimens);
blitz::Array<float,2> tiedCount((float*) count.to_C(),tinyDimens);
tiedSf=0.;
tiedCount=0.;
for(int i=0;i<N;i++){
for(int j=0;j<N;j++){
if(!isnan(tiedTau(i,j)))
{
for(int k=0;k<N;k++){
for(int l=0;l<N;l++){
if(!isnan(tiedTau(k,l)))
{
tiedSf(k-i+N,l-j+N)=tiedSf(k-i+N,l-j+N) + pow((tiedTau(k,l) - tiedTau(i,j)),2);
tiedCount(k-i+N,l-j+N)=tiedCount(k-i+N,l-j+N) + 1.;
}
}
}
}
}
}
for (int i=0; i<Nbig; i++) {
for (int j=0; j<Nbig; j++) {
if(tiedCount(i,j) > 0.){
tiedSf(i,j)=tiedSf(i,j)/tiedCount(i,j);
}
}
}
Dict output;
output["count"]=count;
output["struct"]=sf;
return new_reference_to(output);
}
extern "C" void initstrucfun();
static ExtensionModule* strucfun;
void initstrucfun()
{
// experimental initialization stuff
strucfun = new ExtensionModule("strucfun");
strucfun->add("strucout", ex_strucout, "calculate the structure function");
}
--
------------------------------------------------------------
Phil Austin (phil@geog.ubc.ca)
Department of Geography, Tel: (604) 822-2663
University of British Columbia, B.C. Fax: (604) 822-6150
From tbyang@llnl.gov Fri Dec 11 18:11:27 1998
From: tbyang@llnl.gov (T.-Y. Brian Yang)
Date: Fri, 11 Dec 1998 10:11:27 -0800 (PST)
Subject: [Matrix-SIG] Re: blitz++ vs. NumPy?
In-Reply-To: <Pine.SOL.3.93.981211095400.5779A-100000@gandalf>
Message-ID: <Pine.SOL.3.93.981211095802.5779C-100000@gandalf>
On Wed, 9 Dec 1998, "Paul F. Dubois" wrote:
> Thinking more outside the box, and perhaps closer to what the original
> poster may have intended, is the question, could/should NumPy be rewritten
> to use Blitz in its implementation, or should a similar facility be
> written
> that wraps Blitz. I think those questions are worthy of study: clearly one
> can do it, the question is, is it a good idea?
>
> I'll look into it.
>
>
It will be worthwhile only if the following Python expression does
not generate temporary arrays.
>>> a = a + b + c
But this requires changes to the Python interpreter. Short of that, the
following will be fine.
>>> Numeric.expression ("a = a + b + c")
a little parser in the module turn the expression into a C++
call that uses expression template.
Brian Yang
From da@skivs.ski.org Fri Dec 11 18:56:41 1998
From: da@skivs.ski.org (David Ascher)
Date: Fri, 11 Dec 1998 10:56:41 -0800 (Pacific Standard Time)
Subject: [Matrix-SIG] Re: blitz++ vs. NumPy?
In-Reply-To: <Pine.SOL.3.93.981211095802.5779C-100000@gandalf>
Message-ID: <Pine.WNT.4.04.9812111048030.217-100000@rigoletto.ski.org>
> It will be worthwhile only if the following Python expression does
> not generate temporary arrays.
>
> >>> a = a + b + c
>
> But this requires changes to the Python interpreter.
I'm not sure that's true. the __add__ operation could return 'lazy'
wrapper objects around the operation bound w/ references to the operands,
which would get evaluated when the contents of the array are accessed.
Similarly, wrappers around compound operands could be smart and avoid the
need for temporary arrays.
Non-trivial, as math teachers are fond of saying, and sweeping, but
something to think about in a redesign.
--david
PS: For an even more radical approach, see the papers by
Kiczales et al. on open implementation and aspect-oriented
programming, e.g.
http://www.parc.xerox.com/spl/groups/eca/pubs/papers/Kiczales-ECOOP97/for-web.pdf
and the larger collection at
http://www.parc.xerox.com/spl/groups/eca/pubs/
From just@letterror.com Mon Dec 14 12:57:51 1998
From: just@letterror.com (Just van Rossum)
Date: Mon, 14 Dec 1998 13:57:51 +0100
Subject: [Matrix-SIG] NumPy inconsistency?
Message-ID: <l03102802b29aba40a334@[193.78.237.131]>
Is there a reason why adding a 2-tuple to 2d array is allowed but
multiplying a 2d array by a 2-tuple isn't? See here:
>>> from Numeric import *
>>> a = arange(6)
>>> a.shape = -1,2
>>> a
array([[0, 1],
[2, 3],
[4, 5]])
>>> a + (1.5, 2.0)
array([[ 1.5, 3. ],
[ 3.5, 5. ],
[ 5.5, 7. ]])
>>> a * (1.5, 2.0)
Traceback (innermost last):
File "<input>", line 1, in ?
TypeError: can't multiply sequence with non-int
>>>
Just
From jac@lanl.gov Mon Dec 14 17:09:20 1998
From: jac@lanl.gov (James A. Crotinger)
Date: Mon, 14 Dec 1998 10:09:20 -0700
Subject: [Matrix-SIG] Re: blitz++ vs. NumPy?
In-Reply-To: <Pine.SOL.3.93.981211095802.5779C-100000@gandalf>
References: <Pine.SOL.3.93.981211095400.5779A-100000@gandalf>
Message-ID: <3.0.3.32.19981214100920.00a7c28c@cic-mail.lanl.gov>
At 10:11 AM 12/11/98 -0800, T.-Y. Brian Yang wrote:
>On Wed, 9 Dec 1998, "Paul F. Dubois" wrote:
>
>> Thinking more outside the box, and perhaps closer to what the original
>> poster may have intended, is the question, could/should NumPy be rewritten
>> to use Blitz in its implementation, or should a similar facility be
>> written
>> that wraps Blitz. I think those questions are worthy of study: clearly one
>> can do it, the question is, is it a good idea?
>>
>> I'll look into it.
I'm not exactly sure what people are after here, but the true value of
Blitz and of the Pooma II Array class (http://www.acl.lanl.gov/pooma) is
that they do expression templates in order to avoid temporaries and binary
evaluation of expressions. There is no way to take direct advantage of
these from Python without doing C++ runtime code generation. One could, on
the other hand, try to do something like expression templates using Python
directly, overloading the operators to build a runtime parse tree for the
expression and then having the assignment operator perform the actual
calculation. Don't know how the overhead for doing this at runtime would
stack up compared to the overhead of direct binary evaluation.
On the other hand, it is a useful goal to be able to interact with C++
codes that use Blitz (and Pooma II) arrays. If that is what you are after,
then I agree that this is a good idea. The SILOON project at LANL is
developing a tool that should be able to do this in a fairly automatic
manner. I hope to spend some time with them in January looking at how this
works with codes that use Pooma II arrays. They have a web site
(http://www.acl.lanl.gov/siloon) that has their first release (the DOWNLOAD
and ABOUT buttons are the only things on the web site that actually do
anything). Don't know if it is ready for prime time or not yet.
Jim
------------------------------------------------------------------------
James A. Crotinger Los Alamos National Lab email: jac@lanl.gov
Technical Staff Member CIC-ACL, MS B287 phone: 505-665-6022
Pooma Team Los Alamos, NM 87545 fax: 505-665-4939
From hinsen@cnrs-orleans.fr Mon Dec 14 17:40:59 1998
From: hinsen@cnrs-orleans.fr (Konrad Hinsen)
Date: Mon, 14 Dec 1998 18:40:59 +0100
Subject: [Matrix-SIG] NumPy inconsistency?
In-Reply-To: <l03102802b29aba40a334@[193.78.237.131]> (message from Just van
Rossum on Mon, 14 Dec 1998 13:57:51 +0100)
References: <l03102802b29aba40a334@[193.78.237.131]>
Message-ID: <199812141740.SAA24782@dirac.cnrs-orleans.fr>
> Is there a reason why adding a 2-tuple to 2d array is allowed but
> multiplying a 2d array by a 2-tuple isn't? See here:
It seems to me that neither "should" work, i.e. is documented to
work, but addition can be made to work (and it works), whereas
multiplication can't.
The interpreter decides that the operations to be applied are
sequence addition and sequence multiplication. Sequence addition
is unrestricted and the implementation in NumPy accepts any
kind of sequence as argument. But sequence multiplication is
restricted to integer prefactors, which is why you get an
error message.
So in the end the confusion is a consequence of the double status
of arrays as both a number-like type and a sequence type.
Konrad.
--
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsen@cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.55.69
Rue Charles Sadron | Fax: +33-2.38.63.15.17
45071 Orleans Cedex 2 | Deutsch/Esperanto/English/
France | Nederlands/Francais
-------------------------------------------------------------------------------
From Oliphant.Travis@mayo.edu Sun Dec 13 23:41:19 1998
From: Oliphant.Travis@mayo.edu (Travis E. Oliphant)
Date: Sun, 13 Dec 1998 17:41:19 -0600
Subject: [Matrix-SIG] Python crash when sort is given zero length list
References: <199812141700.MAA17832@python.org>
Message-ID: <3674509F.26638416@mayo.edu>
I'm running NumPy 1.3 on Linux libc6 and get a floating
point exception when I try to sort a list with zero elements.
Here is the session.
Anybody seen this and made the fix.
>>> from Numeric import sort
>>> sort([])
Floating point exception
----------------------------------------------------
Travis Oliphant 200 First St SW
Rochester MN 55905
Ultrasound Research Lab (507) 286-5923
Mayo Graduate School Oliphant.Travis@mayo.edu
From dubois1@llnl.gov Tue Dec 15 15:32:50 1998
From: dubois1@llnl.gov (Paul F. Dubois)
Date: Tue, 15 Dec 1998 07:32:50 -0800
Subject: [Matrix-SIG] Re: blitz++ vs. NumPy?
Message-ID: <008201be2840$2e676aa0$f4160218@c1004579-c.plstn1.sfba.home.com>
Several correspondants have opined that wrapping Blitz is of value only if
its performance characteristics could be exploited. I respectfully disagree.
Assuming Blitz became a widely-used C++ numerics vehicle, having a wrapped
version for Python would enable easy and efficient wrapping of whatever
numerical packages appear that make use of it. And, FWIW, eliminate a lot of
code we have to maintain/debug. It is easier to wrap than create, ne? And
NumPy will have to be rewritten, and soon, if it is to be maintainable.
From hinsen@cnrs-orleans.fr Tue Dec 15 16:13:25 1998
From: hinsen@cnrs-orleans.fr (Konrad Hinsen)
Date: Tue, 15 Dec 1998 17:13:25 +0100
Subject: [Matrix-SIG] Re: blitz++ vs. NumPy?
In-Reply-To: <008201be2840$2e676aa0$f4160218@c1004579-c.plstn1.sfba.home.com>
(dubois1@llnl.gov)
References: <008201be2840$2e676aa0$f4160218@c1004579-c.plstn1.sfba.home.com>
Message-ID: <199812151613.RAA26380@dirac.cnrs-orleans.fr>
> numerical packages appear that make use of it. And, FWIW, eliminate a lot of
> code we have to maintain/debug. It is easier to wrap than create, ne? And
> NumPy will have to be rewritten, and soon, if it is to be maintainable.
How much of the current NumPy code could be eliminated if NumPy were
to use Blitz as much as possible? The Python interface to arrays would
have to remain, I suppose, as well as the ufunc module. That's the
majority of the core code; maybe the indexing stuff could be simpler,
perhaps even a lot. The additional modules (LinearAlgebra, FFT, etc.)
are already small wrappers around existing libraries.
And what would be the consequences of a change to Blitz for C
extensions that use arrays? Would it still be possible to interface
with code using traditional C/Fortran style arrays? Would it be
possible to do so without knowing C++? What would happen to people
whose C++ compiler is not (yet?) suitable for Blitz? According to
their Web page, that's still the majority of platforms.
I don't want to leave the impression of wanting to block progress, but
I write and distribute code that is supposed to be used and modified
by today's real-life scientists. I can't expect them to get a
particular compiler (or perhaps even a new machine) to use my code,
and I can't expect them to learn C++.
--
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsen@cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.55.69
Rue Charles Sadron | Fax: +33-2.38.63.15.17
45071 Orleans Cedex 2 | Deutsch/Esperanto/English/
France | Nederlands/Francais
-------------------------------------------------------------------------------
From cgw@pgt.com Tue Dec 15 16:20:34 1998
From: cgw@pgt.com (Charles G Waldman)
Date: Tue, 15 Dec 1998 11:20:34 -0500 (EST)
Subject: [Matrix-SIG] Python crash when sort is given zero length list
In-Reply-To: <3674509F.26638416@mayo.edu>
References: <199812141700.MAA17832@python.org>
<3674509F.26638416@mayo.edu>
Message-ID: <13942.35922.881228.950845@janus.pgt.com>
Travis E. Oliphant writes:
> >>> from Numeric import sort
> >>> sort([])
> Floating point exception
This was quite easy to fix. Here's the (one-line) patch.
--- multiarraymodule.c.orig Thu Aug 20 14:59:22 1998
+++ multiarraymodule.c Tue Dec 15 11:17:11 1998
@@ -390,6 +390,7 @@
elsize = ap->descr->elsize;
m = ap->dimensions[ap->nd-1];
+ if (m==0) goto fail;
n = PyArray_SIZE(ap)/m;
for (ip = ap->data, i=0; i<n; i++, ip+=elsize*m) {
qsort(ip, m, elsize, compare_func);
From HYoon@exchange.ml.com Tue Dec 15 16:25:34 1998
From: HYoon@exchange.ml.com (Yoon, Hoon (CICG - NY Program Trading))
Date: Tue, 15 Dec 1998 11:25:34 -0500
Subject: [Matrix-SIG] Re: blitz++ vs. NumPy?
Message-ID: <A1AFC4253BA3D111BFF70001FA7E9AAC4DDBF6@EWFD19>
I don't know why we can't have 2 Numerical packages, when we have several
GUI choices. More the better. I suppose that people can upgrade or choose
gradually.
While we have better indexing, can we also have missing-values in arrays?
Sorry to keep bringing this up, but it's major importance to my work.
I also have minor problem, when 1/2 does not come back with 0.5.I just use
float for everything. I get NaN as missing and 1.0/2.0 always returns 0.5.
Frankly, this is way some Stat packages do it anyway. I don't think it would
be solved by Blitz, but mentioning it anyway.
**************************************************************
S. Hoon Yoon (Quant) Merrill Lynch
Equity Trading
yelled@yahoo.com hoon@bigfoot.com(w)
"Miracle is always only few standard deviations away, but so is
catastrophe."
* Expressed opinions are often my own, but NOT my employer's.
"I feel like a fugitive from the law of averages." Mauldin
**************************************************************
> -----Original Message-----
> From: Konrad Hinsen
> Sent: Tuesday, December 15, 1998 11:13 AM
> To: dubois1@llnl.gov
> Cc: matrix-sig@python.org
> Subject: Re: [Matrix-SIG] Re: blitz++ vs. NumPy?
>
> > numerical packages appear that make use of it. And, FWIW, eliminate a
> lot of
> > code we have to maintain/debug. It is easier to wrap than create, ne?
> And
> > NumPy will have to be rewritten, and soon, if it is to be maintainable.
>
> How much of the current NumPy code could be eliminated if NumPy were
> to use Blitz as much as possible? The Python interface to arrays would
> have to remain, I suppose, as well as the ufunc module. That's the
> majority of the core code; maybe the indexing stuff could be simpler,
> perhaps even a lot. The additional modules (LinearAlgebra, FFT, etc.)
> are already small wrappers around existing libraries.
>
> And what would be the consequences of a change to Blitz for C
> extensions that use arrays? Would it still be possible to interface
> with code using traditional C/Fortran style arrays? Would it be
> possible to do so without knowing C++? What would happen to people
> whose C++ compiler is not (yet?) suitable for Blitz? According to
> their Web page, that's still the majority of platforms.
>
> I don't want to leave the impression of wanting to block progress, but
> I write and distribute code that is supposed to be used and modified
> by today's real-life scientists. I can't expect them to get a
> particular compiler (or perhaps even a new machine) to use my code,
> and I can't expect them to learn C++.
> --
> --------------------------------------------------------------------------
> -----
> Konrad Hinsen | E-Mail: hinsen@cnrs-orleans.fr
> Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.55.69
> Rue Charles Sadron | Fax: +33-2.38.63.15.17
> 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/
> France | Nederlands/Francais
> --------------------------------------------------------------------------
> -----
>
> _______________________________________________
> Matrix-SIG maillist - Matrix-SIG@python.org
> http://www.python.org/mailman/listinfo/matrix-sig
From cgw@pgt.com Tue Dec 15 16:34:16 1998
From: cgw@pgt.com (Charles G Waldman)
Date: Tue, 15 Dec 1998 11:34:16 -0500 (EST)
Subject: [Matrix-SIG] Python crash when sort is given zero length
Message-ID: <13942.36744.594262.921284@janus.pgt.com>
list, take 2
In-Reply-To: <13942.35922.881228.950845@janus.pgt.com>
References: <199812141700.MAA17832@python.org>
<3674509F.26638416@mayo.edu>
<13942.35922.881228.950845@janus.pgt.com>
X-Mailer: VM 6.62 under 21.0 "Poitou60" XEmacs Lucid (beta60)
Charles G Waldman writes:
> This was quite easy to fix. Here's the (one-line) patch.
Oops, I attached the wrong file. The patch I sent is bogus. Sorry
about that! Here's the right fix.
--- multiarraymodule.c.orig Thu Aug 20 14:59:22 1998
+++ multiarraymodule.c Tue Dec 15 11:27:08 1998
@@ -390,7 +390,7 @@
elsize = ap->descr->elsize;
m = ap->dimensions[ap->nd-1];
- n = PyArray_SIZE(ap)/m;
+ n = (m==0) ? 0 : PyArray_SIZE(ap)/m;
for (ip = ap->data, i=0; i<n; i++, ip+=elsize*m) {
qsort(ip, m, elsize, compare_func);
}
From dubois1@llnl.gov Tue Dec 15 18:43:36 1998
From: dubois1@llnl.gov (Paul F. Dubois)
Date: Tue, 15 Dec 1998 10:43:36 -0800
Subject: [Matrix-SIG] Re: blitz++ vs. NumPy?
Message-ID: <002001be285a$d5546c40$f4160218@c1004579-c.plstn1.sfba.home.com>
-----Original Message-----
From: Yoon, Hoon (CICG - NY Program Trading) <HYoon@exchange.ml.com>
Cc: matrix-sig@python.org <matrix-sig@python.org>
Date: Tuesday, December 15, 1998 8:32 AM
Subject: RE: [Matrix-SIG] Re: blitz++ vs. NumPy?
>
>While we have better indexing, can we also have missing-values in arrays?
>Sorry to keep bringing this up, but it's major importance to my work.
>I also have minor problem, when 1/2 does not come back with 0.5.I just use
>float for everything. I get NaN as missing and 1.0/2.0 always returns 0.5.
Dean Williams of LLNL has a patched version of NumPy that supports missing
values (but also a lot of other stuff that is quite project-specific for his
own project). Assuming the paperwork goes through I will be working with
Dean some time after the new year, and while we have yet to firm up a
priority list, making a missing-value version of NumPy is an important goal.
To do this right probably involves rewriting NumPy in some way. Of course, I
will attempt in so doing to create a missing-value array component
independent of the project-dependent parts.
From dubois1@llnl.gov Tue Dec 15 18:49:59 1998
From: dubois1@llnl.gov (Paul F. Dubois)
Date: Tue, 15 Dec 1998 10:49:59 -0800
Subject: [Matrix-SIG] Re: blitz++ vs. NumPy?
Message-ID: <002701be285b$b7b6d460$f4160218@c1004579-c.plstn1.sfba.home.com>
I am aware of the concerns you expressed. Like I said, I will look into it.
I was just remarking in my most recent post that performance would not be
the primary reason for wanting to wrap Blitz, as others had assumed or
argued.
BTW, several people have encountered strange problems that eventually traced
back to the inclusion of a version of the BLAS in NumPy. I mentioned before
that including such things in the main package is a mistake, but gave in to
your argument that convenience for naive users mattered more. None of this
is very easy to decide.
From Oliphant.Travis@mayo.edu Tue Dec 15 21:58:26 1998
From: Oliphant.Travis@mayo.edu (Travis Oliphant)
Date: Tue, 15 Dec 1998 15:58:26 -0600 (CST)
Subject: [Matrix-SIG] Floating point exception when sort is given zero length list
In-Reply-To: <008901be2840$495863a0$f4160218@c1004579-c.plstn1.sfba.home.com>
Message-ID: <Pine.LNX.3.96.981215155007.12301B-100000@us2.mayo.edu>
I received some good feedback on my post of the floating-point exception
in NumPy. To clarify my purposes in posting, I did not find this error
as part of some bug search but while programming. I find the NumPy
distribution to be normally quite robust and so was surprised by the
error.
I was using Oliver Andrich's rpm which I discovered is based on
LLNLPython4. I upgraded to LLNLPython6 which is the lastest distribution
I could find and still found the same behavior.
The fix suggested by Charles Waldman was most helpful. I applied the
simple patch and now no longer get the exception, instead sort([])
returns a respectable empty array. Thank you very much.
Thanks to all who responded.
Travis O.
On Tue, 15 Dec 1998, Paul F. Dubois wrote:
> I believe this is fixed. Please run the most recent NumPy.
>
From dubois1@llnl.gov Tue Dec 15 22:25:00 1998
From: dubois1@llnl.gov (Paul F. Dubois)
Date: Tue, 15 Dec 1998 14:25:00 -0800
Subject: [Matrix-SIG] Re: Floating point exception when sort is given zero length list
Message-ID: <000801be2879$c1047040$f4160218@c1004579-c.plstn1.sfba.home.com>
I'm sorry for the confusion. When I said this was fixed I forgot to check
WHEN it was fixed. It was fixed after the last release so naturally you do
not have it. Here is our Perforce entry:
Change 608 by motteler@motteler on 1998/11/17 15:04:01
(ZCM 11/17/98) Fixed multiarray so that it wouldn't core dump python
when attempting to argmax/argmin, sort, or argsort an empty sequence.
Instead it now throws an exception and gives an informative comment.
Affected files ...
//depot/LLNLDistribution/Numerical/Src/multiarraymodule.c#3 edit
-----Original Message-----
From: Travis Oliphant <Oliphant.Travis@mayo.edu>
To: Paul F. Dubois <dubois1@llnl.gov>
Cc: matrix-sig@python.org <matrix-sig@python.org>
Date: Tuesday, December 15, 1998 2:02 PM
Subject: Floating point exception when sort is given zero length list
>
>I received some good feedback on my post of the floating-point exception
>in NumPy. To clarify my purposes in posting, I did not find this error
>as part of some bug search but while programming. I find the NumPy
>distribution to be normally quite robust and so was surprised by the
>error.
>
>I was using Oliver Andrich's rpm which I discovered is based on
>LLNLPython4. I upgraded to LLNLPython6 which is the lastest distribution
>I could find and still found the same behavior.
>
>The fix suggested by Charles Waldman was most helpful. I applied the
>simple patch and now no longer get the exception, instead sort([])
>returns a respectable empty array. Thank you very much.
>
>Thanks to all who responded.
>
>Travis O.
>
>
>On Tue, 15 Dec 1998, Paul F. Dubois wrote:
>
>> I believe this is fixed. Please run the most recent NumPy.
>>
>
>
From hinsen@cnrs-orleans.fr Wed Dec 16 11:24:00 1998
From: hinsen@cnrs-orleans.fr (Konrad Hinsen)
Date: Wed, 16 Dec 1998 12:24:00 +0100
Subject: [Matrix-SIG] Re: blitz++ vs. NumPy?
In-Reply-To: <002701be285b$b7b6d460$f4160218@c1004579-c.plstn1.sfba.home.com>
(dubois1@llnl.gov)
References: <002701be285b$b7b6d460$f4160218@c1004579-c.plstn1.sfba.home.com>
Message-ID: <199812161124.MAA26548@dirac.cnrs-orleans.fr>
> I am aware of the concerns you expressed. Like I said, I will look into it.
> I was just remarking in my most recent post that performance would not be
> the primary reason for wanting to wrap Blitz, as others had assumed or
> argued.
It is not even clear that a Blitz-based system would be faster (let
alone significantly) than the current implementation. I totally agree
that the biggest problem with NumPy is maintenance - it's a lot of
code, and not particularly "reader-friendly".
> BTW, several people have encountered strange problems that eventually traced
> back to the inclusion of a version of the BLAS in NumPy. I mentioned before
> that including such things in the main package is a mistake, but gave in to
> your argument that convenience for naive users mattered more. None of this
> is very easy to decide.
Once the DistUtils SIG has done its job (it's just starting!),
NumPy packaging should be reconsidered. It would then be no more effort
to install three packages at once than to install just a single one.
What exactly where the BLAS-related problems? I know that f2c-compiled
code is in principle not 100% portable C code, but I have never had
any problems in practice, although I have used it on most of the popular
Unix platforms.
Konrad.
--
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsen@cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.55.69
Rue Charles Sadron | Fax: +33-2.38.63.15.17
45071 Orleans Cedex 2 | Deutsch/Esperanto/English/
France | Nederlands/Francais
-------------------------------------------------------------------------------
From hinsen@cnrs-orleans.fr Wed Dec 16 11:30:18 1998
From: hinsen@cnrs-orleans.fr (Konrad Hinsen)
Date: Wed, 16 Dec 1998 12:30:18 +0100
Subject: [Matrix-SIG] Re: blitz++ vs. NumPy?
In-Reply-To: <A1AFC4253BA3D111BFF70001FA7E9AAC4DDBF6@EWFD19>
(HYoon@exchange.ml.com)
References: <A1AFC4253BA3D111BFF70001FA7E9AAC4DDBF6@EWFD19>
Message-ID: <199812161130.MAA20460@dirac.cnrs-orleans.fr>
> I don't know why we can't have 2 Numerical packages, when we have several
> GUI choices. More the better. I suppose that people can upgrade or choose
> gradually.
There's nothing wrong with having multiple numerics packages in
principle, but there is an important difference to the GUI situation:
the GUI community is so much larger than the numerics community
(speaking about Python of course) that it can afford duplicate
efforts more easily. In other words, the problem is limited
manpower!
Konrad.
--
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsen@cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.55.69
Rue Charles Sadron | Fax: +33-2.38.63.15.17
45071 Orleans Cedex 2 | Deutsch/Esperanto/English/
France | Nederlands/Francais
-------------------------------------------------------------------------------
From zcm@llnl.gov Wed Dec 16 15:33:25 1998
From: zcm@llnl.gov (Zane Motteler)
Date: Wed, 16 Dec 98 07:33:25 PST
Subject: [Matrix-SIG] Python crash when sort is given zero length list
In-Reply-To: <3674509F.26638416@mayo.edu>
References: <199812141700.MAA17832@python.org>
Message-ID: <v03007800b29d1366734a@[128.115.36.161]>
Hi Travis,
You wrote:
>I'm running NumPy 1.3 on Linux libc6 and get a floating
>point exception when I try to sort a list with zero elements.
>
>Here is the session.
>
>Anybody seen this and made the fix.
>
>>>> from Numeric import sort
>>>> sort([])
>Floating point exception
Yes, we are aware of the problem, and it will be fixed in the
next distribution.
What puczzles me is why you need a computer to sort a list
with 0 items ? ;-}
Zane
--------------------
Zane C. Motteler, Ph. D.
Professor Emeritus of Computer Science and Engineering
California Polytechnic State University, San Luis Obispo
zmottel@calpoly.edu
Currently:
Computer Scientist
Lawrence Livermore National Laboratory
P O Box 808, L-038 (Street address 7000 East Avenue, L-038)
Livermore, CA 94551-9900
zcm@llnl.gov 925/423-2143, FAX 925/423-9969
From cgw@pgt.com Wed Dec 16 15:50:54 1998
From: cgw@pgt.com (Charles G. Waldman)
Date: Wed, 16 Dec 1998 10:50:54 -0500 (Eastern Standard Time)
Subject: [Matrix-SIG] raising an exception on sorting zero-length arrays
Message-ID: <13943.55006.280000.630119@HERMES>
Are you sure that raising an exception when a user tries to sort a
zero-length array is the best thing to do? The Python "list" type
does not have this behavior:
>>> l = []
[]
>>> l.sort()
Why should the array type behave so much differently, raising an
exception? I think returning a zero-length array is the reasonable
thing to do in this case.
From HYoon@exchange.ml.com Wed Dec 16 20:25:20 1998
From: HYoon@exchange.ml.com (Yoon, Hoon (CICG - NY Program Trading))
Date: Wed, 16 Dec 1998 15:25:20 -0500
Subject: [Matrix-SIG] Re: blitz++ vs. NumPy?
Message-ID: <A1AFC4253BA3D111BFF70001FA7E9AAC4DDBFC@EWFD19>
Konrad,
I think Paul is saying that moving to a developed package like Blitz++
will reduce men-power input in long run. Kinda like Tkinter being only a
wrapper
and not something developed from scratch. If people are convinced that NumPy
needs to be
rewritten, then I guess this is the better way to go. Unfortunately, I am
sufficiently ignorant
of any details about underlying NumPy or C extension to form an opinion for
myself. Is this some
concensus reached by some subset of NumPy users (at least in LLNL) at this
point? I have not
heard a challenge about rewriting or updating NumPy alot, which Paul
mentioned.
Having a good documentation on the extension package underneath will
hopefully help things,
if it happens.
**************************************************************
S. Hoon Yoon (Quant) Merrill Lynch
Equity Trading
yelled@yahoo.com hoon@bigfoot.com(w)
"Miracle is always only few standard deviations away, but so is
catastrophe."
* Expressed opinions are often my own, but NOT my employer's.
"I feel like a fugitive from the law of averages." Mauldin
**************************************************************
> -----Original Message-----
> From: Konrad Hinsen
> Sent: Wednesday, December 16, 1998 6:30 AM
> To: HYoon@exchange.ml.com
> Cc: matrix-sig@python.org
> Subject: Re: [Matrix-SIG] Re: blitz++ vs. NumPy?
>
> > I don't know why we can't have 2 Numerical packages, when we have
> several
> > GUI choices. More the better. I suppose that people can upgrade or
> choose
> > gradually.
>
> There's nothing wrong with having multiple numerics packages in
> principle, but there is an important difference to the GUI situation:
> the GUI community is so much larger than the numerics community
> (speaking about Python of course) that it can afford duplicate
> efforts more easily. In other words, the problem is limited
> manpower!
>
> Konrad.
> --
From amullhau@ix.netcom.com Wed Dec 16 21:05:57 1998
From: amullhau@ix.netcom.com (Andrew P. Mullhaupt)
Date: Wed, 16 Dec 1998 16:05:57 -0500
Subject: [Matrix-SIG] Re: blitz++ vs. NumPy?
Message-ID: <177a01be2937$e23ad880$ce0709c0@amullhau.ix.netcom.com>
> I think Paul is saying that moving to a developed package like Blitz++
>will reduce men-power input in long run
Does Blitz++ support multidimensional arrays, etc.?
Later,
Andrew Mullhaupt
From da@skivs.ski.org Wed Dec 16 21:10:08 1998
From: da@skivs.ski.org (David Ascher)
Date: Wed, 16 Dec 1998 13:10:08 -0800 (Pacific Standard Time)
Subject: [Matrix-SIG] Re: blitz++ vs. NumPy?
In-Reply-To: <177a01be2937$e23ad880$ce0709c0@amullhau.ix.netcom.com>
Message-ID: <Pine.WNT.4.04.9812161307520.228-100000@rigoletto.ski.org>
> Does Blitz++ support multidimensional arrays, etc.?
Yes. Even the etc. =)
A quick glance at the documentation will give you a flavor of the thing:
http://monet.uwaterloo.ca/blitz/manual/
Look e.g. at http://monet.uwaterloo.ca/blitz/manual/blitz02.html
--david
From turner@blueskystudios.com Wed Dec 16 21:30:53 1998
From: turner@blueskystudios.com (John Turner)
Date: Wed, 16 Dec 1998 16:30:53 -0500 (EST)
Subject: [Matrix-SIG] Re: blitz++ vs. NumPy?
In-Reply-To: <177a01be2937$e23ad880$ce0709c0@amullhau.ix.netcom.com>
References: <177a01be2937$e23ad880$ce0709c0@amullhau.ix.netcom.com>
Message-ID: <13944.9869.495758.371055@oneida.blueskyprod.com>
>>>>> "APM" == Andrew P Mullhaupt <amullhau@ix.netcom.com> writes:
APM> Does Blitz++ support multidimensional arrays, etc.?
Yes. It's almost as good as Fortran 9x that way.
(ducking)
--
John A. Turner, Ph.D. Senior Research Associate
Blue Sky | VIFX http://www.bluesky-vifx.com/
One South Road, Harrison, NY 10528 http://www.lanl.gov/home/turner/
Phone: 914-381-8400 Fax: 914-381-9790/1
Check out our newly-released fully computer-generated short film at:
http://bunny.blueskystudios.com/
You've never seen computer-generated images like this...
From phil@geog.ubc.ca Wed Dec 16 22:57:55 1998
From: phil@geog.ubc.ca (Phil Austin)
Date: Wed, 16 Dec 1998 14:57:55 -0800
Subject: [Matrix-SIG] Re: blitz++ vs. NumPy?
In-Reply-To: <13944.9869.495758.371055@oneida.blueskyprod.com>
References: <177a01be2937$e23ad880$ce0709c0@amullhau.ix.netcom.com>
<13944.9869.495758.371055@oneida.blueskyprod.com>
Message-ID: <199812162257.OAA00918@brant.geog.ubc.ca>
>>>>> "John" == John Turner <turner@blueskystudios.com> writes:
>>>>> "APM" == Andrew P Mullhaupt <amullhau@ix.netcom.com> writes:
APM> Does Blitz++ support multidimensional arrays, etc.?
John> Yes. It's almost as good as Fortran 9x that way.
.. and that's essentially the design objective--Blitz makes
efficient F90 array operations available in C++. There is
another expression-template style array class that provides
many standard linear algebra operations:
http://www.lsc.nd.edu/research/mtl/doc/index.html
Blitz and mtl can work together, since mtl (like Blitz) can
construct dense arrays from existing allocated memory.
Phil
From frank@ned.dem.csiro.au Thu Dec 17 02:00:53 1998
From: frank@ned.dem.csiro.au (Frank Horowitz)
Date: Thu, 17 Dec 1998 10:00:53 +0800
Subject: [Matrix-SIG] Re: blitz++ vs. NumPy?
In-Reply-To: <Pine.WNT.4.04.9812161307520.228-100000@rigoletto.ski.org>
References: <177a01be2937$e23ad880$ce0709c0@amullhau.ix.netcom.com>
Message-ID: <v03130307b29e147a0b54@[130.116.145.101]>
At 5:10 AM +0800 17/12/98, David Ascher wrote:
>> Does Blitz++ support multidimensional arrays, etc.?
>
>Yes. Even the etc. =)
>
>A quick glance at the documentation will give you a flavor of the thing:
>
>http://monet.uwaterloo.ca/blitz/manual/
>
>Look e.g. at http://monet.uwaterloo.ca/blitz/manual/blitz02.html
>
At the risk of sounding overly pessimistic, can I ask if anyone else out
there is bothered by Blitz++'s heavy reliance on C++ template semantics?
Our group's experience in developing a C++ code with both SGI and
Alpha/DUnix compilers has taught us to be quite wary of subtle differences
in compiler template semantics.
Personally, if NumPy is moved to a Blitz++ base, I'd keep a copy of the
older C version around, "just in case".
(I'm not a C++ guru. That's why I like Python! :-)
Frank Horowitz
--
Frank Horowitz frank@ned.dem.csiro.au
Australian Geodynamics Cooperative Research Centre, and
CSIRO-Exploration & Mining, PO Box 437, Nedlands, WA 6009, AUSTRALIA
Direct: +61 8 9284 8431; FAX: +61 8 9389 1906; Reception: +61 8 9389 8421
From amullhau@ix.netcom.com Thu Dec 17 02:21:42 1998
From: amullhau@ix.netcom.com (Andrew P. Mullhaupt)
Date: Wed, 16 Dec 1998 21:21:42 -0500
Subject: [Matrix-SIG] Re: blitz++ vs. NumPy?
Message-ID: <190201be2963$fcfd1c60$ce0709c0@amullhau.ix.netcom.com>
-----Original Message-----
From: Frank Horowitz <frank@ned.dem.csiro.au>
To: matrix-sig@python.org <matrix-sig@python.org>
Date: Wednesday, December 16, 1998 9:01 PM
Subject: Re: [Matrix-SIG] Re: blitz++ vs. NumPy?
>At the risk of sounding overly pessimistic, can I ask if anyone else out
>there is bothered by Blitz++'s heavy reliance on C++ template semantics?
That sounds pretty unhappy. C++ is bad enough but templates are scattered
enough to be a pain in the ass last time I looked, which was when I gave STL
a try.
I haven't seen any C++ tools yet that had memory management that was
industrial strength performance. Is anyone familiar enough with Blitz++ and
advanced VM to know whether they knew which way was up when they wrote it?
Later,
Andrew Mullhaupt
From dubois1@llnl.gov Thu Dec 17 05:32:28 1998
From: dubois1@llnl.gov (Paul F. Dubois)
Date: Wed, 16 Dec 1998 21:32:28 -0800
Subject: [Matrix-SIG] Re: blitz++ vs. NumPy?
Message-ID: <001b01be297e$a4f6ac00$f4160218@c1004579-c.plstn1.sfba.home.com>
>I haven't seen any C++ tools yet that had memory management that was
>industrial strength performance. Is anyone familiar enough with Blitz++ and
>advanced VM to know whether they knew which way was up when they wrote it?
>
>Later,
>Andrew Mullhaupt
>
The most superficial survey of the literature will answer your question in
the affirmative.
It is true that Blitz stresses the template facility more than almost any
other software, in order to achieve maximum performance. I've had no trouble
with even the Microsoft compiler for the simpler kind of template work I've
done in CXX.
From amullhau@ix.netcom.com Thu Dec 17 06:09:18 1998
From: amullhau@ix.netcom.com (Andrew P. Mullhaupt)
Date: Thu, 17 Dec 1998 01:09:18 -0500
Subject: [Matrix-SIG] Re: blitz++ vs. NumPy?
Message-ID: <196c01be2983$c9443e60$ce0709c0@amullhau.ix.netcom.com>
-----Original Message-----
From: Paul F. Dubois <dubois1@llnl.gov>
To: Andrew P. Mullhaupt <amullhau@ix.netcom.com>; matrix-sig@python.org
<matrix-sig@python.org>
Date: Thursday, December 17, 1998 12:32 AM
Subject: Re: [Matrix-SIG] Re: blitz++ vs. NumPy?
>
>>I haven't seen any C++ tools yet that had memory management that was
>>industrial strength performance. Is anyone familiar enough with Blitz++
and
>>advanced VM to know whether they knew which way was up when they wrote it?
>>
>The most superficial survey of the literature will answer your question in
>the affirmative.
Such a survey makes it look to me like the answer is negative.
It looks to me like special array constructors would have to be written to
accommodate things like ephemeral mapped file storage, since I saw no
facility for managing the namespace mapping from objects to file systems. It
also looks like users who want to provide path resources for mapped objects
will have to write their own constructors, and keep track of operations to
determine when the file handles can be closed.
Looks to me like Blitz expects to manage the VM resources underlying arrays
as if they were on the heap, as opposed to full fledged VM objects.
Later,
Andrew Mullhaupt
From r.hooft@euromail.net Thu Dec 17 07:58:22 1998
From: r.hooft@euromail.net (Rob Hooft)
Date: Thu, 17 Dec 1998 08:58:22 +0100 (MET)
Subject: [Matrix-SIG] Re: blitz++ vs. NumPy?
In-Reply-To: <196c01be2983$c9443e60$ce0709c0@amullhau.ix.netcom.com>
References: <196c01be2983$c9443e60$ce0709c0@amullhau.ix.netcom.com>
Message-ID: <13944.47518.451173.759388@octopus.chem.uu.nl>
There is a lot of discussion about the subject, but I haven't seen
anyone that was worried about their own extensions to NumPy that would
need to be rewritten if NumPy would be redesigned.
I'd like to express this worry for my code.... The current design of
NumPy works great for me.
Another worry is that almost inevitably the copy/overwrite policy of
array objects will change when an external library is used. This means
largely incompatible changes at the Python level as well.
I agree that there are a few points where my code could benefit from
some reprogramming:
- If I profile the code, it seems a lot of my time is spent in arrayrange,
I might be tricked into rewriting this beast in C for my purposes.
- Sometimes I concatenate 20 576x625 arrays, to a 20x576x625 array, at
the concatenation time, this means all data is there twice. However,
I do not see how to avoid this problem.
For the rest I have rewritten a few subroutines that took too much
time in C as extensions of NumPy (eg variants on 'clip'), and I don't
really feel like doing that again....
Am I really alone?
Rob
--
===== R.Hooft@EuroMail.net http://www.xs4all.nl/~hooft/rob/ =====
===== R&D, Nonius BV, Delft http://www.nonius.nl/ =====
===== PGPid 0xFA19277D ========================== Use Linux! =========
From P.S.Craig@durham.ac.uk Thu Dec 17 09:21:47 1998
From: P.S.Craig@durham.ac.uk (P.S.Craig@durham.ac.uk)
Date: Thu, 17 Dec 1998 09:21:47 +0000 (GMT)
Subject: [Matrix-SIG] Re: blitz++ vs. NumPy?
In-Reply-To: <13944.47518.451173.759388@octopus.chem.uu.nl> from "Rob Hooft" at Dec 17, 98 08:58:22 am
Message-ID: <19892.199812170921@laplace>
Rob Hooft writes:
>
> There is a lot of discussion about the subject, but I haven't seen
> anyone that was worried about their own extensions to NumPy that would
> need to be rewritten if NumPy would be redesigned.
>
> I'd like to express this worry for my code.... The current design of
> NumPy works great for me.
>
.
.
.
> Am I really alone?
>
> Rob
>
Not at all, I should say. If I wanted to use C++, I wouldn't be using
python. I came to python in despair at the state of C++ and its
non-reusability/non-portability about 4-5 years ago. Of course C++ may
have improved since then?
I think that NumPy is pretty wonderful. It has a simple design, is
reasonably efficient, is (fairly) internally consistent, and mostly
interacts well with core python. The actual C code is sometimes a bit
obscure, but I have always viewed that as a temporary
matter. Moreover, about a year ago I worked out how to do more
flexible indexing (non-stride based) and still retain the efficiency
of the current implementation for code that doesn't need it. I just
haven't had the time to write the code.
One final point in defense of NumPy as it stands: I use
multi-dimensional arrays of arbitrary python Objects extensively in
some database related code and I would hate to lose that facility,
which I imagine would be the case with the Blitz++ approach.
While writing this, I have realised that I feel strongly enough that I
would want to save the existing code and build on it, under a
different module name if necessary.
Cheers,
Peter Craig
#--------------------------------------------------------------------#
| E-mail: P.S.Craig@durham.ac.uk Telephone: +44-91-3742376 (Work) |
| Fax: +44-91-3747388 +44-91-3860448 (Home) |
| |
| WWW: http://fourier.dur.ac.uk:8000/stats/people/psc/psc.html |
| |
| Snail: Peter Craig, Dept. of Math. Sciences, Univ. of Durham, |
| South Road, Durham DH1 3LE, England |
#--------------------------------------------------------------------#
From turner@blueskystudios.com Thu Dec 17 15:51:58 1998
From: turner@blueskystudios.com (John Turner)
Date: Thu, 17 Dec 1998 10:51:58 -0500 (EST)
Subject: [Matrix-SIG] Re: blitz++ vs. NumPy?
In-Reply-To: <199812162257.OAA00918@brant.geog.ubc.ca>
References: <177a01be2937$e23ad880$ce0709c0@amullhau.ix.netcom.com>
<13944.9869.495758.371055@oneida.blueskyprod.com>
<199812162257.OAA00918@brant.geog.ubc.ca>
Message-ID: <13945.10398.331932.772510@oneida.blueskyprod.com>
>>>>> "APM" == Andrew P Mullhaupt <amullhau@ix.netcom.com>
>>>>> "John" == John Turner <turner@blueskystudios.com>
>>>>> "PA" == Phil Austin <phil@geog.ubc.ca>
APM> Does Blitz++ support multidimensional arrays, etc.?
John> Yes. It's almost as good as Fortran 9x that way.
PA> .. and that's essentially the design objective--Blitz makes
PA> efficient F90 array operations available in C++.
I understand that objective. If forced to write something that needs
efficient array operations in C++, there's a good chance I would use
Blitz++. But for such a project, given the choice I'd personally
rather use a language that had arrays as 1st-class objects and leave
these kinds of issues to the compiler writers.
Languages are tools. Some are screwdrivers, some are pliers, some are
swiss army knives. They have different strengths and weaknesses.
However, to bring this somewhat back to NumPy relevance, a move to
Blitz++ may very well be the correct direction for NumPy. It would
have advantages over the current situation, and I can't very well
argue for a move to a modern Fortran replacement for the current
LAPACK/BLAS base since:
o there's no free F9x compiler,
o there's no free F9x to C converter (that I know of), and
o many commercial F9x compilers are still immature
Though Blitz++ suffers from the immaturity of many commercial C++
compilers, too, my impression is that since there is communication
with egcs development there will continue to be a freely-available
compiler capable of building Blitz++ (even if sporadically).
PA> There is another expression-template style array class that
PA> provides many standard linear algebra operations:
That's the great thing about C++. There are so many array classes to
choose from (A++/P++, MV++, TNT, etc. etc. etc.).
(Sorry. I'm being a jerk.)
--
John A. Turner
Senior Research Associate, Blue Sky | VIFX
http://www.bluesky-vifx.com/
http://www.lanl.gov/home/turner/
"If your only tool is C++, everything starts to look like a thumb."
From hinsen@cnrs-orleans.fr Thu Dec 17 20:18:41 1998
From: hinsen@cnrs-orleans.fr (Konrad Hinsen)
Date: Thu, 17 Dec 1998 21:18:41 +0100
Subject: [Matrix-SIG] Re: blitz++ vs. NumPy?
In-Reply-To: <13944.47518.451173.759388@octopus.chem.uu.nl>
(r.hooft@euromail.net)
References: <196c01be2983$c9443e60$ce0709c0@amullhau.ix.netcom.com> <13944.47518.451173.759388@octopus.chem.uu.nl>
Message-ID: <199812172018.VAA14946@dirac.cnrs-orleans.fr>
> There is a lot of discussion about the subject, but I haven't seen
> anyone that was worried about their own extensions to NumPy that would
> need to be rewritten if NumPy would be redesigned.
I did mention compatibility as a point of concern recently - but not
quite as strongly as you do now, because I was expecting that even a
rewritten NumPy would keep its C API as close as possible to the
current one.
I have plenty of modules that uses NumPy at the C level, but none
of it makes any assumptions about the way NumPy executes its operations.
All I require is a linear data array and the dimension lists and data
type indicators.
Konrad.
--
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsen@cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.55.69
Rue Charles Sadron | Fax: +33-2.38.63.15.17
45071 Orleans Cedex 2 | Deutsch/Esperanto/English/
France | Nederlands/Francais
-------------------------------------------------------------------------------
From Oliphant.Travis@mayo.edu Fri Dec 18 03:11:02 1998
From: Oliphant.Travis@mayo.edu (Travis Oliphant)
Date: Thu, 17 Dec 1998 21:11:02 -0600 (CST)
Subject: [Matrix-SIG] RedHat RPMS i386/src of Cephes module
In-Reply-To: <199812171700.MAA28531@python.org>
Message-ID: <Pine.LNX.3.96.981217210535.8388A-100000@us2.mayo.edu>
For those interested I've just released an rpm version of the cephes
module that provides special functions for arbitrary arrays with NumPy.
i386 binary and source RPM's are provided.
With this module it is a snap to define kaiser windows in python as
in Matlab:
def kaiser(M,beta):
n = arange(0,M)
alpha = (M-1)/2.0
return cephes.i0(beta * sqrt(1-((n-alpha)/alpha)**2))/cephes.i0(beta)
Any feedback is most welcome,
----------------------------------------------------
Travis Oliphant 200 First St SW
Rochester MN 55905
Ultrasound Research Lab (507) 286-5923
Mayo Graduate School Oliphant.Travis@mayo.edu
From Oliphant.Travis@mayo.edu Fri Dec 18 03:19:15 1998
From: Oliphant.Travis@mayo.edu (Travis Oliphant)
Date: Thu, 17 Dec 1998 21:19:15 -0600 (CST)
Subject: [Matrix-SIG] cephes RPM -- where to get!!
In-Reply-To: <199812171700.MAA28531@python.org>
Message-ID: <Pine.LNX.3.96.981217211801.8395B-100000@us2.mayo.edu>
Sorry,
I forgot to mention where to get the binary and source RPM's
http://oliphant.netpedia.net
Regards
----------------------------------------------------
Travis Oliphant 200 First St SW
Rochester MN 55905
Ultrasound Research Lab (507) 286-5923
Mayo Graduate School Oliphant.Travis@mayo.edu
From edcjones@erols.com Sun Dec 20 00:52:17 1998
From: edcjones@erols.com (Edward C. Jones)
Date: Sat, 19 Dec 1998 19:52:17 -0500
Subject: [Matrix-SIG] Proposal: Array of arrays and non-copying slices
Message-ID: <367C4A41.E3E76318@erols.com>
I have always liked the "reduceat" method. It is powerful and
useful. The
NumPy documentation is wrong when it says "This is a weird
function, and
most people should just ignore it." This method essentially
divides an array
into blocks (along one axis only) and calls "reduce" on each one.
Operating on
blocks like this is very common in spreadsheets, numerical
analysis, image
processing, and computer graphics.
I feel that NumPy can be made more useful and "reduceat" more
intuitive by
adding:
1. A non-copying slice operation in addition to the current
copying slice
operation.
With this, one can filter part of an image or average part of a
spreadsheet
without expensive copying operations. The non-copying slice can
also be used to
divide an array into an array of blocks.
What is a good notation for this? There would be a flame war if I
proposed
using {} instead of [] for non-copying arrays. So perhaps, if
a = [0,1,2,3,4,5], let a.sub[1:4] == [1,2,3].
2. The ability to treat an array of arrays as a single array in
either of two
senses as explained below.
Consider the arrays:
a = [[0 1]
[2 3]]
b = [[10 11]
[12 13]]
c = [[20 21]
[22 23]]
d = [[30 31]
[32 33]]
abcd = [[ 0 1 10 11] (1)
[ 2 3 12 13]
[20 21 30 31]
[22 23 32 33]]
Suppose I start with the small arrays a, b, c, and d. Currently I
can create
an array of arrays by
A = Numeric.array([[0,0],[0,0]], 'O') # Entries can be any Python
object.
A[0][0] = a
A[0][1] = b
A[1][0] = c
A[1][1] = d
This can viewed as adding dimensions. Indeed, "print A" gives
[[[[0 1] [2 3]] (2)
[[10 11] [12 13]] ]
[[[20 21] [22 23]]
[[30 31] [32 33]] ]]
which has shape (2,2,2,2) (but A.shape actually is (2,2)).
But it can be also visualized as creating the array abcd of shape
(4,4).
This second interpretation is would be useful to many users. For
example, in
numerical analysis arrays are often divided into blocks in order
to prevent
thrashing or as part of an fft or other algorithm. Moreover, in
computer
vision, image processing and computer graphics pyramid or
quadtree methods
are widely used.
One can also start with an actual array of shape (2,2,2,2) or
(4,4) and
think of it as an array of arrays. One could then start with a
single
actual array and operate on it as if it were an array of arrays.
Currently, one character strings are used for typecodes in
defining arrays.
Add the new typecodes 'A' and 'B' where 'A' defines an array of
arrays
interpreted as adding dimensions and 'B' defines an array of
arrays
interpreted as a tiling. So AA = Numeric.array([[a,b],[c,d]],
'A') would
print as (2) above while BB = Numeric.array([[a,b],[c,d]], 'B')
would print
as (1) above. Subscripts would behave according to
interpretation:
AA[1,0,1,0] == 22 and BB[1,2] == 12.
To divide an array into an array of blocks define an array of
non-copying
subarrays and call Numeric.array. This is a painful way to do a
common
operation. I suggest using "indices" as used in the "reduceat"
function.
To divide an 8 by 8 array, A88, into 4 by 4 blocks one might use
something
like Numeric.array.blocks(A88, ((0,4), (0,4)) ) or
A.blocks( ((0,4), (0,4)) ).
Now "reduceat" becomes intuitive: "reduceat" called for an array
of arrays
does a "reduce" on each array in the array of arrays.
From dubois1@llnl.gov Mon Dec 21 19:24:57 1998
From: dubois1@llnl.gov (Paul F. Dubois)
Date: Mon, 21 Dec 1998 11:24:57 -0800
Subject: [Matrix-SIG] Release 7
Message-ID: <000801be2d17$99b49900$f4160218@c1004579-c.plstn1.sfba.home.com>
David,
Will you send this message to comp.lang.python and comp.lang.python.announce
for me? My news server seems to be dead.
At ftp://ftp-icf.llnl.gov/pub/python, LLNLPython.tgz and NumPy.exe now point
to release 7.
1.. Added inverse_fft2d supplied by a user.
2.. Fixed sort to return copy of input array if input array empty.
3.. Fixed argsort to return empty integer array if input array empty.
4.. Fixed argmax and argmin to throw exceptions if input array empty.
5.. Changed #! startups to use /usr/bin/env instead of /bin/env.
6.. Added docstring for array2string to ArrayPrinter.py.
7.. Fixed a link in doc/home.html.
8.. Added warnings about conflicts with native BLAS on some machines.
From cgw@pgt.com Mon Dec 21 19:52:33 1998
From: cgw@pgt.com (Charles G Waldman)
Date: Mon, 21 Dec 1998 14:52:33 -0500 (EST)
Subject: [Matrix-SIG] Release 7
In-Reply-To: <000801be2d17$99b49900$f4160218@c1004579-c.plstn1.sfba.home.com>
References: <000801be2d17$99b49900$f4160218@c1004579-c.plstn1.sfba.home.com>
Message-ID: <13950.42753.615009.180682@janus.pgt.com>
Paul F. Dubois writes:
>
> At ftp://ftp-icf.llnl.gov/pub/python, LLNLPython.tgz and NumPy.exe now point
> to release 7.
I just downloaded this to see if it fixes some of the core-dumps I've
been getting. I notice that the indentation in Lib/Numeric.py around
line 171 is badly messed up... here's a patch.
--- Numeric.py.orig Mon Dec 21 14:02:01 1998
+++ Numeric.py Mon Dec 21 14:49:54 1998
@@ -168,23 +168,23 @@
new_axes [0:0] = [axis1, axis2]
a = transpose (a, new_axes)
s = a.shape
- if len (s) == 2:
- n1 = s [0]
- n2 = s [1]
- n = n1 * n2
- s = (n,) + s [2:]
- a = reshape (a, s)
- if offset < 0:
- offset = - n2 * offset
- last = n
- else:
- last = n - n2 * offset
- return take (a, range (offset, last, n2 + 1), 0)
- else :
- my_diagonal = []
- for i in range (s [0]) :
- my_diagonal.append (diagonal (a [i], offset))
- return array (my_diagonal)
+ if len (s) == 2:
+ n1 = s [0]
+ n2 = s [1]
+ n = n1 * n2
+ s = (n,) + s [2:]
+ a = reshape (a, s)
+ if offset < 0:
+ offset = - n2 * offset
+ last = n
+ else:
+ last = n - n2 * offset
+ return take (a, range (offset, last, n2 + 1), 0)
+ else :
+ my_diagonal = []
+ for i in range (s [0]) :
+ my_diagonal.append (diagonal (a [i], offset))
+ return array (my_diagonal)
def trace(a, offset=0, axis1=0, axis2=1):
From cgw@pgt.com Mon Dec 21 19:56:29 1998
From: cgw@pgt.com (Charles G Waldman)
Date: Mon, 21 Dec 1998 14:56:29 -0500 (EST)
Subject: [Matrix-SIG] Release 7
In-Reply-To: <000801be2d17$99b49900$f4160218@c1004579-c.plstn1.sfba.home.com>
References: <000801be2d17$99b49900$f4160218@c1004579-c.plstn1.sfba.home.com>
Message-ID: <13950.42989.491575.533793@janus.pgt.com>
Paul F. Dubois writes:
> 5.. Changed #! startups to use /usr/bin/env instead of /bin/env.
Isn't it a safe bet to assume "env" can be found by the standard
search path and just write
#!env python
This seems to me like the most portable construction.
From dubois1@llnl.gov Mon Dec 21 20:11:53 1998
From: dubois1@llnl.gov (Paul F. Dubois)
Date: Mon, 21 Dec 1998 12:11:53 -0800
Subject: [Matrix-SIG] Release 7
Message-ID: <001701be2d1e$26d03280$f4160218@c1004579-c.plstn1.sfba.home.com>
Accckkkk. I fixed it at 12:10 PST.
Thanks for catching it.
-----Original Message-----
From: Charles G Waldman <cgw@pgt.com>
To: Paul F. Dubois <dubois1@llnl.gov>
Cc: matrix-sig@python.org <matrix-sig@python.org>
Date: Monday, December 21, 1998 11:52 AM
Subject: [Matrix-SIG] Release 7
>Paul F. Dubois writes:
> >
> > At ftp://ftp-icf.llnl.gov/pub/python, LLNLPython.tgz and NumPy.exe now
point
> > to release 7.
>
>I just downloaded this to see if it fixes some of the core-dumps I've
>been getting. I notice that the indentation in Lib/Numeric.py around
>line 171 is badly messed up... here's a patch.
>
>
>--- Numeric.py.orig Mon Dec 21 14:02:01 1998
>+++ Numeric.py Mon Dec 21 14:49:54 1998
>@@ -168,23 +168,23 @@
> new_axes [0:0] = [axis1, axis2]
> a = transpose (a, new_axes)
> s = a.shape
>- if len (s) == 2:
>- n1 = s [0]
>- n2 = s [1]
>- n = n1 * n2
>- s = (n,) + s [2:]
>- a = reshape (a, s)
>- if offset < 0:
>- offset = - n2 * offset
>- last = n
>- else:
>- last = n - n2 * offset
>- return take (a, range (offset, last, n2 + 1), 0)
>- else :
>- my_diagonal = []
>- for i in range (s [0]) :
>- my_diagonal.append (diagonal (a [i], offset))
>- return array (my_diagonal)
>+ if len (s) == 2:
>+ n1 = s [0]
>+ n2 = s [1]
>+ n = n1 * n2
>+ s = (n,) + s [2:]
>+ a = reshape (a, s)
>+ if offset < 0:
>+ offset = - n2 * offset
>+ last = n
>+ else:
>+ last = n - n2 * offset
>+ return take (a, range (offset, last, n2 + 1), 0)
>+ else :
>+ my_diagonal = []
>+ for i in range (s [0]) :
>+ my_diagonal.append (diagonal (a [i], offset))
>+ return array (my_diagonal)
>
>
> def trace(a, offset=0, axis1=0, axis2=1):
>
From cgw@pgt.com Mon Dec 21 21:35:06 1998
From: cgw@pgt.com (Charles G Waldman)
Date: Mon, 21 Dec 1998 16:35:06 -0500 (EST)
Subject: [Matrix-SIG] NumPy core dump
In-Reply-To: <002701be2d0f$e03854a0$f4160218@c1004579-c.plstn1.sfba.home.com>
References: <002701be2d0f$e03854a0$f4160218@c1004579-c.plstn1.sfba.home.com>
Message-ID: <13950.48906.902609.541364@janus.pgt.com>
Several weeks ago, I wrote:
> Using Numeric version 1.5, from the LLNLPython6 distribution, on top
> of Python1.5.2b1, the following piece of code causes a core-dump:
>
> >>>from Numeric import *
> >>>AM = fromfunction(lambda x:sin(x*pi/512))
>
> I know the code is incorrect (missing second arg. to fromfunction) but
> shouldn't this raise an exception rather than dumping core?
This behavior is still there in today's 1.7 version. However I think
it may be due to a compiler bug. I've been hearing (on the
xemacs-beta mailing list) of a bug in gcc 2.7.2.3 related to register
saving. Using the "-fno-caller-saves" flag is alleged to work around
this problem. So, I rebuilt Python1.5.2 and the Numeric extensions
with the -fno-caller-saves option. Now I get a most peculiar
behavior: no longer is there an immediate core-dump; I get
a TypeError, as expected, but not until after I enter another line.
It doesn't seem to matter if the line is blank or contains
code, as the following examples show:
janus:~> python
Python 1.5.2b1 (#3, Dec 21 1998, 15:59:39) [GCC 2.7.2.3] on linux2
Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
>>> from Numeric import *
>>> AM = fromfunction(lambda x:sin(x*pi/512))
>>>
TypeError: not enough arguments; expected 2, got 1
janus:~> python
Python 1.5.2b1 (#3, Dec 21 1998, 15:59:39) [GCC 2.7.2.3] on linux2
Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
>>> from Numeric import *
>>> AM = fromfunction(lambda x:sin(x*pi/512))
>>> print AM
TypeError: not enough arguments; expected 2, got 1
From mal@lemburg.com Mon Dec 21 22:15:47 1998
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 21 Dec 1998 23:15:47 +0100
Subject: [Matrix-SIG] Re: [PSA MEMBERS] Re: [Matrix-SIG] NumPy core dump
References: <002701be2d0f$e03854a0$f4160218@c1004579-c.plstn1.sfba.home.com> <13950.48906.902609.541364@janus.pgt.com>
Message-ID: <367EC893.E21ECA0@lemburg.com>
Charles G Waldman wrote:
>
> Several weeks ago, I wrote:
>
> > Using Numeric version 1.5, from the LLNLPython6 distribution, on top
> > of Python1.5.2b1, the following piece of code causes a core-dump:
> >
> > >>>from Numeric import *
> > >>>AM = fromfunction(lambda x:sin(x*pi/512))
> >
> > I know the code is incorrect (missing second arg. to fromfunction) but
> > shouldn't this raise an exception rather than dumping core?
>
> This behavior is still there in today's 1.7 version. However I think
> it may be due to a compiler bug. I've been hearing (on the
> xemacs-beta mailing list) of a bug in gcc 2.7.2.3 related to register
> saving. Using the "-fno-caller-saves" flag is alleged to work around
> this problem. So, I rebuilt Python1.5.2 and the Numeric extensions
> with the -fno-caller-saves option. Now I get a most peculiar
> behavior: no longer is there an immediate core-dump; I get
> a TypeError, as expected, but not until after I enter another line.
> It doesn't seem to matter if the line is blank or contains
> code, as the following examples show:
I don't know anything about NumPy, but the error you describe
is typical of a situation where C code sets an exception using
PyExc_SetString() without returning NULL to the caller.
--
Marc-Andre Lemburg Y2000: 375 days left
---------------------------------------------------------------------
: Python Pages >>> http://starship.skyport.net/~lemburg/ :
---------------------------------------------------------
From Takahiro.Oishi@db.com Tue Dec 22 06:20:30 1998
From: Takahiro.Oishi@db.com (Oishi, Takahiro)
Date: Tue, 22 Dec 1998 15:20:30 +0900
Subject: [Matrix-SIG] I couldn't find 'outer' in the name space.
Message-ID: <367F3A2D.81522459@db.com>
Hi,
I'm trying to use MLab.py.
But I found errors with some functions defined in MLab.py.
I can't use 'outer' (defined in 'ufuncobject.c) like below.
>>> from MLab import *
>>> cov(a) # 'a' is an array
Traceback (innermost last):
File "<stdin>", line 1, in ?
File "/usr/local/python/lib/python1.5/MLab.py",
line 162, in cov
sum_cov = sum_cov+outer(multiply, v,v)
NameError: outer
>>>
Of course, I can't use 'corrcoef' using 'outer', either.
Has anyone found same kind of problem?
Did not I build NumPy and python correctly?
Or if anyone have any suggestion, plese let me know.
I appreciate that.
My python runs under Red Hat Linux 4.2 (kernel 2.0.33) on Pentium II
and Solaris 2.5.
(Same kind of errors happened under both unix.)
Basically I succeeded to build NumPy with no special options.
(Version of NumPy and NumPyLib
NumPy-1.0b3.tar.gz
NumPyLib-1.0b2.tar.gz )
As regards,
T.Oishi
From Oliphant.Travis@mayo.edu Tue Dec 22 08:03:52 1998
From: Oliphant.Travis@mayo.edu (Travis E. Oliphant)
Date: Tue, 22 Dec 1998 02:03:52 -0600
Subject: [Matrix-SIG] Core dump in NumPy when multiplying with complex numbers.
References: <199812181700.MAA10743@python.org>
Message-ID: <367F5268.D1A74FF3@mayo.edu>
I've had a few posts with core dumps lately as I've been trying to use
NumPy quite a bit.
The latest segmentation fault occurs when I try to multiply a Numeric
array by a complex number.
The session:
Python 1.5.1 (#1, Aug 8 1998, 13:33:40) [GCC egcs-2.90.29 980515 (egc
on linux2
Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
>>> import Numeric
>>> (1+0j)*Numeric.array([1,2])
Segmentation fault (core dumped)
Does this happen to anyone else or is it compiler-related...?
----------------------------------------------------
Travis Oliphant 200 First St SW
Rochester MN 55905
Ultrasound Research Lab (507) 286-5923
Mayo Graduate School Oliphant.Travis@mayo.edu
From jhauser@ifm.uni-kiel.de Tue Dec 22 08:47:55 1998
From: jhauser@ifm.uni-kiel.de (Janko Hauser)
Date: Tue, 22 Dec 1998 09:47:55 +0100 (CET)
Subject: [Matrix-SIG] Core dump in NumPy when multiplying with complex numbers.
In-Reply-To: <367F5268.D1A74FF3@mayo.edu>
References: <199812181700.MAA10743@python.org> <367F5268.D1A74FF3@mayo.edu>
Message-ID: <13951.23596.273123.636738@lisboa.ifm.uni-kiel.de>
Python 1.5.2b1 (#1, Dec 11 1998, 11:30:32) [GCC 2.7.2.3] on linux2
Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
>>> import Numeric
>>> (1+0j)*Numeric.array([1,2])
array([ 1.+0.j, 2.+0.j])
>>>
Debian 2.0+ LLNL6
Python 1.5.2a2 (#2, Oct 20 1998, 10:00:20) [C] on osf1V4
Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
>>> import Numeric
>>> (1+0j)*Numeric.array([1,2])
array([ 1.+0.j, 2.+0.j])
>>>
Alpha Workstation
No problem here.
__Janko
From Takahiro.Oishi@db.com Tue Dec 22 09:08:47 1998
From: Takahiro.Oishi@db.com (Oishi, Takahiro)
Date: Tue, 22 Dec 1998 18:08:47 +0900
Subject: [Matrix-SIG] Core dump in NumPy when multiplying with complex numbers.
References: <199812181700.MAA10743@python.org> <367F5268.D1A74FF3@mayo.edu>
Message-ID: <367F619F.35FD03E9@db.com>
Python 1.5.1 (#29, Dec 17 1998, 13:14:51) [GCC 2.7.2.2] on sunos5
Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
>>> import Numeric
>>> (1+0j)*Numeric.array([1,2])
array([ 1.+0.j, 2.+0.j])
>>>
I have no problem under SPARC either.
T.Oishi
From cgw@pgt.com Tue Dec 22 15:16:16 1998
From: cgw@pgt.com (Charles G Waldman)
Date: Tue, 22 Dec 1998 10:16:16 -0500 (EST)
Subject: [Matrix-SIG] Core dump in NumPy when multiplying with complex numbers.
Message-ID: <13951.47040.302404.837259@janus.pgt.com>
Travis Oliphant writes:
> Python 1.5.1 (#1, Aug 8 1998, 13:33:40) [GCC egcs-2.90.29 980515 (egc
> on linux2
> Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
> >>> import Numeric
> >>> (1+0j)*Numeric.array([1,2])
> Segmentation fault (core dumped)
>
> Does this happen to anyone else or is it compiler-related...?
I'm using the exact same compiler, here's my session:
Python 1.5.2b1 (#2, Dec 22 1998, 00:47:40) [GCC egcs-2.90.29 980515 (egc on linux2
Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
>>> import Numeric
>>> (1+0j)*Numeric.array([1,2])
array([ 1.+0.j, 2.+0.j])
We've got to get to the bottom of these NumPy core-dumps. I'm working
on it...
From cgw@pgt.com Tue Dec 22 17:00:22 1998
From: cgw@pgt.com (Charles G Waldman)
Date: Tue, 22 Dec 1998 12:00:22 -0500 (EST)
Subject: [Matrix-SIG] Re: [PSA MEMBERS] Re: [Matrix-SIG]NumPy core dump
Message-ID: <13951.53286.793480.314569@janus.pgt.com>
Me:
> Now I get a most peculiar
> behavior: no longer is there an immediate core-dump; I get
> a TypeError, as expected, but not until after I enter another line.
> It doesn't seem to matter if the line is blank or contains
> code, as the following examples show:
Marc-Andre Lemburg:
> I don't know anything about NumPy, but the error you describe
> is typical of a situation where C code sets an exception using
> PyExc_SetString() without returning NULL to the caller.
Thanks for the tip. I have (finally) solved this problem.
I had been using a slightly out-of-date version of Python-1.5.2 that I
got via CVS; in this version of ceval.c, eval_code2 failed to
initialize "retval" to NULL, so the "goto fail" lines resulted in
returning a nonsense value. I was just about to report this and
submit a patch when I realized that the current version (2.159)
of ceval.c has this problem fixed already.
I suppose the moral is to do "cvs update" before getting deeply
involved in debugging!
Thanks to Jeremy, Marc-Andre, and all others who offered hints.
From cgw@pgt.com Tue Dec 22 22:49:51 1998
From: cgw@pgt.com (Charles G Waldman)
Date: Tue, 22 Dec 1998 17:49:51 -0500 (EST)
Subject: [Matrix-SIG] Fix for a long-standing NumPy bug
Message-ID: <13952.8719.103233.528981@janus.pgt.com>
Here's a fix for a bug I first reported back in September:
http://www.python.org/pipermail/matrix-sig/1998-September/002337.html
It can be triggered by the following code:
from Numeric import *
a = array((1,2,3,4,5,6,7,8))
a.shape = 2,2,2
x,y = a
x.shape = 4,1
Well, using the wonderful "MALLOC_CHECK_" feature of recent libc
versions (set environment variable MALLOC_CHECK_ to 1) I was able to
trace this down to a free() of an invalid pointer. With that
information in hand, it wasn't too hard to track down the offending
code. Here's a patch:
--- arrayobject.c.orig Tue Dec 22 17:34:28 1998
+++ arrayobject.c Tue Dec 22 17:35:31 1998
@@ -1399,9 +1399,9 @@
if (strcmp(name, "shape") == 0) {
/* This can be made more efficient by copying code from array_reshape if needed */
if ((ap = (PyArrayObject *)PyArray_Reshape(self, op)) == NULL) return -1;
- free(self->dimensions);
+ if (self->flags & OWN_DIMENSIONS) free(self->dimensions);
self->dimensions = ap->dimensions;
- free(self->strides);
+ if (self->flags & OWN_STRIDES) free(self->strides);
self->strides = ap->strides;
self->nd = ap->nd;
self->flags &= ~(OWN_DIMENSIONS | OWN_STRIDES);
From Oliphant.Travis@mayo.edu Wed Dec 23 18:51:28 1998
From: Oliphant.Travis@mayo.edu (Travis Oliphant)
Date: Wed, 23 Dec 1998 12:51:28 -0600 (CST)
Subject: [Matrix-SIG] Using Python1.5.2b1 no longer gives me core dumps
Message-ID: <Pine.LNX.3.96.981223124454.25338A-100000@us2.mayo.edu>
This is an update to a previous post where I described segmentation faults
when trying to multiply complex numbers by an array.
The problem went away when I compiled the Python1.5.2b1 available
from python.org today.
Thanks to those who checked their installations to give me feedback.
----------------------------------------------------
Travis Oliphant 200 First St SW
Rochester MN 55905
Ultrasound Research Lab (507) 286-5293
Mayo Graduate School Oliphant.Travis@mayo.edu
From Takahiro.Oishi@db.com Thu Dec 24 01:04:50 1998
From: Takahiro.Oishi@db.com (Oishi, Takahiro)
Date: Thu, 24 Dec 1998 10:04:50 +0900
Subject: [Matrix-SIG] I couldn't find 'outer' in the name space.
References: <001a01be2db9$7450bac0$f4160218@c1004579-c.plstn1.sfba.home.com>
Message-ID: <36819331.A614286C@db.com>
Hi,
Thank you for the reply.
> Please get ftp://ftp-icf.llnl.gov/pub/python/LLNLDistribution.tgz and
> build Numerical. I don't even recognize the names of the files you
> say you are using; they must be very old.
I can find The file, 'ufuncobject.c' I meant is in ./LLNLPython7/Numerical/Src.
Anyway I have built the package 'LLNLPython7.tgz' but
'outer' couldn't be used as well.
Doesn't any same error happen under your python?
A friend of mine (he is a Japanese!) has a same error.
I'll check my configuraion more.
Src/ufuncobject.c:
...
/* -------------------------------------------------------- */
PyObject *ufunc_outer(PyUFuncObject *self, PyObject *args) {
int i;
...
return ret;
}
...
static struct PyMethodDef ufunc_methods[] = {
{"reduce", (PyCFunction)ufunc_reduce, 1},
{"accumulate", (PyCFunction)ufunc_accumulate, 1},
{"reduceat", (PyCFunction)ufunc_reduceAt, 1},
{"outer", (PyCFunction)ufunc_outer, 1},
{NULL, NULL} /* sentinel */
};
...
As regards,
T.Oishi
From Oliphant.Travis@mayo.edu Wed Dec 23 06:51:54 1998
From: Oliphant.Travis@mayo.edu (Travis E. Oliphant)
Date: Wed, 23 Dec 1998 00:51:54 -0600
Subject: [Matrix-SIG] Outer is a method of ufuncs
Message-ID: <3680930A.9FA6E68F@mayo.edu>
The confusion you are experiencing with outer is because
MLab.py is very old and many of the functions are outdated
and based on old interfaces.
outer is a now method of any ufuncobject (such as mutliply,
divide, add, or any of the binary functions in the cephes
module). Thus, in Mlab.py all references to outer should
be modified.
so sum_cov = sum_cov+outer(multiply,v,v)
should be
sum_cov = sum_cov + multiply.outer(v,v)
I've found other functions in MLab.py that don't work
properly also (diag for example).
Regards,
----------------------------------------------------
Travis Oliphant 200 First St SW
Rochester MN 55905
Ultrasound Research Lab (507) 286-5923
Mayo Graduate School Oliphant.Travis@mayo.edu
From Takahiro.Oishi@db.com Thu Dec 24 08:33:17 1998
From: Takahiro.Oishi@db.com (Oishi, Takahiro)
Date: Thu, 24 Dec 1998 17:33:17 +0900
Subject: [Matrix-SIG] Outer is a method of ufuncs
References: <3680930A.9FA6E68F@mayo.edu>
Message-ID: <3681FC4C.9AE8A17B@db.com>
Hi,
Thank you very much.
You meant a current MLab.py included in LLNLPython7.tgz or other latest
distribution
is not uptodate, right?
Anyway I got to be able to change and use MLab.py, thanks to your
advise.
I editted to change MLab.py a little, bellow:
MLab.py:
34c34
< m = equal(subtract.outer(arange(N), arange(M)), -k)
---
> m = outer(subtract, arange(N), arange(M)).equal(-k)
37c37
< m = equal(subtract.outer(i, range(M)),-k)
---
> m = outer(subtract, i, range(M)).equal(-k)
46c46
< m = subtract.outer(arange(N), arange(M)).greater_equal(-k)
---
> m = outer(subtract, arange(N), arange(M)).greater_equal(-k)
49c49
< m = subtract.outer(i, arange(M)).greater_equal(-k)
---
> m = outer(subtract, i, arange(M)).greater_equal(-k)
155c155
< return c/sqrt(multiply.outer(d,d))
---
> return c/sqrt(outer(multiply, d,d))
162,163c162,163
< sum_cov = sum_cov+multiply.outer(v,v)
< return (sum_cov-len(m)*multiply.outer(mu,mu))/(len(m)-1)
---
> sum_cov = sum_cov+outer(multiply, v,v)
> return (sum_cov-len(m)*outer(multiply,mu,mu))/(len(m)-1)
I think there are better solutions or uptodate MLab.py
Because MLab.py is very convenient for MATLAB and OCTAVE users.
Many thanks,
T.Oishi
From dubois1@llnl.gov Thu Dec 24 15:51:04 1998
From: dubois1@llnl.gov (Paul F. Dubois)
Date: Thu, 24 Dec 1998 07:51:04 -0800
Subject: [Matrix-SIG] Outer is a method of ufuncs
Message-ID: <000e01be2f55$3705b880$f4160218@c1004579-c.plstn1.sfba.home.com>
If someone would mail me a fixed version of Mlab.py I'll put it in the
archive.
This is an example of my previous concern about "higher-level" stuff being
mixed in with NumPy. We have a maintenance problem and the more non-core
stuff we put in the worse it gets. On the other hand, everyone wants a
complete, Matlab/IDL/Basis/Yorick-like collection.
-----Original Message-----
From: Travis E. Oliphant <Oliphant.Travis@mayo.edu>
To: Takahiro.Oishi@db.com <Takahiro.Oishi@db.com>; matrix-sig@python.org
<matrix-sig@python.org>
Date: Wednesday, December 23, 1998 9:27 PM
Subject: [Matrix-SIG] Outer is a method of ufuncs
>The confusion you are experiencing with outer is because
>MLab.py is very old and many of the functions are outdated
>and based on old interfaces.
>
>outer is a now method of any ufuncobject (such as mutliply,
>divide, add, or any of the binary functions in the cephes
>module). Thus, in Mlab.py all references to outer should
>be modified.
>
>so sum_cov = sum_cov+outer(multiply,v,v)
>
>should be
>
>sum_cov = sum_cov + multiply.outer(v,v)
>
>I've found other functions in MLab.py that don't work
>properly also (diag for example).
>
>Regards,
>
>
>----------------------------------------------------
>Travis Oliphant 200 First St SW
> Rochester MN 55905
>Ultrasound Research Lab (507) 286-5923
>Mayo Graduate School Oliphant.Travis@mayo.edu
>
>_______________________________________________
>Matrix-SIG maillist - Matrix-SIG@python.org
>http://www.python.org/mailman/listinfo/matrix-sig
>
>
From Oliphant.Travis@mayo.edu Wed Dec 23 20:19:57 1998
From: Oliphant.Travis@mayo.edu (Travis E. Oliphant)
Date: Wed, 23 Dec 1998 14:19:57 -0600
Subject: [Matrix-SIG] A Fixed version of MLab.py
References: <000e01be2f55$3705b880$f4160218@c1004579-c.plstn1.sfba.home.com>
Message-ID: <3681506D.32F38A16@mayo.edu>
This is a multi-part message in MIME format.
--------------55B8511DC514F83C10E6D0E7
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Here is a fixed version of MLab.py with a couple of additions.
I've tested it a bit, but not extensively.
----------------------------------------------------
Travis Oliphant 200 First St SW
Rochester MN 55905
Ultrasound Research Lab (507) 286-5923
Mayo Graduate School Oliphant.Travis@mayo.edu
--------------55B8511DC514F83C10E6D0E7
Content-Type: text/plain; charset=us-ascii; name="MLab.py"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline; filename="MLab.py"
"""Matlab(tm) compatibility functions.
This will hopefully become a complete set of the basic functions available in
matlab. The syntax is kept as close to the matlab syntax as possible. One
fundamental change is that the first index in matlab varies the fastest (as in
FORTRAN). That means that it will usually perform reductions over columns,
whereas with this object the most natural reductions are over rows. It's perfectly
possible to make this work the way it does in matlab if that's desired.
"""
from Numeric import *
# Elementary Matrices
# zeros is from matrixmodule in C
# ones is from Numeric.py
import RandomArray
def rand(*args):
"""rand(d1,...,dn) returns a matrix of the given dimensions
which is initialized to random numbers from a uniform distribution
in the range [0,1).
"""
return RandomArray.random(args)
def eye(N, M=None, k=0, typecode=None):
"""eye(N, M=N, k=0, typecode=None) returns a N-by-M matrix where the
k-th diagonal is all ones, and everything else is zeros.
"""
if M == None: M = N
if type(M) == type('d'):
typecode = M
M = N
m = equal(subtract.outer(arange(N), arange(M)),-k)
return asarray(m,typecode=typecode)
def tri(N, M=None, k=0, typecode=None):
if M == None: M = N
if type(M) == type('d'):
typecode = M
M = N
m = greater_equal(subtract.outer(arange(N), arange(M)),-k)
return asarray(m,typecode=typecode)
# Matrix manipulation
def diag(v, k=0):
v = asarray(v)
s = v.shape
if len(s)==1:
n = s[0]+abs(k)
if k > 0:
v = concatenate((zeros(k, v.typecode()),v))
elif k < 0:
v = concatenate((v,zeros(-k, v.typecode())))
return eye(n, k=k)*v
elif len(s)==2:
v = add.reduce(eye(s[0], s[1], k=k)*v)
if k > 0: return v[k:]
elif k < 0: return v[:k]
else: return v
def fliplr(m):
m = asarray(m)
if len(m.shape) != 2:
raise ValueError, "Input must be 2-D."
return m[:, ::-1]
def flipud(m):
m = asarray(m)
if len(m.shape) != 2:
raise ValueError, "Input must be 2-D."
return m[::-1]
# reshape(x, m, n) is not used, instead use reshape(x, (m, n))
def rot90(m, k=1):
m = asarray(m)
if len(m.shape) != 2:
raise ValueError, "Input must be 2-D."
k = k % 4
if k == 0: return m
elif k == 1: return transpose(fliplr(m))
elif k == 2: return fliplr(flipud(m))
elif k == 3: return fliplr(transpose(m))
def tril(m, k=0):
return tri(m.shape[0], m.shape[1], k=k, typecode=m.typecode())*m
def triu(m, k=0):
return (1-tri(m.shape[0], m.shape[1], k-1, m.typecode()))*m
# Data analysis
# Basic operations
def max(m):
return maximum.reduce(m)
def min(m):
return minimum.reduce(m)
# Actually from BASIS, but it fits in so naturally here...
def ptp(m):
return max(m)-min(m)
def mean(m):
return add.reduce(m)/len(m)
# sort is done in C but is done row-wise rather than column-wise
def msort(m):
return transpose(sort(transpose(m)))
def median(m):
return msort(m)[m.shape[0]/2]
def std(m):
mu = mean(m)
return sqrt(add.reduce(pow(m-mu,2)))/sqrt(len(m)-1)
def sum(m):
return add.reduce(m)
def cumsum(m):
return add.accumulate(m)
def prod(m):
return multiply.reduce(m)
def cumprod(m):
return multiply.accumulate(m)
def trapz(y, x=None):
"""Integrate f using the trapezoidal rule, where y is f(x).
"""
if x == None: d = 1
else: d = diff(x)
return sum(d * (y[1:]+y[0:-1])/2)
def diff(x, n=1):
"""Discrete difference approximation to the derivative
"""
if n > 1:
return diff(x[1:]-x[:-1], n-1)
else:
return x[1:]-x[:-1]
def corrcoef(x, y=None):
"""The correlation coefficients
"""
c = cov(x, y)
d = diag(c)
return c/sqrt(multiply.outer(d,d))
def cov(m,y=None):
m = asarray(m)
mu = mean(m)
if y != None: m = concatenate((m,y))
sum_cov = 0.0
for v in m:
sum_cov = sum_cov+multiply.outer(v,v)
return (sum_cov-len(m)*multiply.outer(mu,mu))/(len(m)-1.0)
import LinearAlgebra
try:
import cephes
except:
print "Some functions depend on the cephes module which is not on your system."
def exit():
import sys; sys.exit()
def squeeze(a):
b = asarray(a.shape)
b = compress(not_equal(b,1),b)
a.shape = tuple(b)
return
# This window depends on the cephes module for modified bessel function i0
def kaiser(M,beta):
n = arange(0,M)
alpha = (M-1)/2.0
return cephes.i0(beta * sqrt(1-((n-alpha)/alpha)**2))/cephes.i0(beta)
def blackman(M):
n = arange(0,M)
return 0.42-0.5*cos(2*pi*n/M) + 0.08*cos(4*pi*n/M)
def rectangular(M):
return ones((M),)
def bartlett(M):
n = arange(0,M)
return where(less_equal(n,M/2.0),2.0*n/M,2-2.0*n/M)
def hanning(M):
n = arange(0,M)
return 0.5-0.5*cos(2*pi*n/M)
def hamming(M):
n = arange(0,M)
return 0.54-0.46*cos(2*pi*n/M)
def sinc(x):
return where(equal(x,0),1,sin(pi*x)/(pi*x))
def eig(v):
return LinearAlgebra.eigenvectors(v)
def svd(v):
return LinearAlgebra.singular_value_decomposition(v)
--------------55B8511DC514F83C10E6D0E7--
From cgw@pgt.com Tue Dec 29 21:24:30 1998
From: cgw@pgt.com (Charles G Waldman)
Date: Tue, 29 Dec 1998 16:24:30 -0500 (EST)
Subject: [Matrix-SIG] Unapplied patches
Message-ID: <13961.18574.896074.266682@sirius>
>>> from Numeric import *
>>> a = array((0,1,2))
>>> a**2
Traceback (innermost last):
File "<stdin>", line 1, in ?
OverflowError: math range error
>>>
This was discussed back in October - see:
http://www.python.org/pipermail/matrix-sig/1998-October/002356.html
and a patch was proposed. However, the problem is still present in
NumPy 1.7. The Gnuplot.py demo fails due to this problem, since it
does something like x=arange(10); y=x**2; plot(x,y)
I also would like to call attention to the patch posted in
http://www.python.org/pipermail/matrix-sig/1998-December/002504.html
Perhaps it's just because of the busy holiday season, but I haven't
heard back from anyone yet about this patch. Will it be applied to
the next NumPy release?
From dubois1@llnl.gov Tue Dec 29 23:49:58 1998
From: dubois1@llnl.gov (Paul F. Dubois)
Date: Tue, 29 Dec 1998 15:49:58 -0800
Subject: [Matrix-SIG] Unapplied patches
Message-ID: <000801be3385$f21ce740$f4160218@c1004579-c.plstn1.sfba.home.com>
I apologize, I missed it. I have the second message sitting in my NumPyToDo
folder but I didn't do it.
So, the patch isn't in release 8 either.
You can hope for release 9, but then again, the same idiot is in charge.
The first one I just plain lost. I don't even remember it.
-----Original Message-----
From: Charles G Waldman <cgw@pgt.com>
To: matrix-sig@python.org <matrix-sig@python.org>
Date: Tuesday, December 29, 1998 1:29 PM
Subject: [Matrix-SIG] Unapplied patches
>
>>>> from Numeric import *
>>>> a = array((0,1,2))
>>>> a**2
>Traceback (innermost last):
> File "<stdin>", line 1, in ?
>OverflowError: math range error
>>>>
>
>This was discussed back in October - see:
>
>http://www.python.org/pipermail/matrix-sig/1998-October/002356.html
>
>and a patch was proposed. However, the problem is still present in
>NumPy 1.7. The Gnuplot.py demo fails due to this problem, since it
>does something like x=arange(10); y=x**2; plot(x,y)
>
>
>I also would like to call attention to the patch posted in
>
>http://www.python.org/pipermail/matrix-sig/1998-December/002504.html
>
>Perhaps it's just because of the busy holiday season, but I haven't
>heard back from anyone yet about this patch. Will it be applied to
>the next NumPy release?
>
>
>_______________________________________________
>Matrix-SIG maillist - Matrix-SIG@python.org
>http://www.python.org/mailman/listinfo/matrix-sig
>
>