From boyle@pcmdi.llnl.gov Mon Jun 2 16:47:33 1997
From: boyle@pcmdi.llnl.gov (James Boyle)
Date: Mon, 2 Jun 1997 08:47:33 -0700 (PDT)
Subject: [PYTHON MATRIX-SIG] handling missing data values in NumPy
Message-ID: <199706021547.IAA17052@cobra.llnl.gov>
I often deal with time sequences of 2D arrays that have
some values missing at random times. I want to obtain
an average of these sequences along the time axis
taking into account the missing values.
I may just be dense but I do not see any slick way to
accomplish this task without writing an extension
like the add.reduce but checking each element for
a missing value.
Does anyone have any insight to this problem or is an
extension the solution?
Thanks for any information.
Jim Boyle
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From hugunin@mit.edu Mon Jun 2 18:18:35 1997
From: hugunin@mit.edu (Jim Hugunin)
Date: Mon, 2 Jun 1997 13:18:35 -0400
Subject: [PYTHON MATRIX-SIG] handling missing data values in NumPy
Message-ID: <9706021719.AA03501@goldilocks>
> I often deal with time sequences of 2D arrays that have
> some values missing at random times. I want to obtain
> an average of these sequences along the time axis
> taking into account the missing values.
I can tell you how I solve the problem in my own code. No guarantees that this is
appropriate for you though...
>>> from Numeric import *
>>> data = array([1,2,666,4,5])
>>> valid = array([1,1, 0,1,1])
>>> print sum(data*valid)/sum(valid)
3
If you do a lot of this sort of thing, you could encapsulate this way of
representing arrays with missing values into a new python object, but that's
possibly overkill.
-Jim
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From Daniel.Michelson@smhi.se Tue Jun 3 14:37:08 1997
From: Daniel.Michelson@smhi.se (Daniel.Michelson)
Date: Tue, 3 Jun 1997 15:37:08 +0200
Subject: [PYTHON MATRIX-SIG] HDF
Message-ID: <9706031337.AA20283@ax004.smhi.se>
Greetings!
I have a vague recollection of someone mentioning that the LLNL was =
working=20
on a Python interface to the HDF library. Am I right? If so, what shape =
is=20
it in and is it targeted for public release?
curiously, -daniel
+------------------+--------------------------------------------------+
| | Daniel B. Michelson |
| /\ |\ /| | | | | Swedish Meteorological and Hydrological Institute|
| \ | \/ | |__| | | S-601 76 Norrk=F6ping, Sweden =
|
| \ | | | | | | Telephone: +46 11 - 15 84 94 |
| \/ | | | | | | Telefax: +46 11 - 17 02 07 |
| | Internet: Daniel.Michelson@smhi.se |
+------------------+--------------------------------------------------+
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From dubois1@llnl.gov Tue Jun 3 17:10:21 1997
From: dubois1@llnl.gov (Paul F. Dubois)
Date: Tue, 3 Jun 1997 09:10:21 -0700
Subject: [PYTHON MATRIX-SIG] HDF
Message-ID: <9706031613.AA23039@icf.llnl.gov.llnl.gov>
We have a basic interface to the PDB file format. This is a format that is
part of the PACT system developed by Stuart Brown of LLNL. This interface
will be released within the next two weeks; we in fact were just doing the
paperwork yesterday. Unfortunately, we have not done an interface to HDF as
we do not use it.
See coral.llnl.gov/pub/users/pact to get PACT. It is free. PDB is a portable,
self-defining file. It is significantly fast. It allows "directories" inside
the file and can store entire pointered C structures in a
morphology-preserving way. There is both a C and Fortran interface, and a
nice document (which you have to get separately from the same site). It has
had extensive use at LLNL for many years.
See S. A. Brown et. al, "Software for Portable Scientific Data Management",
Computers in Physics 7(3), p 304 (1993) for an article comparing HDF, netCDF,
and PDB, written by people connected with each project.
See S.A. Brown, P. F. Dubois, and D. H. Munro, "Creating and Using PDB
Files", Computers in Physics 9(2), p 173 (1995) for a tutorial.
Our Python interface uses the Numerical Python package to turn data in the
files into NumPy arrays. The interface should be considered a beta when we
release it. It will be part of an "LLNL" release which will help people
create python with NumPy, Gist, PDB, Tk, and C++ support. (The C++ support is
not even an alpha, but there is at least an option in configure to use a C++
main program and add modules written in C++).
Our work has gone more slowly than we had hoped due to an oversubscribed work
force, but it is coming along. Please be patient.
Paul Dubois
Project Leader, Computer Science
X-Division, LLNL
----------
From: Daniel.Michelson <Daniel.Michelson@smhi.se>
To: matrix-sig@python.org
Subject: [PYTHON MATRIX-SIG] HDF
Date: Tuesday, June 03, 1997 6:37 AM
Greetings!
I have a vague recollection of someone mentioning that the LLNL was working
on a Python interface to the HDF library. Am I right? If so, what shape is
it in and is it targeted for public release?
curiously, -daniel
+------------------+--------------------------------------------------+
| | Daniel B. Michelson |
| /\ |\ /| | | | | Swedish Meteorological and Hydrological Institute|
| \ | \/ | |__| | | S-601 76 Norrkvping, Sweden |
| \ | | | | | | Telephone: +46 11 - 15 84 94 |
| \/ | | | | | | Telefax: +46 11 - 17 02 07 |
| | Internet: Daniel.Michelson@smhi.se |
+------------------+--------------------------------------------------+
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
----------
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From furnish@acl.lanl.gov Tue Jun 3 17:23:30 1997
From: furnish@acl.lanl.gov (Geoffrey Furnish)
Date: Tue, 3 Jun 1997 10:23:30 -0600 (MDT)
Subject: [PYTHON MATRIX-SIG] plotting software
In-Reply-To: <199705262001.NAA23212@curlew.geog.ubc.ca>
References: <199705211242.IAA08520@jigster>
<199705220639.BAA05980@tillamook-sharp.eaton.net>
<199705262001.NAA23212@curlew.geog.ubc.ca>
Message-ID: <199706031623.KAA03005@steam.acl.lanl.gov>
Phil Austin writes:
> >>>>> "JWE" == John W Eaton <jwe@bevo.che.wisc.edu> writes:
> JWE> A similar library that is distributed under the GPL is
> JWE> plplot, but I'm not sure that it is being actively maintained
> JWE> these days.
>
> Lack of a good free plotting package is a sticking point for several
> numerically-oriented interpreted languages. There has been some
> discussion of this on the Numeric Python mailing list as well, and
> perhaps a group effort spanning Octave, Python, Perl data language,
> etc. could produce something. We'd like to see the most portable
> possible solution, and the Tk driver in Plplot shows promise in that
> direction, although the move of Geoff Furnish, the Plplot maintainer,
> from Livermore to Los Alamos may have derailed that (there was also
> some talk several months about about merging Plplot with gist, a
> free Livermore graphics package that runs on X under Yorick or
> Python). One person on the Python list is writing plotting software
> in Java to run under Python (see http://estel.uindy.edu/PESSci/ph280/)
I can only speak for PLplot. It has been mentioned here several times
in the last few months. I have twice composed responses and then
deleted them instead of sending them. I'll try again now.
Bottom line: PLplot's Python support does and will continue to exist,
and will even continue to improve. I am using Python and PLplot in my
work, past, present, and future, and PLplot will continue to improve
as a consequence.
I have been extremely reticient to chime in with info about PLplot on
the Python Matrix SIG over the last few months specifically because I
am absolutely sick to death of Pythonians publicly maligning PLplot
for its allegedly poor autoconf support, whilst never bothering to
carefully document their problems and provide them to the PLplot
mailing list where the PLplot maintainer (me) or other PLplot hotshots
(of whom there are several), might have a fighting chance of helping
them. The list of the guilty here is long and grievous, and I am
really sick of it.
I am especially sick of the misplaced condemnation. Writing autoconf
support is *hard* and the Python community seems to be somewhat unique
among autoconf users in terms of their oblivion to the issues,
apparently specifically b/c Python's "auto"-conf doesn't
"automatically" configure very much at all (it checks the system OS
characteristics in order to configure the supported Python modules
which interact with the OS, but it does not do any meaningful search
for auxilliary packages a user might have installed). With Python you
have to manually configure practically everything, which is /much/
more burdensome on the user than is PLplot. The problem is that
autoconf isn't perfect. It is at best an asymptotically converging
system. For 99% of all installed computers systems, autoconf does a
fantastic job. For 1% of the installed computers, the sysadmins have
succeeded in so incomparably screwing things up that autoconf can't
sort it out, and the users on these systems invariably blame the
package maintainer. For instance, at my previous place of employment,
the installation of X on a particular computer that I used a lot, was
so perniciously mishandled that no autoconf version from 2.4 through
2.10 could /automatically/ recognize that the system had X.
Fortunately autoconf suports user overides, and I was abble to pass
configure the --x-includes=/where/they/actually/were flag, to overcome
this annoyance.
PLplot has drawn an extremely disproportionate amount of criticism
from one community--the Python community--for its allegedly
insufferable configuration support. I will be the first to admit that
PLplot's autoconf support has required a lot of attention. Many many
bugs were fixed over the summer/fall '96 timeframe. There were
PLplot snapshots to provide many of these improvemtns to external
PLplot users. I do still have additional fixes which have not yet
been propagated to an externally accessible snapshot. There will be
more PLplot snapshots, and when they show up, people will be able to
benefit from the improvements which have been made.
I cannot schedule these. I am not paid to hack PLplot. I am paid to
do science, and I hack PLplot only in so far as it is undeniably
justifiable in the pursuit of my mandated job functions. As with all
GPL'd software, it is possible to obtain professional support for
PLplot.
Besides the autoconf issues with PLplot, Pythonians have also
complained regularly about uneven support for a dynamically loadable
Python extension module for PLplot. Again, it is no trouble to
compile PLplot with -fpic or the equivalent for your compiler. There
are, however, major issues to be addressed in terms of the total
python integration picture. I certainly view the Python module import
system as very primitive, and nowhere near as sophisticated as that
seen in certain other scripting environments, and have the very strong
sensation that this is poorly appreciated in the Python community.
Python extensions composed of /both/ Python source /and/ compiled
extensions, are not well served by simply "import"-ing a dynloadable
module, since it may be necessary to simultaneously update the
PYTHONPATH to find the *.py files that go with the .so. Support for
this kind of thing is very poor in Python. The PLplot approach to
dealing with this is to favor static builds. Moreover, it is also
very hard to support dynloading of PLplot in combination with its Tk
support, because of the poor design of the tkinter module. I have
found it exceedingly hard to build a Python which seemlessly supports
dynloadable tkmodule with selective inclusion of Tcl extension
modules. The -DWITH_APPINIT thing is woefully inadequate in the face
of the sort of freewheeling dynloadable extension style that Python
seems to want to encourage.
The last time these issues were discussed in this forum, the moderator
told us to quit talking about it, even though the discussion followed
directly from issues relating to numeric plotting extensions, which
are surely germane to this list, and also even though the NumPy
extension itself suffers from many of the same issues. I am flatly
exasperated by the Python community with respect to the prevailing
sentiments about PLplot, and the deafening silence that has followed
is a direct reflection of this exasperation (rather than an indication
that PLplot is currently unsupported). This message is very much an
attempt to bring some closure to the matter in this forum. I do not
intend to go out of my way to publicize PLplot's Python support to the
Python community in the future. I will publicize PLplot's Python
support to the PLplot community.
As a final reference to Phil's comments about a multi-script-language
plotting library, let me just say the following:
PLplot has long aimed to provide the same plotting library across
every interesting programming architecture. This started with support
for multiple compiled language interfaces (fortran, C and C++), and
has proceeded also to cover multiple script languages (Tcl, Scheme,
Python). Perl-dl hasn't been done yet, but it surely will someday.
Same for Java--it's not on the calendar, but it is surely going to
happen one of these days. PLplot is already bound to packages like
Ocatve and Rlab, and similar such support from other packages is
inevitable. Bottom line, if you are a PLplot user, and you start
working in a new environment one day, hopefully you will find that
PLplot is already there, or can be taken there easily. This has been
an enduring characteristic of PLplot for many years, and will continue
irrespective of the prevailing mood of the Python community.
Cheers,
--
Geoffrey Furnish email: furnish@lanl.gov
LANL CIC-19 POOMA/RadTran phone: 505-665-4529 fax: 505-665-7880
"Here are your ball-peen hammers. Now go malleate some heads!" -Jim Morel
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From R.Hooft@EuroMail.com Wed Jun 4 07:58:04 1997
From: R.Hooft@EuroMail.com (Rob Hooft)
Date: Wed, 4 Jun 1997 08:58:04 +0200 (MET DST)
Subject: [PYTHON MATRIX-SIG] HDF
In-Reply-To: <9706031613.AA23039@icf.llnl.gov.llnl.gov>
References: <9706031613.AA23039@icf.llnl.gov.llnl.gov>
Message-ID: <199706040658.IAA22790@nu>
>>>>> "PFD" == Paul F Dubois <dubois1@llnl.gov> writes:
PFD> We have a basic interface to the PDB file format.
That is nice. Did you know that Konrad Hinsen also has an interface to
a PDB file format? But that one is a bit more specialized: the _P_rotein
_D_ata _B_ank file can only be used to store 3D structures of biological
macromolecules... And this database format was already published in 1977!
@Article{bernstein,
author = "F. C. Bernstein and T. F. Koetzle and G. J. B.
Williams and E. F. {Meyer Jr} and M. D. Brice and J. R.
Rodgers and O. Kennard and T. Shimanouchi and M.
Tasumi",
title = "The protein data bank: a computer-based archival file
for macro-molecular structures",
journal = "J. Mol. Biol.",
year = "1977",
volume = "112",
pages = "535--542",
}
Regards,
--
===== R.Hooft@EuroMail.com http://www.Sander.EMBL-Heidelberg.DE/rob/ ==
==== In need of protein modeling? http://www.Sander.EMBL-Heidelberg.DE/whatif/
Validation of protein structures? http://biotech.EMBL-Heidelberg.DE:8400/ ====
== PGPid 0xFA19277D == Use Linux! Free Software Rules The World! =============
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From hinsen@ibs.ibs.fr Wed Jun 4 10:10:03 1997
From: hinsen@ibs.ibs.fr (Konrad Hinsen)
Date: Wed, 4 Jun 1997 11:10:03 +0200
Subject: [PYTHON MATRIX-SIG] HDF
In-Reply-To: <9706031613.AA23039@icf.llnl.gov.llnl.gov> (dubois1@llnl.gov)
Message-ID: <199706040910.LAA24716@lmspc1.ibs.fr>
> See coral.llnl.gov/pub/users/pact to get PACT. It is free. PDB is a portable,
And don't forget to use FTP for that - if you try http, you'll be asked
for a password!
PACT looks quite interesting. Do you plan to do Python interfaces for the
whole package, or just for PDB?
BTW, my major complaint about PDB is its name. In the biology
community, one of the first acronyms that everybody learns is PDB -
the Protein Data Bank (see http://www.pdb.bnl.gov/ for details). The
PDB file format is the most heavily used (and abused) file format in
computational chemistry, and many people have set up their Web
browsers to feed ".pdb" to a PDB viewer. I doubt anyone in my field
would want to use another package called PDB; the confusion would be
too much.
--
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr
Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28
Institut de Biologie Structurale | Fax: +33-4.76.88.54.94
41, av. des Martyrs | Deutsch/Esperanto/English/
38027 Grenoble Cedex 1, France | Nederlands/Francais
-------------------------------------------------------------------------------
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From hinsen@ibs.ibs.fr Wed Jun 4 10:20:15 1997
From: hinsen@ibs.ibs.fr (Konrad Hinsen)
Date: Wed, 4 Jun 1997 11:20:15 +0200
Subject: [PYTHON MATRIX-SIG] plotting software
In-Reply-To: <199706031623.KAA03005@steam.acl.lanl.gov> (message from Geoffrey
Furnish on Tue, 3 Jun 1997 10:23:30 -0600 (MDT))
Message-ID: <199706040920.LAA24754@lmspc1.ibs.fr>
> Python extensions composed of /both/ Python source /and/ compiled
> extensions, are not well served by simply "import"-ing a dynloadable
> module, since it may be necessary to simultaneously update the
> PYTHONPATH to find the *.py files that go with the .so. Support for
> this kind of thing is very poor in Python. The PLplot approach to
This looks rather easy - just import sys and append something to
sys.path. Easily doable even in a C extension. But I am not at all
sure that it's a good idea to change the search path dynamically,
because it could cause unpleasant surprises. For example, a module
in the new path could have the same name as a module somewhere else
in the path. I'd rather have such name clashed static so that I can
find a workaround.
That aside, I do agree that Python's support for installing and
configuring nontrivial extension packages is not perfect. What I'd
like to see is a package system like ni to become standard (i.e. built
into the interpreter and fast), plus a package installation script to
automate whatever has to be done to get the path set correctly.
But in general, the Python installation procedure seems OK to me. What
it does automatically works, and what it can't do automatically it
doesn't try to do. I don't mind a few minutes of manual configuration
(mostly editing Modules/Setup), it's much better than trying to figure
out why some fancy automatic configuration doesn't work.
--
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr
Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28
Institut de Biologie Structurale | Fax: +33-4.76.88.54.94
41, av. des Martyrs | Deutsch/Esperanto/English/
38027 Grenoble Cedex 1, France | Nederlands/Francais
-------------------------------------------------------------------------------
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From dars@soton.ac.uk Wed Jun 4 10:45:24 1997
From: dars@soton.ac.uk (Dave Stinchcombe)
Date: Wed, 4 Jun 1997 10:45:24 +0100 (BST)
Subject: [PYTHON MATRIX-SIG] Singular Values
Message-ID: <199706040945.KAA13667@oak.sucs.soton.ac.uk>
Dear Folks,
can any one tell me the reason why the
LinearAlgerba.singular_value_decomposition routine returns U,S,W^T, where
the matrix being decomposed is U*S*W^T, when it seems more
conventional to return U,S,W ?
Yours gratefully
Dave Stinchcombe
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From hinsen@ibs.ibs.fr Wed Jun 4 11:35:21 1997
From: hinsen@ibs.ibs.fr (Konrad Hinsen)
Date: Wed, 4 Jun 1997 12:35:21 +0200
Subject: [PYTHON MATRIX-SIG] Singular Values
In-Reply-To: <199706040945.KAA13667@oak.sucs.soton.ac.uk> (dars@soton.ac.uk)
Message-ID: <199706041035.MAA25024@lmspc1.ibs.fr>
> can any one tell me the reason why the
> LinearAlgerba.singular_value_decomposition routine returns U,S,W^T, where
> the matrix being decomposed is U*S*W^T, when it seems more
> conventional to return U,S,W ?
The convention with respect to SVD is to return U, S, W^T, because
the product of these matrices is the original matrix. Matlab, for
example, does the same.
--
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr
Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28
Institut de Biologie Structurale | Fax: +33-4.76.88.54.94
41, av. des Martyrs | Deutsch/Esperanto/English/
38027 Grenoble Cedex 1, France | Nederlands/Francais
-------------------------------------------------------------------------------
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From dubois1@llnl.gov Wed Jun 4 16:17:11 1997
From: dubois1@llnl.gov (Paul F. Dubois)
Date: Wed, 4 Jun 1997 08:17:11 -0700
Subject: [PYTHON MATRIX-SIG] HDF
Message-ID: <9706041520.AA07877@icf.llnl.gov.llnl.gov>
----------
> From: Konrad Hinsen <hinsen@ibs.ibs.fr>
> To: dubois1@llnl.gov
> Cc: matrix-sig@python.org
> Subject: Re: [PYTHON MATRIX-SIG] HDF
> Date: Wednesday, June 04, 1997 2:10 AM
>
> > See coral.llnl.gov/pub/users/pact to get PACT. It is free. PDB is a
portable,
>
> And don't forget to use FTP for that - if you try http, you'll be asked
> for a password!
>
> PACT looks quite interesting. Do you plan to do Python interfaces for the
> whole package, or just for PDB?
>
One of the other people at the Lab is working on a Python interface to the
the graphics part of PACT.
I do not know the status of that.
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From amullhau@ix.netcom.com Thu Jun 5 01:18:07 1997
From: amullhau@ix.netcom.com (Andrew P. Mullhaupt)
Date: Wed, 04 Jun 1997 20:18:07 -0400
Subject: [PYTHON MATRIX-SIG] Singular Values
In-Reply-To: <199706041035.MAA25024@lmspc1.ibs.fr>
References: <199706040945.KAA13667@oak.sucs.soton.ac.uk>
Message-ID: <3.0.1.32.19970604201807.006c2424@popd.netcruiser>
At 12:35 PM 6/4/97 +0200, Konrad Hinsen wrote:
>The convention with respect to SVD is to return U, S, W^T, because
>the product of these matrices is the original matrix. Matlab, for
>example, does the same.
This isn't really that much of a convention as a concession to
languages such as C and Fortran which do not easily do the transpose.
S, for example returns V, not V^T, as do some APL versions.
The U and V matrices are normally formed by accumulating elementary
orthogonal transformations, and it is just as easy to form the matrices
or their transposes.
Later,
Andrew Mullhaupt
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From furnish@acl.lanl.gov Thu Jun 5 15:32:13 1997
From: furnish@acl.lanl.gov (Geoffrey Furnish)
Date: Thu, 5 Jun 1997 08:32:13 -0600 (MDT)
Subject: [PYTHON MATRIX-SIG] handling missing data values in NumPy
In-Reply-To: <199706021547.IAA17052@cobra.llnl.gov>
References: <199706021547.IAA17052@cobra.llnl.gov>
Message-ID: <199706051432.IAA02549@steam.acl.lanl.gov>
James Boyle writes:
> I often deal with time sequences of 2D arrays that have
> some values missing at random times. I want to obtain
> an average of these sequences along the time axis
> taking into account the missing values.
>
> I may just be dense but I do not see any slick way to
> accomplish this task without writing an extension
> like the add.reduce but checking each element for
> a missing value.
>
> Does anyone have any insight to this problem or is an
> extension the solution?
>
> Thanks for any information.
I suggest using two arrays. One with abscissa (time) values, and one
with ordinate (the real data) values. You could think of missing data
on regular intervals as being just another form of data on irregular
intervals.
--
Geoffrey Furnish email: furnish@lanl.gov
LANL CIC-19 POOMA/RadTran phone: 505-665-4529 fax: 505-665-7880
"Here are your ball-peen hammers. Now go malleate some heads!" -Jim Morel
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From dars@soton.ac.uk Thu Jun 5 20:50:22 1997
From: dars@soton.ac.uk (Dave Stinchcombe)
Date: Thu, 5 Jun 1997 20:50:22 +0100 (BST)
Subject: [PYTHON MATRIX-SIG] Singular Values
In-Reply-To: <199706041035.MAA25024@lmspc1.ibs.fr> from "Konrad Hinsen" at Jun 4, 97 12:35:21 pm
Message-ID: <199706051950.UAA28284@oak.sucs.soton.ac.uk>
>
> > can any one tell me the reason why the
> > LinearAlgerba.singular_value_decomposition routine returns U,S,W^T, where
> > the matrix being decomposed is U*S*W^T, when it seems more
> > conventional to return U,S,W ?
>
> The convention with respect to SVD is to return U, S, W^T, because
> the product of these matrices is the original matrix. Matlab, for
> example, does the same.
I don't wish to sound ungratefull, as at the moment my work is completely
dependant on NumPy, and his friends. But Matlab , Maple and Octave all
return U,V,W where the matrix is U*V*W^T. The reason I make note is not to
request a change in LinearAlgebra, but simply to discover the logic behind
the current choice. The reason for wanting U, W returned is for ease of
handling by other (in my case C) routines, but there is no great hardship
in adding another line of python to form the transpose.
Matlab online help:
>> help svd
SVD Singular value decomposition.
[U,S,V] = SVD(X) produces a diagonal matrix S, of the same
dimension as X and with nonnegative diagonal elements in
decreasing order, and unitary matrices U and V so that
X = U*S*V'.
Maple online help:
Svd(X,U,V) returns the singular values and the left and right singular
vectors in U and V respectively. The singular vectors together with the
singular values satisfy U'XV = D where U' is the transpose of U and U
is n by n, V is p by p, X is n by p, and D is n by p where D[i,i] is/are
the singular value/values of X.
I hope this letter does not appear rude, it is not meant to be, as I'm very
gratefull for all the work done by this group, indeed currently NumPy is
easily the most effective language I have to hand and work would be an order
of magnitude more painfull without it. I'm just trying to
understand something that niggles me.
Yours
Dave Stinchcombe
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From drh@cse.unl.edu Thu Jun 5 22:03:44 1997
From: drh@cse.unl.edu (Doug Heisterkamp)
Date: Thu, 5 Jun 1997 16:03:44 -0500 (CDT)
Subject: [PYTHON MATRIX-SIG] Singular Values
In-Reply-To: <199706051950.UAA28284@oak.sucs.soton.ac.uk> from "Dave Stinchcombe" at Jun 5, 97 08:50:22 pm
Message-ID: <199706052103.QAA07339@cse.unl.edu>
>
> >
> > > can any one tell me the reason why the
> > > LinearAlgerba.singular_value_decomposition routine returns U,S,W^T, where
> > > the matrix being decomposed is U*S*W^T, when it seems more
> > > conventional to return U,S,W ?
> >
The underlying LAPACK routines return the transpose. Note: if you are using
a complex matrix, the conjugate transpose is returned.
Doug Heisterkamp
drh@cse.unl.edu
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From hinsen@ibs.ibs.fr Fri Jun 6 13:30:12 1997
From: hinsen@ibs.ibs.fr (Konrad Hinsen)
Date: Fri, 6 Jun 1997 14:30:12 +0200
Subject: [PYTHON MATRIX-SIG] Singular Values
In-Reply-To: <199706051950.UAA28284@oak.sucs.soton.ac.uk> (dars@soton.ac.uk)
Message-ID: <199706061230.OAA00387@lmspc1.ibs.fr>
> of magnitude more painfull without it. I'm just trying to
> understand something that niggles me.
Well, the ultimate reason for choosing this arrangement is that
LAPACK returns the matrices this way. After all, it's an arbitrary
convention anyway; there is little objective motivation for doing
it this or that way.
--
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr
Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28
Institut de Biologie Structurale | Fax: +33-4.76.88.54.94
41, av. des Martyrs | Deutsch/Esperanto/English/
38027 Grenoble Cedex 1, France | Nederlands/Francais
-------------------------------------------------------------------------------
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From da@maigret.cog.brown.edu Sat Jun 7 20:08:23 1997
From: da@maigret.cog.brown.edu (David Ascher)
Date: Sat, 7 Jun 1997 15:08:23 -0400 (EDT)
Subject: [PYTHON MATRIX-SIG] request for idioms
Message-ID: <Pine.SGI.3.95q.970607145844.22296A-100000@maigret>
I will be giving a tutorial on Numeric Python at SPAM-6, with Konrad
following up with a tutorial on using Python for science or something like
that. While I have a pretty good idea about most of what I want to talk
about in the three hours, I would appreciate input for one section in
particular. I would like to present common 'idioms' in NumPy, and my
feeling is that those idioms are very field-specific. I've developed a
few doing my own work, but I'm sure y'all have as well. I'm especially
interested in:
* things you came up with which map well to constructs you were used
to from other systems (e.g. Matlab, S, etc.).
* things which are very python-specific
* things which are very cool -- use your own judgment.
Obviously, there's no rush on this. If you can just keep this in mind and
when you come up with something you'd like to share, send it my way. I
feel that such a collection would be a good add-on for the more permanent
online tutorial as well.
If all goes well, I'll be swamped with suggestions and will have to select
among them. I will also probably adjust them to fit within a 'story' for
didactic reasons, but I'll try to acknowledge authorship for the
non-obvious tricks.
Please try to explain what your code does in English -- remember that I
probably have a different background than you do! =)
Cheers,
--david
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From hinsen@ibs.ibs.fr Mon Jun 9 12:53:43 1997
From: hinsen@ibs.ibs.fr (Konrad Hinsen)
Date: Mon, 9 Jun 1997 13:53:43 +0200
Subject: [PYTHON MATRIX-SIG] request for idioms
In-Reply-To: <Pine.SGI.3.95q.970607145844.22296A-100000@maigret> (message from
David Ascher on Sat, 7 Jun 1997 15:08:23 -0400 (EDT))
Message-ID: <199706091153.NAA11258@lmspc1.ibs.fr>
> about in the three hours, I would appreciate input for one section in
> particular. I would like to present common 'idioms' in NumPy, and my
> feeling is that those idioms are very field-specific. I've developed a
I am not sure there are many field-specific idioms. Idioms depend
largely on what a given array represents, e.g. "a table of function
values" or "a 3d function on a grid" or whatever. The kind of data you
want to store in an array is of course somewhat field-specific, but
not that much.
> * things you came up with which map well to constructs you were used
> to from other systems (e.g. Matlab, S, etc.).
Probably most, since our arrays were designed with such systems as
a model. And there are books with idioms around, so maybe you should
use them as a starting point.
> Obviously, there's no rush on this. If you can just keep this in mind and
> when you come up with something you'd like to share, send it my way. I
OK, if I don't forget...
--
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr
Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28
Institut de Biologie Structurale | Fax: +33-4.76.88.54.94
41, av. des Martyrs | Deutsch/Esperanto/English/
38027 Grenoble Cedex 1, France | Nederlands/Francais
-------------------------------------------------------------------------------
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From hinsen@ibs.ibs.fr Mon Jun 9 14:08:48 1997
From: hinsen@ibs.ibs.fr (Konrad Hinsen)
Date: Mon, 9 Jun 1997 15:08:48 +0200
Subject: [PYTHON MATRIX-SIG] FFT question
Message-ID: <199706091308.PAA11463@lmspc1.ibs.fr>
Could someone explain to me how the functions real_fft and
inverse_real_fft in FFT.py are supposed to be used? The first one
returns a complex array, which the second one won't accept as input.
--
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr
Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28
Institut de Biologie Structurale | Fax: +33-4.76.88.54.94
41, av. des Martyrs | Deutsch/Esperanto/English/
38027 Grenoble Cedex 1, France | Nederlands/Francais
-------------------------------------------------------------------------------
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From hugunin@mit.edu Mon Jun 9 17:27:39 1997
From: hugunin@mit.edu (Jim Hugunin)
Date: Mon, 9 Jun 1997 12:27:39 -0400
Subject: [PYTHON MATRIX-SIG] FFT question
Message-ID: <9706091628.AA29771@goldilocks>
> Could someone explain to me how the functions real_fft and
> inverse_real_fft in FFT.py are supposed to be used? The first one
> returns a complex array, which the second one won't accept as input.
These functions are based on the corresponding functionality in FFTPACK.
Both functions are applied to an array of real numbers. In both cases, the result
of the transform will be symetric and so only one half of this result is be
returned.
>From the docs:
---
real_fft(a, n=None, axis=-1)
Will return the n point discrete Fourier transform of the real valued array a. n
defaults to the length of a. This is most efficient for n a power of two. The
returned array will be one half of the symmetric complex transform of the real
array.
---
inverse_real_fft isn't in the documentation yet, but if you replace Fourier
transform with inverse-Fourier transform you'll have it's docs.
Note: These function are both available for efficiency only. There is no reason
that you can't use the ordinary fft function on an array of reals. The result you
get back will be much easier to deal with.
-Jim
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From hinsen@ibs.ibs.fr Tue Jun 10 11:19:11 1997
From: hinsen@ibs.ibs.fr (Konrad Hinsen)
Date: Tue, 10 Jun 1997 12:19:11 +0200
Subject: [PYTHON MATRIX-SIG] FFT question
In-Reply-To: <9706091628.AA29771@goldilocks> (hugunin@mit.edu)
Message-ID: <199706101019.MAA14452@lmspc1.ibs.fr>
> inverse_real_fft isn't in the documentation yet, but if you replace Fourier
> transform with inverse-Fourier transform you'll have it's docs.
So inverse_real_fft should do the inverse of real_fft, right? That's not
what it does at the moment.
> Note: These function are both available for efficiency only. There is no reason
> that you can't use the ordinary fft function on an array of reals. The result you
> get back will be much easier to deal with.
Of course, and I am not planning to use them. I am just preparing the next
part of my Python tutorial, which will cover linear algebra, FFT, etc.
I guess I'll just say nothing about real FFTs!
Konrad.
--
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr
Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28
Institut de Biologie Structurale | Fax: +33-4.76.88.54.94
41, av. des Martyrs | Deutsch/Esperanto/English/
38027 Grenoble Cedex 1, France | Nederlands/Francais
-------------------------------------------------------------------------------
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From R.Hooft@EuroMail.com Tue Jun 10 12:11:22 1997
From: R.Hooft@EuroMail.com (Rob Hooft)
Date: Tue, 10 Jun 1997 13:11:22 +0200 (MET DST)
Subject: [PYTHON MATRIX-SIG] FFT question
In-Reply-To: <199706101019.MAA14452@lmspc1.ibs.fr>
References: <9706091628.AA29771@goldilocks>
<199706101019.MAA14452@lmspc1.ibs.fr>
Message-ID: <199706101111.NAA26146@nu>
>>>>> "KH" == Konrad Hinsen <hinsen@ibs.ibs.fr> writes:
>> inverse_real_fft isn't in the documentation yet, but if you
>> replace Fourier transform with inverse-Fourier transform you'll
>> have it's docs.
KH> So inverse_real_fft should do the inverse of real_fft, right?
KH> That's not what it does at the moment.
Nope, that's not how I understand it.
real_fft : real input --> complex output, Fourier transform
inverse_real_fft: real input --> complex output, inverse Fourier transform
Regards,
Rob Hooft.
--
===== R.Hooft@EuroMail.com http://www.Sander.EMBL-Heidelberg.DE/rob/ ==
==== In need of protein modeling? http://www.Sander.EMBL-Heidelberg.DE/whatif/
Validation of protein structures? http://biotech.EMBL-Heidelberg.DE:8400/ ====
== PGPid 0xFA19277D == Use Linux! Free Software Rules The World! =============
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From hinsen@ibs.ibs.fr Tue Jun 10 13:42:18 1997
From: hinsen@ibs.ibs.fr (Konrad Hinsen)
Date: Tue, 10 Jun 1997 14:42:18 +0200
Subject: [PYTHON MATRIX-SIG] FFT question
In-Reply-To: <199706101111.NAA26146@nu> (R.Hooft@EuroMail.com)
Message-ID: <199706101242.OAA14848@lmspc1.ibs.fr>
> Nope, that's not how I understand it.
>
> real_fft : real input --> complex output, Fourier transform
> inverse_real_fft: real input --> complex output, inverse Fourier transform
I'd call the second one "real_inverse_fft", not "inverse_real_fft", which
to me implies "inverse of real_fft". Moreover, what's the point of real_fft
if there is no straightforward way to get the inverse?
--
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr
Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28
Institut de Biologie Structurale | Fax: +33-4.76.88.54.94
41, av. des Martyrs | Deutsch/Esperanto/English/
38027 Grenoble Cedex 1, France | Nederlands/Francais
-------------------------------------------------------------------------------
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From jbaddor@phy.ulaval.ca Tue Jun 10 14:35:09 1997
From: jbaddor@phy.ulaval.ca (Jean-Bernard ADDOR)
Date: Tue, 10 Jun 1997 09:35:09 -0400 (EDT)
Subject: [PYTHON MATRIX-SIG] FFT question
In-Reply-To: <199706101242.OAA14848@lmspc1.ibs.fr>
Message-ID: <Pine.SUN.3.95.970610092843.16311B-100000@mills>
Konrad HINSEN:
> I'd call the second one "real_inverse_fft", not "inverse_real_fft", which
> to me implies "inverse of real_fft". Moreover, what's the point of real_fft
> if there is no straightforward way to get the inverse?
I have an application where I used real_fft to compute the spectrum of a
very long time serie (length: 2**22), without any need to come back in
real space. I was just happy not to have to use too much memory with
real_fft. From that point of view, I think it could be very useful to have
it in the tutorial.
Jean-Bernard
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From jac@gandalf.llnl.gov Tue Jun 10 15:40:25 1997
From: jac@gandalf.llnl.gov (Jim Crotinger)
Date: Tue, 10 Jun 1997 07:40:25 -0700
Subject: [PYTHON MATRIX-SIG] FFT question
In-Reply-To: <199706101111.NAA26146@nu>
References: <9706091628.AA29771@goldilocks>
<199706101019.MAA14452@lmspc1.ibs.fr>
<199706101111.NAA26146@nu>
Message-ID: <199706101440.HAA17670@gandalf.llnl.gov>
Rob Hooft writes:
>
> real_fft : real input --> complex output, Fourier transform
> inverse_real_fft: real input --> complex output, inverse Fourier transform
Surely not. The latter is just the complex conjugate of the former -
we don't need a seperate function to compute it. Since "real fft"s
usually pack the amplitude of the highest mode (which is phaseless)
into the same complex number as the amplitude of the 0 mode, it is
useful to have an inverse (I haven't tried the Python one - I'm just
assuming that this is the way that it works).
Jim
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From phil@geog.ubc.ca Wed Jun 11 00:26:39 1997
From: phil@geog.ubc.ca (Phil Austin)
Date: Tue, 10 Jun 1997 16:26:39 -0700
Subject: [MATRIX-SIG] Re: [PYTHON MATRIX-SIG] request for idioms
In-Reply-To: <Pine.SGI.3.95q.970607145844.22296A-100000@maigret>
References: <Pine.SGI.3.95q.970607145844.22296A-100000@maigret>
Message-ID: <199706102326.QAA04851@curlew.geog.ubc.ca>
>>>>> "David" == David Ascher <da@maigret.cog.brown.edu> writes:
David> I will be giving a tutorial on Numeric Python at SPAM-6,
...
David> * things you came up with which map well to constructs
David> you were used to from other systems (e.g. Matlab, S,
David> etc.). things which are very python-specific things which
David> are very cool -- use your own judgment.
One thing that impressed me was Konrad's rewrite of our NumPy-extended
netcdf module.
For our initial attempt, we simply implemented the netcdf C++
interface, which set the origin for an array slice using
set_cur. Thus to read a 4x3 slice of an array starting a (1,2)
you did:
a=nc.open('demo.nc', nc.WRITE)
b=a.var('temp')
b.set_cur(1,2)
print b.get(4,3)
In contrast, Konrad's module handles it like this:
a=nc.open('demo.nc', nc.WRITE)
b=a.var('temp')
print b[1:5,2:5]
I don't know of another language that let's you pass subscripts to
a C extension in this way.
Regards, Phil
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From phil@geog.ubc.ca Wed Jun 11 00:26:39 1997
From: phil@geog.ubc.ca (Phil Austin)
Date: Tue, 10 Jun 1997 16:26:39 -0700
Subject: [MATRIX-SIG] Re: [PYTHON MATRIX-SIG] request for idioms
In-Reply-To: <Pine.SGI.3.95q.970607145844.22296A-100000@maigret>
References: <Pine.SGI.3.95q.970607145844.22296A-100000@maigret>
Message-ID: <199706102326.QAA04851@curlew.geog.ubc.ca>
>>>>> "David" == David Ascher <da@maigret.cog.brown.edu> writes:
David> I will be giving a tutorial on Numeric Python at SPAM-6,
...
David> * things you came up with which map well to constructs
David> you were used to from other systems (e.g. Matlab, S,
David> etc.). things which are very python-specific things which
David> are very cool -- use your own judgment.
One thing that impressed me was Konrad's rewrite of our NumPy-extended
netcdf module.
For our initial attempt, we simply implemented the netcdf C++
interface, which set the origin for an array slice using
set_cur. Thus to read a 4x3 slice of an array starting a (1,2)
you did:
a=nc.open('demo.nc', nc.WRITE)
b=a.var('temp')
b.set_cur(1,2)
print b.get(4,3)
In contrast, Konrad's module handles it like this:
a=nc.open('demo.nc', nc.WRITE)
b=a.var('temp')
print b[1:5,2:5]
I don't know of another language that let's you pass subscripts to
a C extension in this way.
Regards, Phil
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From amullhau@ix.netcom.com Wed Jun 11 01:37:25 1997
From: amullhau@ix.netcom.com (Andrew P. Mullhaupt)
Date: Tue, 10 Jun 1997 20:37:25 -0400
Subject: [MATRIX-SIG] Re: [PYTHON MATRIX-SIG] request for idioms
References: <Pine.SGI.3.95q.970607145844.22296A-100000@maigret> <199706102326.QAA04851@curlew.geog.ubc.ca>
Message-ID: <339DF345.32CA@ix.netcom.com>
Phil Austin wrote:
>
> In contrast, Konrad's module handles it like this:
>
> print b[1:5,2:5]
>
> I don't know of another language that let's you pass subscripts to
> a C extension in this way.
It's not that uncommon, presumably it's that you haven't used the
languages where it is possible/easy. The S language, and some of the
post-APL languages do this, and with their more general subscripting as
well.
Of course, this is not to take anything away from Konrad's module.
Later,
Andrew Mullhaupt
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From amullhau@ix.netcom.com Wed Jun 11 01:37:25 1997
From: amullhau@ix.netcom.com (Andrew P. Mullhaupt)
Date: Tue, 10 Jun 1997 20:37:25 -0400
Subject: [MATRIX-SIG] Re: [PYTHON MATRIX-SIG] request for idioms
References: <Pine.SGI.3.95q.970607145844.22296A-100000@maigret> <199706102326.QAA04851@curlew.geog.ubc.ca>
Message-ID: <339DF345.32CA@ix.netcom.com>
Phil Austin wrote:
>
> In contrast, Konrad's module handles it like this:
>
> print b[1:5,2:5]
>
> I don't know of another language that let's you pass subscripts to
> a C extension in this way.
It's not that uncommon, presumably it's that you haven't used the
languages where it is possible/easy. The S language, and some of the
post-APL languages do this, and with their more general subscripting as
well.
Of course, this is not to take anything away from Konrad's module.
Later,
Andrew Mullhaupt
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From acura@vcn.bc.ca Sat Jun 14 22:32:10 1997
From: acura@vcn.bc.ca (Kelvin Lee)
Date: Sat, 14 Jun 1997 14:32:10 -0700 (PDT)
Subject: [MATRIX-SIG] Varimax rotation?
Message-ID: <Pine.GSO.3.95.970614142922.18010C-100000@opus.vcn.bc.ca>
I was wondering if anyone out there has created a script to do
Varimax rotation? I'm using NumPy to perform PCA and would
like to rotate my retained components. Before writing
something myself I wanted to check to see if anyone's already
done this. Failing that, does anyone have a good text that
describes the procedure (the multivariate stats books I have
are a little light describing the actual rotation procedure).
Thanks,
Kelvin
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From arw@dante.mh.lucent.com Mon Jun 16 22:13:16 1997
From: arw@dante.mh.lucent.com (Aaron Watters)
Date: Mon, 16 Jun 1997 17:13:16 -0400
Subject: [MATRIX-SIG] cmp?
Message-ID: <199706162110.RAA06244@dante.mh.lucent.com>
I'm certain there are good reasons
why array cmp works the way it does,
please educate me.
>>> cmp(xx[0],xx[1])
-1
>>> cmp(xx[1],xx[0])
-1
>>> xx
array([[1, 1, 1, 0, 0, 0, 2, 2, 2],
[1, 2, 3, 1, 2, 3, 1, 2, 3]])
what's the rationale here? I want to sort arrays.
If I must write my own cmp function, it'll be really
really slow.
Also
>>> sort(xx)
array([[0, 0, 0, 1, 1, 1, 2, 2, 2],
[1, 1, 1, 2, 2, 2, 3, 3, 3]])
Each sub array is sorted individually. It seems
to me this should be written
array(map(sort, xx))
and in the former the subarrays should be sorted
lexicographically as Python sorts tuples and lists.
Please explain. I'm missing something here.
Sorry if I'm being stupid again.
Thanks, Aaron Watters
ps: the motivation is, for fun, I'm trying
to implement some database style functionality
with the help of Numeric, heading towards
accounting-type applications.
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From hochberg@wwa.com Tue Jun 17 03:16:58 1997
From: hochberg@wwa.com (Timothy A. Hochberg)
Date: Mon, 16 Jun 1997 21:16:58 -0500 (CDT)
Subject: [MATRIX-SIG] cmp?
In-Reply-To: <199706162110.RAA06244@dante.mh.lucent.com>
Message-ID: <Pine.BSI.3.95.970616203911.5516B-100000@shoga.wwa.com>
On Mon, 16 Jun 1997, Aaron Watters wrote:
> I'm certain there are good reasons
> why array cmp works the way it does,
> please educate me.
>
> >>> cmp(xx[0],xx[1])
> -1
> >>> cmp(xx[1],xx[0])
> -1
> >>> xx
> array([[1, 1, 1, 0, 0, 0, 2, 2, 2],
> [1, 2, 3, 1, 2, 3, 1, 2, 3]])
>
> what's the rationale here? I want to sort arrays.
> If I must write my own cmp function, it'll be really
> really slow.
I believe that what we would like to do here is a) return an array that
compares cmp element by element, or b) raise an exception. Since these
are(were) not possible, Jim H. did c) punt. (I believe I heard that 1.5 is
gonna let exceptions be raised in in comparisons, so maybe this will be
"fixed".)
How do you want to compare your arrays? If you just want to compare them
like tuples you could use:
> >>> cmp(tuple(xx[0]),tuple(xx[1]))
If you want to compare the sums of the elements, you could use:
cmp(sum(xx[0]), sum(xx[1]))
or the product of the absolute values of the cosines:
cmp(product(abs(cos(xx[0])), product(abs(cos(xx[1]))))
or if you want to compare element be element look at less, less_equal,
etc.
> Also
>
> >>> sort(xx)
> array([[0, 0, 0, 1, 1, 1, 2, 2, 2],
> [1, 1, 1, 2, 2, 2, 3, 3, 3]])
>
> Each sub array is sorted individually. It seems
> to me this should be written
>
> array(map(sort, xx))
>
> and in the former the subarrays should be sorted
> lexicographically as Python sorts tuples and lists.
>
> Please explain. I'm missing something here.
> Sorry if I'm being stupid again.
Hmmm...again, if you want to sort them lexigraphically like tuples and
lists, you could convert them to tuples and lists and sort them then
convert them back. The numeric functions all operate pretty much the same
way - they operate on the smallest chunk of the matrix that they can and
they do it over and over again. Since that's amazingly unclear, let me try
some examples:
cos(xx) # applies cos to every element of the matrix.
sort(xx) # sorts every sub vector of xx along axis -1 (=1)
sort(xx,0) # sorts every sub vector of xx along axis 0
sum(xx,1) # sum every sub vector along axis 1 - returns array([ 9 18])
determinant(xx) # wouldn't work with your xx, but gets the determinant of
# every sub matrix of xx (imagine xx was 2x3x3)
Now again how do you want to sort your array. If you want to sort
according to some numeric criteria other than lexigraphic ordering, I
would a) construct a sort key using sum, abs, +, *, etc, b) use argsort to
determine the sort ordering, then use take to get the sorted matrix. So,
if I wanted to sort the columns of xx by sum I'd use:
take(xx, argsort(sum(xx)), 1)
Hope that's more enlightening than confusing....
____
/im
+------------------------------------------------+
|Tim Hochberg Research Assistant |
|hochberg <at> wwa.com University of Illinois |
| Electrical Engineering |
+------------------------------------------------+
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From arw@dante.mh.lucent.com Tue Jun 17 14:44:49 1997
From: arw@dante.mh.lucent.com (Aaron Watters)
Date: Tue, 17 Jun 1997 09:44:49 -0400
Subject: [MATRIX-SIG] cmp?
Message-ID: <199706171343.JAA17434@dante.mh.lucent.com>
Thanks for the response
> How do you want to compare your arrays? If you just want to compare them
> like tuples you could use:
> > >>> cmp(tuple(xx[0]),tuple(xx[1]))
> If you want to compare the sums of the elements, you could use:
The only problem with this is that the internal representation
for a tuple of floats must be 12x larger than for a numeric array,
not to mention the speed penalty for all those mallocs.
Of course, for now, this is exactly what I will do.
I suppose for my immediate purpose I'd like to compare arrays
any way so that I could use the list.sort builtin on a list of arrays
and get all equal arrays grouped together. This would require
that if cmp(a1, a2) is negative then cmp(a2, a1) is positive nonzero
and if cmp(a2, a3) is also negative then cmp(a1, a3) is negative,
and a1==a4 iff cmp(a1,a4)==0.
These are the "axioms of cmp" that are assumed by list.sort
and other Python facilities.
Is there any problem with lexicographic ordering? I understand
that cmp() may have no reasonable interpretation from a
numeric perspective per se, but you'd still like to be able to do
"row sorts".
>From a database perspective I'd like to implement the
analogue of SQL "group by", ie, as in
select a,b, avg(d+e)
from array1
group by a,b
The "natural" way to do this would be to
align the rows of array1 in a list of [a,b,d,e]
and then sort the list lexicographically to
group equal a,b pairs together. At least *I*
think it would be natural. -- Aaron Watters
ps: alternatively if there were some way
to get partitions of indices where a and b
were equal this might do just as well in
this instance. I can't figure out how to do
this efficiently for all pairings of a,b at once
(maybe my ignorance).
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From hugunin@javanet.com Tue Jun 17 16:12:31 1997
From: hugunin@javanet.com (Jim Hugunin)
Date: Tue, 17 Jun 1997 11:12:31 -0400
Subject: [MATRIX-SIG] cmp?
Message-ID: <199706171512.LAA16034@kona.javanet.com>
> Is there any problem with lexicographic ordering? I understand
> that cmp() may have no reasonable interpretation from a
> numeric perspective per se, but you'd still like to be able to do
> "row sorts".
Tim H. did a good job of explaining why this isn't done. Because it's
unlikely that cmp will be able to return an array of booleans anytime soon,
what I really want to do is raise an exception here. In order to avoid any
problems with backwards incompatibility I've decided to make cmp as useless
as possible for arrays until I can have it start raising exceptions.
> >From a database perspective I'd like to implement the
> analogue of SQL "group by", ie, as in
>
> select a,b, avg(d+e)
> from array1
> group by a,b
For those of us with minimal SQL knowledge, could you go into a few more
details on what "group by" is supposed to do? A concrete example of where
you'd like to do this (on a database of floats I assume?) would be helpful.
I have the strong feeling that either you really shouldn't be using NumPy
to do this, or you're missing some obvious better approach. Of course,
this might just be my general desire not to admit that a lexocographic cmp
could possibly be valuable for arrays.
Please send more info - Jim
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From hochberg@wwa.com Tue Jun 17 16:13:51 1997
From: hochberg@wwa.com (Timothy A. Hochberg)
Date: Tue, 17 Jun 1997 10:13:51 -0500 (CDT)
Subject: [MATRIX-SIG] cmp?
In-Reply-To: <199706171343.JAA17434@dante.mh.lucent.com>
Message-ID: <Pine.BSI.3.95.970617100035.15104B-100000@shoga.wwa.com>
On Tue, 17 Jun 1997, Aaron Watters wrote:
> Thanks for the response
>
[I give unsatisfactory solution]
[AW says noone understands him]
> I suppose for my immediate purpose I'd like to compare arrays
> any way so that I could use the list.sort builtin on a list of arrays
> and get all equal arrays grouped together. [...]
How about this (barely tested) code:
from Numeric import *
xx = array([[1, 1, 1, 0, 0, 0, 2, 2, 2],
[1, 2, 3, 1, 2, 3, 1, 2, 3],
[1, 6, 3, 1, 2, 3, 1, 2, 3],
[1, 2, 3, 1, 3, 3, 1, 2, 3]])
def aw_sort(A, axis=0):
"""Sort a 2d array along axis."""
A = asarray(A)
if axis: A = transpose(A)
S = shape(A)
if len(S) != 2:
raise ValueError, "array must be 2D."
for i in range(S[1]-1, -1, -1):
A = take(A, argsort(A[:,i]), 0)
if axis: A = transpose(A)
return A
if __name__ == '__main__':
print aw_sort(xx)
print
print aw_sort(xx, 1)
It should give you lexicographic sorting on 2d arrays. Extending it to
arrays of arbitrary dimension is left as an exercise to the reader. How
efficient it is, I don't know.
> Is there any problem with lexicographic ordering? I understand
> that cmp() may have no reasonable interpretation from a
> numeric perspective per se, but you'd still like to be able to do
> "row sorts".
If I recall correctly, there was some concern with doing something that
would be regretted later if functionality for __less__, etc. ever gets
added to Python.
> >From a database perspective I'd like to implement the
> analogue of SQL "group by", ie, as in
>
> select a,b, avg(d+e)
> from array1
> group by a,b
I think I understand this, but I'm a database uninitiate...
> The "natural" way to do this would be to
> align the rows of array1 in a list of [a,b,d,e]
> and then sort the list lexicographically to
> group equal a,b pairs together. At least *I*
> think it would be natural. -- Aaron Watters
Sounds good to me.
> ps: alternatively if there were some way
> to get partitions of indices where a and b
> were equal this might do just as well in
> this instance. I can't figure out how to do
> this efficiently for all pairings of a,b at once
> (maybe my ignorance).
I can't think of one either. Try aw_sort - let me know if it works for
you.
____
/im
+------------------------------------------------+
|Tim Hochberg Research Assistant |
|hochberg <at> wwa.com University of Illinois |
| Electrical Engineering |
+------------------------------------------------+
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From arw@dante.mh.lucent.com Tue Jun 17 18:06:22 1997
From: arw@dante.mh.lucent.com (Aaron Watters)
Date: Tue, 17 Jun 1997 13:06:22 -0400
Subject: [MATRIX-SIG] cmp?
Message-ID: <199706171832.OAA23582@dante.mh.lucent.com>
You're probably right that Numeric deserves
to have an array as the result of cmp. The
desire to generalize cmp has occured to many,
including me.
FWIW what I'm doing now
is using numeric to hold columns of
numeric data where each column
may be of a different format.
Say I have a table with columns
product_id, dept_number, sales expenses
I want to be able to do the analogue of
select product_id, dept_number, sum(sales-expenses)
from sales_info
group by product_id, dept_number
with result columns
product_id, dept_number, sum(sales-expenses) for dept&prod
This would result in a summary report listing
the total sales less expenses for each product
id and department number. To do this I need
some mechanism to group product_id, dept_number
matches together...
I can get by translating arrays to tuples if I must.
An alternative would be to provide an array
lexicographic comparator not in the type structure
(implemented in C and specialized to
arrays for speed). Then I could
write
list_of_arrays.sort(lexicographic)
or whatever.
In sum, I now understand why it works the way it
does, and I am content. -- Aaron Watters
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From ta.short@pti-us.com Tue Jun 17 18:41:03 1997
From: ta.short@pti-us.com (Tom Short)
Date: Tue, 17 Jun 1997 17:41:03 +0000
Subject: [MATRIX-SIG] cmp?
In-Reply-To: <199706171832.OAA23582@dante.mh.lucent.com>
Message-ID: <199706172141.RAA08969@hobbes.pti-us.com>
> select product_id, dept_number, sum(sales-expenses)
> from sales_info
> group by product_id, dept_number
>
> with result columns
>
> product_id, dept_number, sum(sales-expenses) for dept&prod
In Yorick, the "where" function is great for this sort of thing:
> a=[1,2,3,4,5]; b=a*10; c=a*100
> idx=where(a==2&b==20)
> write, a(idx), b(idx), c(idx)
2 20 200
In Python, you can use the same approach, but it is more complex
(primarily because the boolean comparisons have to be done with
functions):
>>> a=array([1,2,3,4,5])
>>> b=10*a
>>> c=100*a
>>> idx = nonzero(where(boolean_and(equal(a,2),equal(b,20)),1,0))
>>> print take(a,idx), take(b,idx), take(c,idx)
2 20 200
- Tom
--------------------------
See the PTI Lightning Photo Gallery: http://www.pti-us.com/pti/consult/dist/photos.htm
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From hinsen@ibs.ibs.fr Wed Jun 18 09:10:36 1997
From: hinsen@ibs.ibs.fr (Konrad Hinsen)
Date: Wed, 18 Jun 1997 10:10:36 +0200
Subject: [MATRIX-SIG] cmp?
In-Reply-To: <199706171832.OAA23582@dante.mh.lucent.com>
(arw@dante.mh.lucent.com)
Message-ID: <199706180810.KAA12706@lmspc1.ibs.fr>
> You're probably right that Numeric deserves
> to have an array as the result of cmp. The
> desire to generalize cmp has occured to many,
> including me.
Generalizing cmp() seems obvious, but creates other problems.
If cmp() returns an array, what should if and while statement
do with the result? One answer is to apply the "check for zero",
but it is not at all clear how that should be implemented for
arrays.
--
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr
Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28
Institut de Biologie Structurale | Fax: +33-4.76.88.54.94
41, av. des Martyrs | Deutsch/Esperanto/English/
38027 Grenoble Cedex 1, France | Nederlands/Francais
-------------------------------------------------------------------------------
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From hugunin@mit.edu Fri Jun 20 20:45:52 1997
From: hugunin@mit.edu (Jim Hugunin)
Date: Fri, 20 Jun 1997 15:45:52 -0400
Subject: [MATRIX-SIG] FYI: My future status (and Numeric Python's)
Message-ID: <9706201946.AA13094@goldilocks.LCS.MIT.EDU>
As of last week I have officially taken a leave of absence from MIT in order to
pursue my ideas relating to Python and Java full-time. This means that I'm likely
to have even less time in the future to spend working on NumPy than I have recently.
The good news is that I consider the basic project essentially finished. As soon as
possible I'll have the last crop of bug reports fixed and I'll be putting out
NumPy1.0 final. After that point I'll probably be putting only minimal effort into
this project. Obviously I'll keep handling bug reports and putting out updated
versions, but I doubt that I'll be pushing forward to add any new features for quite
some time (I think the current feature set is reasonably complete).
The web pages for this project are moving to Starship Python over this weekend.
I'll send out more information when the move is complete to let people know where to
find things.
Just thought you should know - Jim Hugunin - hugunin@javanet.com
PS - I can probably be persuaded to do paid consulting work on Numeric Python if
there is something you really need. Contact me to discuss terms and rates.
-----------Extra stuff for the truly interested----------------
Much of my motivation to look into Python/Java came from my experiences with
creating and distributing Numeric Python. I feel that the three key flaws in NumPy
are:
1) Hard to install
-solved by portable Java bytecodes
2) No portable graph packages (arguable I know)
-all graphics are portable with Java
3) Hard to build python interfaces to existing libraries
-trivial to use Java libraries with Python
-I expect to see Java bindings for almost every major library
in the not too distant future (this one's just a guess).
-Java's features (garbage collection, exceptions, classes)
make writing python extensions FAR easier
I fully intend to move my Numeric Extensions to Java once the rest of the system I'm
building falls into place. I don't think the performance will suffer much at all
(I've even come up with some fun ideas involving run-time generated bytecodes that
could offer some serious performance boosts). I also think that the three most
significant flaws that I mentioned above will go away with such a system.
I also realize that many people out there (probably on this list more than anywhere
else in the Python community) are using Python primarily as a glue between existing
custom C and FORTRAN libraries. In this case, the greater difficulty of writing a
Java interface for your native code vs. a Python interface is probably a big deal.
I also realize that many people on this list run in environments where Java is not
available (most supercomputers). So I'm not claiming that Python/Java is a
universal solution to anything.
Just thought somebody out there might want to know - Jim
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From raptor10@airmail.net Fri Jun 20 23:30:57 1997
From: raptor10@airmail.net (Dominick Pescatore)
Date: Fri, 20 Jun 1997 15:30:57 -0700
Subject: [MATRIX-SIG] pickled sausages
Message-ID: <33AB04A1.3E77@airmail.net>
I am looking for a recipe for pickling sausages. Anyone that could help
me, Iwould appreciate it.
Thank You.
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From da@maigret.cog.brown.edu Fri Jun 20 22:02:26 1997
From: da@maigret.cog.brown.edu (David Ascher)
Date: Fri, 20 Jun 1997 17:02:26 -0400 (EDT)
Subject: [MATRIX-SIG] pickled sausages
In-Reply-To: <33AB04A1.3E77@airmail.net>
Message-ID: <Pine.SGI.3.95q.970620170108.20618A-100000@maigret>
On Fri, 20 Jun 1997, Dominick Pescatore wrote:
> I am looking for a recipe for pickling sausages. Anyone that could help
> me, Iwould appreciate it.
1) Find a glass jar with a lid which makes a good tight seal.
2) Fill it half way with vinegar.
3) Insert the sausage
4) Add more vinegar until it's all the way to the top.
5) Wait 3 years, 2 months, 4 days, 26 hours, 22 minutes and 2 seconds.
6) Open, slice, eat, enjoy.
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From drh@cse.unl.edu Sat Jun 21 01:55:34 1997
From: drh@cse.unl.edu (Doug Heisterkamp)
Date: Fri, 20 Jun 1997 19:55:34 -0500 (CDT)
Subject: [MATRIX-SIG] FYI: My future status (and Numeric Python's)
In-Reply-To: <9706201946.AA13094@goldilocks.LCS.MIT.EDU> from "Jim Hugunin" at Jun 20, 97 03:45:52 pm
Message-ID: <199706210055.TAA28257@cse.unl.edu>
>
>
> As of last week I have officially taken a leave of absence from MIT in order to
> pursue my ideas relating to Python and Java full-time. This means that I'm likely
> to have even less time in the future to spend working on NumPy than I have recently.
>
Thanks Jim, for all of the time and effort that you have put into NumPy. I
find it very useful. I wish you the best of fortune in your Java-Python
project.
Maybe we should set up some kind of formal council or committee to handle
bug report fixes and new feature prototypes. That way you could pass
the bug fixing work to different committee members.
Doug Heisterkamp
drh@cse.unl.edu
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From hinsen@ibs.ibs.fr Mon Jun 23 10:39:09 1997
From: hinsen@ibs.ibs.fr (Konrad Hinsen)
Date: Mon, 23 Jun 1997 11:39:09 +0200
Subject: [MATRIX-SIG] FYI: My future status (and Numeric Python's)
In-Reply-To: <9706201946.AA13094@goldilocks.LCS.MIT.EDU> (hugunin@mit.edu)
Message-ID: <199706230939.LAA31982@lmspc1.ibs.fr>
> The good news is that I consider the basic project essentially finished. As soon as
> possible I'll have the last crop of bug reports fixed and I'll be putting out
> NumPy1.0 final. After that point I'll probably be putting only minimal effort into
Sounds good. I agree that the project is essentially finished, as far as
any project can ever be called finished. Thanks for all the work you did!
> Much of my motivation to look into Python/Java came from my experiences with
> creating and distributing Numeric Python. I feel that the three key flaws in NumPy
> are:
>
> 1) Hard to install
> -solved by portable Java bytecodes
Well, yes, but... I am sure the installation of Python extensions can be
improved without reimplementing Python!
> 2) No portable graph packages (arguable I know)
> -all graphics are portable with Java
Within the limits of Java portability!
> 3) Hard to build python interfaces to existing libraries
> -trivial to use Java libraries with Python
> -I expect to see Java bindings for almost every major library
> in the not too distant future (this one's just a guess).
> -Java's features (garbage collection, exceptions, classes)
> make writing python extensions FAR easier
This is partly true, but I don't share your optimism about the second
point. Besides, Java-C integration is far from standard, and I haven't
even heard about attempts at Java-Fortran integration.
> I fully intend to move my Numeric Extensions to Java once the rest of the system I'm
> building falls into place. I don't think the performance will suffer much at all
That will depend a lot on the quality of Java compilers. But I agree
that performance may be good enough for many NumPy applications, e.g.
for data analysis. On the other hand, one of my major uses of NumPy is
interfacing to C modules that burn lots of CPU times, up to weeks per
program run. I am not at all convinced that Java performance will be
acceptable for such stuff in the near future.
Anyway, all that is *your* problem ;-) We just sit back and watch!
Konrad.
--
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr
Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28
Institut de Biologie Structurale | Fax: +33-4.76.88.54.94
41, av. des Martyrs | Deutsch/Esperanto/English/
38027 Grenoble Cedex 1, France | Nederlands/Francais
-------------------------------------------------------------------------------
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From arw@dante.mh.lucent.com Thu Jun 26 21:15:45 1997
From: arw@dante.mh.lucent.com (Aaron Watters)
Date: Thu, 26 Jun 1997 16:15:45 -0400
Subject: [MATRIX-SIG] reverse of take?
Message-ID: <199706262012.QAA16688@dante.mh.lucent.com>
Okay, stupid question time.
(I think I just subscribed, assuming the mail address I used
made sense... actually I thought I'd subscribed before, but
there ya go...)
Been fiddling with numeric recently and I can't seem
to find or derive an "inverse" to take. Ie, I'd like a
builtin that will set a collection of values scattered about
inside an array.
Ie, I want to draw a circle on a large array of 0 bytes
(without making another large array). solution: compute
the array of indices for the circle and set all the indices
to one. Might look something like this:
set_by_indices (big_byte_array, indices, 1)
Does the equivalent of
for pair in indices:
big_byte_array(pair[0], pair[1]) = 1
but without the Python overhead.
or whatever. Sorry if this kinda thing
is already there. Please educate me. -- Aaron Watters
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From Fredrik Lundh" <fredrik_lundh@ivab.se Thu Jun 26 22:01:29 1997
From: Fredrik Lundh" <fredrik_lundh@ivab.se (Fredrik Lundh)
Date: Thu, 26 Jun 1997 23:01:29 +0200
Subject: [MATRIX-SIG] reverse of take?
Message-ID: <9706262054.AA14002@arnold.image.ivab.se>
>Ie, I want to draw a circle on a large array of 0 bytes
>(without making another large array). solution: compute
>the array of indices for the circle and set all the indices
>to one.
from ImageDraw import ImageDraw
d = ImageDraw(im)
d.setfill(1)
d.ellipse((x0,y0,x1,y1))
Oops, wrong SIG, sorry for the intrusion ;-)
Cheers /F
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From arw@dante.mh.lucent.com Thu Jun 26 22:15:21 1997
From: arw@dante.mh.lucent.com (Aaron Watters)
Date: Thu, 26 Jun 1997 17:15:21 -0400
Subject: [MATRIX-SIG] reverse of take?
Message-ID: <199706262112.RAA20624@dante.mh.lucent.com>
Ok. Just to be perverse, say I want to draw a 1d fractal
border on the huge 2d matrix using only the indices.
(I'll definitely have to look into your imagedraw stuff
tho!) -- Aaron Watters
----------
> From: Fredrik Lundh <fredrik_lundh@ivab.se>
> To: matrix-sig@python.org; Aaron Watters <arw@mh.lucent.com>
> Subject: Re: [MATRIX-SIG] reverse of take?
> Date: Thursday, June 26, 1997 5:01 PM
>
> >Ie, I want to draw a circle on a large array of 0 bytes
> >(without making another large array). solution: compute
> >the array of indices for the circle and set all the indices
> >to one.
>
> from ImageDraw import ImageDraw
>
> d = ImageDraw(im)
> d.setfill(1)
>
> d.ellipse((x0,y0,x1,y1))
>
> Oops, wrong SIG, sorry for the intrusion ;-)
>
> Cheers /F
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From amullhau@ix.netcom.com Thu Jun 26 23:01:23 1997
From: amullhau@ix.netcom.com (Andrew P. Mullhaupt)
Date: Thu, 26 Jun 1997 18:01:23 -0400
Subject: [MATRIX-SIG] reverse of take?
In-Reply-To: <199706262012.QAA16688@dante.mh.lucent.com>
Message-ID: <3.0.1.32.19970626180123.006ce0ec@popd.netcruiser>
At 04:15 PM 6/26/97 -0400, Aaron Watters wrote:
>but without the Python overhead.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>or whatever. Sorry if this kinda thing
>is already there. Please educate me. -- Aaron Watters
What you really want is sometimes called scatter/gather indexing, and
there has been occasional discussion in comp.lang.python. It's pretty
easy to implement well - although there are some clever optimizations
deriving from the vector lattice structure of indexing which should not
be missed. There is a large experience with this sort of thing in the
APL derivative languages. NumPy may be alone among array languages in
not supporting this.
Note that in NumPy the advantage of this is multiplied over the
advantage in 1-dimensional indexing, too.
So join the list of people who want more general indexing in Python.
Later,
Andrew Mullhaupt
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From amullhau@ix.netcom.com Thu Jun 26 23:03:24 1997
From: amullhau@ix.netcom.com (Andrew P. Mullhaupt)
Date: Thu, 26 Jun 1997 18:03:24 -0400
Subject: [MATRIX-SIG] reverse of take?
In-Reply-To: <9706262054.AA14002@arnold.image.ivab.se>
Message-ID: <3.0.1.32.19970626180324.006cf410@popd.netcruiser>
At 11:01 PM 6/26/97 +0200, Fredrik Lundh wrote:
>>Ie, I want to draw a circle on a large array of 0 bytes
>>(without making another large array). solution: compute
>>the array of indices for the circle and set all the indices
>>to one.
>
>from ImageDraw import ImageDraw
>
>d = ImageDraw(im)
>d.setfill(1)
>
>d.ellipse((x0,y0,x1,y1))
>
And if it's not a conic section he wants to draw....?
Later,
Andrew Mullhaupt
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From amullhau@ix.netcom.com Thu Jun 26 23:11:53 1997
From: amullhau@ix.netcom.com (Andrew P. Mullhaupt)
Date: Thu, 26 Jun 1997 18:11:53 -0400
Subject: [MATRIX-SIG] reverse of take?
In-Reply-To: <199706262112.RAA20624@dante.mh.lucent.com>
Message-ID: <3.0.1.32.19970626181153.006ce0ec@popd.netcruiser>
At 05:15 PM 6/26/97 -0400, Aaron Watters wrote:
>Ok. Just to be perverse, say I want to draw a 1d fractal
>border on the huge 2d matrix using only the indices.
Just so. Here's how to do it in S, where z is the huge array,
and the x values of the indices are in a vector x, similarly
for y, and v is the vector of corresponding values to assign.
z[cbind(x, y)] <- v
What this does is create an Nx2 matrix with columns x and y, and then
indexes the array z with them. This kind of scatter is really neat
and one of the attractions of the S language. S has by far the nicest
syntax for indexing of all the array languages, and it is the language
closest to Python in terms of object-orientation and argument syntax.
Note that although S is normally very slow compared to Python, the fact
that powerful array indexing is implemented at the "C" level makes up
for a lot of lost ground.
This could easily be added to NumPy (along with a couple of other really
useful forms of indexing). The long history of successful use of the
highly flexible indexing in S is a 'proof of concept'.
Later,
Andrew Mullhaupt
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From Fredrik Lundh" <fredrik_lundh@ivab.se Fri Jun 27 00:26:02 1997
From: Fredrik Lundh" <fredrik_lundh@ivab.se (Fredrik Lundh)
Date: Fri, 27 Jun 1997 01:26:02 +0200
Subject: [MATRIX-SIG] reverse of take?
Message-ID: <9706262319.AA01993@arnold.image.ivab.se>
>And if it's not a conic section he wants to draw....?
d.polygon(xy), perhaps?
or d.point(xy) to draw an arbitrary point set, where
xy can contain either [x,y,x,y...] or [(x,y),(x,y),...].
(but he did say a circle, didn't he?)
Cheers /F
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From phil@geog.ubc.ca Fri Jun 27 00:24:44 1997
From: phil@geog.ubc.ca (Phil Austin)
Date: Thu, 26 Jun 1997 16:24:44 -0700
Subject: [MATRIX-SIG] reverse of take?
In-Reply-To: <3.0.1.32.19970626181153.006ce0ec@popd.netcruiser>
References: <199706262112.RAA20624@dante.mh.lucent.com>
<3.0.1.32.19970626181153.006ce0ec@popd.netcruiser>
Message-ID: <199706262324.QAA29257@curlew.geog.ubc.ca>
>>>>> "AM" == Andrew P Mullhaupt <amullhau@ix.netcom.com> writes:
AM> This kind of scatter
AM> is really neat and one of the attractions of the S language. S
AM> has by far the nicest syntax for indexing of all the array
AM> languages, and it is the language closest to Python in terms
AM> of object-orientation and argument syntax. Note that although
AM> S is normally very slow compared to Python, the fact that
AM> powerful array indexing is implemented at the "C" level makes
AM> up for a lot of lost ground.
AM> This could easily be added to NumPy (along with a couple of
AM> other really useful forms of indexing). The long history of
AM> successful use of the highly flexible indexing in S is a
AM> 'proof of concept'.
One thing that would be a helpful teaching tool, and a useful part of
a motivating sales pitch for S indexing, would be a concordance
giving a variety of index constructs in S, Matlab, Mathematica, etc., and
their python equivalents. I would volunteer to collate such a list if
people wanted to make submissions. At the moment I'm trying to
translate various legacy S constructs into take, repeat, reduce
combinations--it feels like I'm taking (and failing) some kind of IQ
test.
Phil
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From Johann Hibschman <johann@physics.berkeley.edu> Fri Jun 27 02:07:25 1997
From: Johann Hibschman <johann@physics.berkeley.edu> (Johann Hibschman)
Date: Thu, 26 Jun 1997 18:07:25 -0700 (PDT)
Subject: [MATRIX-SIG] reverse of take?
In-Reply-To: <3.0.1.32.19970626181153.006ce0ec@popd.netcruiser>
Message-ID: <Pine.SUN.3.96.970626175850.4801C-100000@physics2.Berkeley.EDU>
On Thu, 26 Jun 1997, Andrew P. Mullhaupt wrote:
> Just so. Here's how to do it in S, where z is the huge array,
> and the x values of the indices are in a vector x, similarly
> for y, and v is the vector of corresponding values to assign.
>
> z[cbind(x, y)] <- v
[...]
>
> This could easily be added to NumPy (along with a couple of other really
> useful forms of indexing). The long history of successful use of the
> highly flexible indexing in S is a 'proof of concept'.
Hmm. Would it be acceptable to define (in a separate extension module)
a function to do this, or do you feel that the built-in syntax is
necessary?
I haven't done much with the NumPy internals, but I'd think that it would
be almost trivial to write a simple extension to do
multi_set(array, indices, values)
by implementing a C for-loop over the indices. Is there a hidden subtlety
that I'm missing?
- Johann
---
Johann A. Hibschman | Grad student in Physics, working in Astronomy.
johann@physics.berkeley.edu | Probing pulsar pair production processes.
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From sdm7g@Virginia.EDU Fri Jun 27 03:21:38 1997
From: sdm7g@Virginia.EDU (Steven D. Majewski)
Date: Thu, 26 Jun 1997 22:21:38 -0400 (EDT)
Subject: [MATRIX-SIG] reverse of take?
In-Reply-To: <199706262324.QAA29257@curlew.geog.ubc.ca>
Message-ID: <Pine.A32.3.96.970626220256.15861D-100000@elvis.med.Virginia.EDU>
I've only followed this discussion lightly ( in some, but perhaps not
all of it's several incarnations ), but since others have mentioned how
S and some other math languages support scatter-gather, I'll offer one
more language for comparison: ( Mainly because the "plumbing" is more
visible in this example. )
LispStat is a version of Lisp oriented to interactive statistics
and statistical graphics. Some of the capabilities may have been
inspired by S+ and some other numerical and statistical packages.
LispStat extends many Lisp functions to vectorized versions, for example:
>( * 10 ( iseq 10 ))
(0 10 20 30 40 50 60 70 80 90)
>( * ( iseq 10 ) ( iseq 10 ))
(0 1 4 9 16 25 36 49 64 81)
This includes comparison functions, which yield a list of boolean values:
> ( setf x ( normal-rand 10 ))
(0.38107432228578 1.96131809922812 2.36068411946823 1.19178363587238
-0.885422972877445 -0.564994188286439 1.04135377723368 -0.595475370701372
0.941913771141252 0.899280862970745)
> ( > x 0 )
(T T T T NIL NIL T NIL T T)
There is a function 'which', which yields a list of the indexes that are
non-nil:
> ( which ( > x 0 ))
(0 1 2 3 6 8 9)
and a select function that takes a sequence and either an index or
a list of indexes:
( select x 3 )
1.19178363587238
> ( select x ( which ( > x 0 )))
(0.38107432228578 1.96131809922812 2.36068411946823 1.19178363587238
1.04135377723368 0.941913771141252 0.899280862970745)
And which can be used as an argument to SETF:
( setf ( select x ( which ( < x 0 ))) 0 )
0
> x
(0.38107432228578 1.96131809922812 2.36068411946823 1.19178363587238 0 0
1.04135377723368 0 0.941913771141252 0.899280862970745)
The value arg to SETF can be a single value or a list of values,
but if they are both lists, the index list and the value list
have to match in length.
---| Steven D. Majewski (804-982-0831) <sdm7g@Virginia.EDU> |---
---| Department of Molecular Physiology and Biological Physics |---
---| University of Virginia Health Sciences Center |---
---| P.O. Box 10011 Charlottesville, VA 22906-0011 |---
All power corrupts and obsolete power corrupts obsoletely." - Ted Nelson
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From amullhau@ix.netcom.com Fri Jun 27 03:44:59 1997
From: amullhau@ix.netcom.com (Andrew P. Mullhaupt)
Date: Thu, 26 Jun 1997 22:44:59 -0400
Subject: [MATRIX-SIG] reverse of take?
In-Reply-To: <199706262324.QAA29257@curlew.geog.ubc.ca>
References: <3.0.1.32.19970626181153.006ce0ec@popd.netcruiser>
<199706262112.RAA20624@dante.mh.lucent.com>
<3.0.1.32.19970626181153.006ce0ec@popd.netcruiser>
Message-ID: <3.0.1.32.19970626224459.006cd598@popd.netcruiser>
At 04:24 PM 6/26/97 -0700, Phil Austin wrote:
>>>>>> "AM" == Andrew P Mullhaupt <amullhau@ix.netcom.com> writes:
> AM> This kind of scatter
> AM> is really neat and one of the attractions of the S language. S
> AM> has by far the nicest syntax for indexing of all the array
> AM> languages,
> At the moment I'm trying to
>translate various legacy S constructs into take, repeat, reduce
>combinations--it feels like I'm taking (and failing) some kind of IQ
>test.
Emigrating from S is likely to be a frightening experience unless you
have experience with another interpreted language. Although python (with
Numerical Extensions) and S are actually very close in most respects, you
are likely to get hung up on the limited indexing of Python and the very
different semantics of argument processing _despite the identical syntax_.
Until something is done about the indexing you really ought to avoid
_transliterating_ S into python; rather you should recast the algorithm
into a more python-friendly form. Keep in mind that python loops are a
_lot_ less punitive than S loops, and although python uses an obsolete
approach to memory management, it is a _good_ implementation of that approach,
as opposed to S, which uses a combination of bad implementation and demented
design decisions to arrive at what may very well be the worst memory behavior
of any actually useful program. (Or, as my new Director of Technology put it,
"worst memory behavior of any actually useful program not used for testing
memory systems....") This means that you don't need to jump through some of
the same memory-saving hoops with python as with S, and this means that
explicit loops which may contain temporary variables are not catastrophic,
as they are in S.
So although it is a pain to write all that code where a few index objects
used to suffice, python can usually get the job done. You're going to be in
a minority - you're coming to python programming from a language which
offers you a lot more in the way of comfortable operations than python
instead of coming from medieval torture chambers like C or Fortran, which
offer so much less. It does seem, though, that several people have
mentioned moving from S to python recently, so maybe it is a growing minority.
Some of us have thought about doing a data.frame class for python. Would
you be interested?
Later,
Andrew Mullhaupt
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From amullhau@ix.netcom.com Fri Jun 27 04:07:10 1997
From: amullhau@ix.netcom.com (Andrew P. Mullhaupt)
Date: Thu, 26 Jun 1997 23:07:10 -0400
Subject: [MATRIX-SIG] reverse of take?
In-Reply-To: <Pine.SUN.3.96.970626175850.4801C-100000@physics2.Berkeley.
EDU>
References: <3.0.1.32.19970626181153.006ce0ec@popd.netcruiser>
Message-ID: <3.0.1.32.19970626230710.006d5054@popd.netcruiser>
At 06:07 PM 6/26/97 -0700, Johann Hibschman wrote:
>On Thu, 26 Jun 1997, Andrew P. Mullhaupt wrote:
>
>> This could easily be added to NumPy (along with a couple of other really
>> useful forms of indexing). The long history of successful use of the
>> highly flexible indexing in S is a 'proof of concept'.
>
>Hmm. Would it be acceptable to define (in a separate extension module)
>a function to do this, or do you feel that the built-in syntax is
>necessary?
>
>I haven't done much with the NumPy internals, but I'd think that it would
>be almost trivial to write a simple extension to do
>
> multi_set(array, indices, values)
>
>by implementing a C for-loop over the indices. Is there a hidden subtlety
>that I'm missing?
There are a couple points.
First, it _looks_ a lot better to anyone who has used an array language to
do
array[indices] = values
instead. It is also a _lot_ easier to remember, and it does the expected
thing. A function such as "multi_set" may or may not be suggestive to the
programmer.
Consider that the "function style" code for
z[k] = foo(x[k], y[k])
looks something like
multi_set(z, k, foo(multi_get(x, k), multi_get(y, k)))
and we haven't even started to deal with anything slippery.
Second, you want to put this operation where it will be exposed to
inheritance,
etc. so you want it to be a _method_. Since it conceptually replaces an
existing method, it's better to use that syntax. I think you can get this
pretty easily with __setitem__ or __setslice__, so the syntax shouldn't
really be an issue.
Later,
Andrew Mullhaupt
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From hinsen@ibs.ibs.fr Fri Jun 27 10:18:29 1997
From: hinsen@ibs.ibs.fr (Konrad Hinsen)
Date: Fri, 27 Jun 1997 11:18:29 +0200
Subject: [MATRIX-SIG] reverse of take?
In-Reply-To: <3.0.1.32.19970626230710.006d5054@popd.netcruiser>
(amullhau@ix.netcom.com)
Message-ID: <199706270918.LAA15972@lmspc1.ibs.fr>
> existing method, it's better to use that syntax. I think you can get this
> pretty easily with __setitem__ or __setslice__, so the syntax shouldn't
> really be an issue.
Indeed. A single subscript of any type is passed to __getitem__:
x[[1, 3, 5]] --> x.__getitem__([1, 3, 5])
Several subscripts are passed as a tuple:
x[[1, 3, 5], [3, 2, 5]] --> x.__getitem__(([1, 3, 5], [3, 2, 5]))
Of course __setitem__ works in the same way. For C types it's a bit
more complicated since integers are passed via the sequence interface
whereas everything else gets passed via the mapping interface, but
it's not a real problem either.
So the difficult part is not syntax, but implementing the indexing
operations in a general and efficient way.
--
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr
Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28
Institut de Biologie Structurale | Fax: +33-4.76.88.54.94
41, av. des Martyrs | Deutsch/Esperanto/English/
38027 Grenoble Cedex 1, France | Nederlands/Francais
-------------------------------------------------------------------------------
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From motteler@laura.llnl.gov Fri Jun 27 15:37:15 1997
From: motteler@laura.llnl.gov (Zane C. Motteler)
Date: Fri, 27 Jun 1997 07:37:15 -0700 (PDT)
Subject: [MATRIX-SIG] reverse of take?
In-Reply-To: <199706262012.QAA16688@dante.mh.lucent.com>
Message-ID: <Pine.HPP.3.95.970627071050.11723A-100000@laura.llnl.gov>
On Thu, 26 Jun 1997, Aaron Watters wrote:
>Okay, stupid question time.
No such thing.
>Been fiddling with numeric recently and I can't seem
>to find or derive an "inverse" to take. Ie, I'd like a
>builtin that will set a collection of values scattered about
>inside an array.
You and me both.
>or whatever. Sorry if this kinda thing
>is already there. Please educate me. -- Aaron Watters
You're right, Numeric has no such capability. I needed this almost from
the first, and in fact I have written a handy-dandy little C extension
called arrayfnsmodule which contains this and a number of other
Yorick-like capabilities that I found necessary.
Our anonymous ftp site contains an entire python1.4 distribution which
contains not only this module, but also graphics modules which connect
to Gist and Narcisse graphics engines (among other things). The ftp site
is ftp-icf.llnl.gov; look in /ftp/pub/python/busby for
python1.4+patches+NumPy.tgz. Just for the heck of it, I have put
arrayfnsmodule.c.gz into /ftp/pub/python/motteler. Feel free to use
whatever you need.
Zane
-----------------------------------------------------------------------------
Zane C. Motteler, Ph. D. ___________ALSO:_________________
Computer Scientist | Professor Emeritus of Computer|
Lawrence Livermore National Laboratory | Science and Engineering |
P O Box 808, L-472 | California Polytechnic State |
Livermore, CA 94551-9900 | University, San Luis Obispo |
510/423-2143 ---------------------------------
FAX 510/423-9969
zcm@llnl.gov or motteler@laura.llnl.gov or zmottel@calpoly.edu
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From motteler@laura.llnl.gov Fri Jun 27 15:48:50 1997
From: motteler@laura.llnl.gov (Zane C. Motteler)
Date: Fri, 27 Jun 1997 07:48:50 -0700 (PDT)
Subject: [MATRIX-SIG] reverse of take?
In-Reply-To: <Pine.SUN.3.96.970626175850.4801C-100000@physics2.Berkeley.EDU>
Message-ID: <Pine.HPP.3.95.970627074739.11723C-100000@laura.llnl.gov>
On Thu, 26 Jun 1997, Johann Hibschman wrote:
>
>Hmm. Would it be acceptable to define (in a separate extension module)
>a function to do this, or do you feel that the built-in syntax is
>necessary?
>
Been there, done that. See my earlier communication to Aaron Watters.
Zane
-----------------------------------------------------------------------------
Zane C. Motteler, Ph. D. ___________ALSO:_________________
Computer Scientist | Professor Emeritus of Computer|
Lawrence Livermore National Laboratory | Science and Engineering |
P O Box 808, L-472 | California Polytechnic State |
Livermore, CA 94551-9900 | University, San Luis Obispo |
510/423-2143 ---------------------------------
FAX 510/423-9969
zcm@llnl.gov or motteler@laura.llnl.gov or zmottel@calpoly.edu
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From hochberg@wwa.com Fri Jun 27 16:02:35 1997
From: hochberg@wwa.com (Timothy A. Hochberg)
Date: Fri, 27 Jun 1997 10:02:35 -0500 (CDT)
Subject: [MATRIX-SIG] Binaries for NumPy
In-Reply-To: <337998C1.43FFF0F5@linux-magazin.de>
Message-ID: <Pine.BSI.3.95.970627092331.1567B-100000@shoga.wwa.com>
I just upgraded to linux 2.0 on my linux box, and lo and behold I was able
to build NumPy and NumPyLib as clean extensions (something I hadn't been
able to get working before for whatever reason....) And it got me thinking
that if we could provide pre-built clean extensions for common unix
platforms, we could make installing NumPy really simple for people. On
Linux the install instructions would be:
(A distribution consisting of the .so and .py (and .pyc?) files in a
directory called, say, NumPyDist would be provided in a tar.gz file.)
1. Unpack the distribution in some directory we'll call <source> (e.g.,
/usr/local/src).
2. Add the following two lines to the end of your .bashrc and/or
.bash_login files (assuming you use bash, otherwise....)
export PYTHONPATH=$PYTHONPATH:<source>/NumPyDist
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:<source>/NumPyDist
Or something like that...
I think this would really lower the investment in time needed to try out
NumPy and consequently we'd end up with more users. What do other people
think? How many platforms would we be able to round up binaries for? Where
would we put it?
____
/im
+------------------------------------------------+
|Tim Hochberg Research Assistant |
|hochberg <at> wwa.com University of Illinois |
| Electrical Engineering |
+------------------------------------------------+
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From hugunin@javanet.com Fri Jun 27 16:14:36 1997
From: hugunin@javanet.com (Jim Hugunin)
Date: Fri, 27 Jun 1997 11:14:36 -0400
Subject: [MATRIX-SIG] Binaries for NumPy
References: <Pine.BSI.3.95.970627092331.1567B-100000@shoga.wwa.com>
Message-ID: <33B3D8DC.A1DA3DC0@javanet.com>
Timothy A. Hochberg wrote:
> I think this would really lower the investment in time needed to try out
> NumPy and consequently we'd end up with more users. What do other people
> think? How many platforms would we be able to round up binaries for? Where
> would we put it?
This was one of my goals initially, but I got really intimidated when I
realized just how many platforms there were out there. Nonetheless,
doing this for a few major platforms would probably be a good idea.
I'm moving the Numeric Python home pages to the Starship Python web site
real soon now, and I think that would also be an appropriate place from
which to distribute these binaries.
-Jim
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From hinsen@ibs.ibs.fr Fri Jun 27 17:05:52 1997
From: hinsen@ibs.ibs.fr (Konrad Hinsen)
Date: Fri, 27 Jun 1997 18:05:52 +0200
Subject: [MATRIX-SIG] reverse of take?
In-Reply-To: <Pine.HPP.3.95.970627071050.11723A-100000@laura.llnl.gov>
(motteler@laura.llnl.gov)
Message-ID: <199706271605.SAA17557@lmspc1.ibs.fr>
> Our anonymous ftp site contains an entire python1.4 distribution which
> contains not only this module, but also graphics modules which connect
> to Gist and Narcisse graphics engines (among other things). The ftp site
Sounds interesting. Could you perhaps provide that stuff separately?
The chance that I will ever be able to download a full Python
distribution at 100 bytes/second plus interruptions is rather small!
Besides, having to search for additions in a full distribution is
not very convenient.
--
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr
Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28
Institut de Biologie Structurale | Fax: +33-4.76.88.54.94
41, av. des Martyrs | Deutsch/Esperanto/English/
38027 Grenoble Cedex 1, France | Nederlands/Francais
-------------------------------------------------------------------------------
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From acura@vcn.bc.ca Sat Jun 28 00:35:18 1997
From: acura@vcn.bc.ca (Kelvin Lee)
Date: Fri, 27 Jun 1997 16:35:18 -0700 (PDT)
Subject: [MATRIX-SIG] newbie question: convert array of floats to integer
In-Reply-To: <199706271605.SAA17557@lmspc1.ibs.fr>
Message-ID: <Pine.GSO.3.95.970627163258.6314A-100000@opus.vcn.bc.ca>
Hopefully this doesn't sound like too naive a question... What's the
easiest way to change the typecode of an array?
Kelvin
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From amullhau@ix.netcom.com Sat Jun 28 01:20:53 1997
From: amullhau@ix.netcom.com (Andrew P. Mullhaupt)
Date: Fri, 27 Jun 1997 20:20:53 -0400
Subject: [MATRIX-SIG] reverse of take?
In-Reply-To: <199706270918.LAA15972@lmspc1.ibs.fr>
References: <3.0.1.32.19970626230710.006d5054@popd.netcruiser>
Message-ID: <3.0.1.32.19970627202053.006d96dc@popd.netcruiser>
At 11:18 AM 6/27/97 +0200, Konrad Hinsen wrote:
>> existing method, it's better to use that syntax. I think you can get this
>> pretty easily with __setitem__ or __setslice__, so the syntax shouldn't
>> really be an issue.
>
>Indeed. A single subscript of any type is passed to __getitem__:
>
> x[[1, 3, 5]] --> x.__getitem__([1, 3, 5])
>
>Several subscripts are passed as a tuple:
>
> x[[1, 3, 5], [3, 2, 5]] --> x.__getitem__(([1, 3, 5], [3, 2, 5]))
>
>Of course __setitem__ works in the same way. For C types it's a bit
>more complicated since integers are passed via the sequence interface
>whereas everything else gets passed via the mapping interface, but
>it's not a real problem either.
Yes. It looks pretty good. The questions that spring to mind are related
to problems which might arise in implementing a python class which gives
S vector and data.frame functionality. You can provide a vector in S with
a vector of character strings and then use the strings to index the original
vector. Higher dimensional S arrays inherit this property. The cool thing
about S data.frame objects is that they inherit from both S arrays and S
lists, (S lists being pretty much like python dictionaries with the column
names being the keys and the columns being the values). It is a surprisingly
cool object to have around, and so it is something you would want in python.
The reservations about expanding python indexing are related to how much of
this picture can be achieved with various choices for python indexing.
>So the difficult part is not syntax, but implementing the indexing
>operations in a general and efficient way.
That's not really the hard part at all, since it's been done to death in the
APL family of languages. See, for example, Budd's book _An APL Compiler_ for
a taste of what's possible. If you really want to get out the hammer and
tongs,
you can actually implement indexing well enough to beat all but impractically
complicated compiled code - the interpreter gives you a very nice way of
re-using
memory bandwidth strategies which are too difficult for general application in
C or Fortran. This is one of the opportunities for interpreters to be
faster than
some well written compiled code, although this doesn't seem to be generally
appreciated outside the post-APL community.
Later,
Andrew Mullhaupt
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From da@maigret.cog.brown.edu Sat Jun 28 03:34:06 1997
From: da@maigret.cog.brown.edu (David Ascher)
Date: Fri, 27 Jun 1997 22:34:06 -0400 (EDT)
Subject: [MATRIX-SIG] reverse of take?
In-Reply-To: <3.0.1.32.19970627202053.006d96dc@popd.netcruiser>
Message-ID: <Pine.SGI.3.95q.970627223312.8214C-100000@maigret>
On Fri, 27 Jun 1997, Andrew P. Mullhaupt wrote:
> At 11:18 AM 6/27/97 +0200, Konrad Hinsen wrote:
> >So the difficult part is not syntax, but implementing the indexing
> >operations in a general and efficient way.
>
> That's not really the hard part at all, since it's been done to death in the
> APL family of languages.
So what *is* the hard part?
--da
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From jbaddor@phy.ulaval.ca Sat Jun 28 14:57:43 1997
From: jbaddor@phy.ulaval.ca (Jean-Bernard ADDOR)
Date: Sat, 28 Jun 1997 09:57:43 -0400 (EDT)
Subject: [MATRIX-SIG] newbie question: convert array of floats to integer
In-Reply-To: <Pine.GSO.3.95.970627163258.6314A-100000@opus.vcn.bc.ca>
Message-ID: <Pine.SUN.3.95.970628095617.13999E-100000@mills>
try somethink like:
new_array = Numeric.array(old_array, Numeric.Integer)
A bientot,
Jean-Bernard
On Fri, 27 Jun 1997, Kelvin Lee wrote:
> Hopefully this doesn't sound like too naive a question... What's the
> easiest way to change the typecode of an array?
>
> Kelvin
>
>
>
> _______________
> MATRIX-SIG - SIG on Matrix Math for Python
>
> send messages to: matrix-sig@python.org
> administrivia to: matrix-sig-request@python.org
> _______________
>
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From hinsen@ibs.ibs.fr Mon Jun 30 10:29:13 1997
From: hinsen@ibs.ibs.fr (Konrad Hinsen)
Date: Mon, 30 Jun 1997 11:29:13 +0200
Subject: [MATRIX-SIG] newbie question: convert array of floats to integer
In-Reply-To: <Pine.SUN.3.95.970628095617.13999E-100000@mills> (message from
Jean-Bernard ADDOR on Sat, 28 Jun 1997 09:57:43 -0400 (EDT))
Message-ID: <199706300929.LAA27662@lmspc1.ibs.fr>
> try somethink like:
>
> new_array = Numeric.array(old_array, Numeric.Integer)
That will likely produce an error message. Use old_array.astype(Integer):
>>> from Numeric import *
from Numeric import *
>>> a = array([1.,2.])
a = array([1.,2.])
>>> b = array(a, Int)
b = array(a, Int)
Traceback (innermost last):
File "<stdin>", line 1, in ?
TypeError: Array can not be safely cast to required type
>>> b = a.astype(Int)
b = a.astype(Int)
Konrad.
--
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr
Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28
Institut de Biologie Structurale | Fax: +33-4.76.88.54.94
41, av. des Martyrs | Deutsch/Esperanto/English/
38027 Grenoble Cedex 1, France | Nederlands/Francais
-------------------------------------------------------------------------------
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From hinsen@ibs.ibs.fr Mon Jun 30 11:00:27 1997
From: hinsen@ibs.ibs.fr (Konrad Hinsen)
Date: Mon, 30 Jun 1997 12:00:27 +0200
Subject: [MATRIX-SIG] reverse of take?
In-Reply-To: <Pine.SGI.3.95q.970627223312.8214C-100000@maigret> (message from
David Ascher on Fri, 27 Jun 1997 22:34:06 -0400 (EDT))
Message-ID: <199706301000.MAA27739@lmspc1.ibs.fr>
> On Fri, 27 Jun 1997, Andrew P. Mullhaupt wrote:
>
> > At 11:18 AM 6/27/97 +0200, Konrad Hinsen wrote:
> > >So the difficult part is not syntax, but implementing the indexing
> > >operations in a general and efficient way.
> >
> > That's not really the hard part at all, since it's been done to death in the
> > APL family of languages.
>
> So what *is* the hard part?
As always: Finding someone to do the actual work!
--
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr
Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28
Institut de Biologie Structurale | Fax: +33-4.76.88.54.94
41, av. des Martyrs | Deutsch/Esperanto/English/
38027 Grenoble Cedex 1, France | Nederlands/Francais
-------------------------------------------------------------------------------
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From hinsen@ibs.ibs.fr Mon Jun 30 12:51:58 1997
From: hinsen@ibs.ibs.fr (Konrad Hinsen)
Date: Mon, 30 Jun 1997 13:51:58 +0200
Subject: [MATRIX-SIG] reverse of take?
In-Reply-To: <3.0.1.32.19970627202053.006d96dc@popd.netcruiser>
(amullhau@ix.netcom.com)
Message-ID: <199706301151.NAA28052@lmspc1.ibs.fr>
> Yes. It looks pretty good. The questions that spring to mind are related
> to problems which might arise in implementing a python class which gives
> S vector and data.frame functionality. You can provide a vector in S with
> a vector of character strings and then use the strings to index the original
> vector. Higher dimensional S arrays inherit this property. The cool thing
For non-integer indexing I usually use a dictionary associated with
an array, which stores the mapping from the generalized index to an
integer. It wouldn't be hard to wrap this up in a class. Well, what
the heck, let's do it:
import Numeric
class NamedIndexArray:
def __init__(self, type, *indices):
self.array = Numeric.zeros(tuple(map(len, indices)), type)
self.indices = []
for index_list in indices:
dict = {}
self.indices.append(dict)
for i in range(len(index_list)):
dict[index_list[i]] = i
def __getitem__(self, item):
if type(item) != type(()):
item = (item, )
index = tuple(map(lambda d, i: d[i], self.indices, item))
return self.array[index]
def __setitem__(self, item, value):
if type(item) != type(()):
item = (item, )
index = tuple(map(lambda d, i: d[i], self.indices, item))
self.array[index] = value
days = NamedIndexArray(Numeric.Int, ['Jan', 'Feb', 'Mar'])
days['Jan'] = 31
days['Feb'] = 28
days['Mar'] = 31
There's more luxury to add, but that's not difficult either. The only
restriction is that keys should not be tuples, since there is no
difference between a[(2, 3)] and a[2, 3].
> That's not really the hard part at all, since it's been done to death in the
> APL family of languages. See, for example, Budd's book _An APL Compiler_ for
Right, but knowledge about the techniques seems to be limited to that
community, which is pretty small. It takes some effort to dig it out
and write the actual code...
--
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr
Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28
Institut de Biologie Structurale | Fax: +33-4.76.88.54.94
41, av. des Martyrs | Deutsch/Esperanto/English/
38027 Grenoble Cedex 1, France | Nederlands/Francais
-------------------------------------------------------------------------------
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From arw@dante.mh.lucent.com Mon Jun 30 15:45:21 1997
From: arw@dante.mh.lucent.com (Aaron Watters)
Date: Mon, 30 Jun 1997 10:45:21 -0400
Subject: [MATRIX-SIG] reverse of take?
Message-ID: <199706301443.KAA23238@dante.mh.lucent.com>
> You're right, Numeric has no such capability. I needed this almost from
> the first, and in fact I have written a handy-dandy little C extension
> called arrayfnsmodule which contains this and a number of other
> Yorick-like capabilities that I found necessary.
I don't know what issues are here (licensing, whatever) but I vote
that arrayfnsmodule be added as to the standard Numeric distribution
for this capability alone, ala the Fourier stuff. It's pretty important to
my mind.
Take should have an inverse, and in my case I am fiddling during
spare moments on a win95 box, and I don't have time right now
to learn how to build other peoples stuff there. I suspect old S
hackers might run into something like this in NumPy and say "huh?"
and go back to S.
Speaking as an arrogant newbie. -- Aaron Watters
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From hinsen@ibs.ibs.fr Mon Jun 30 16:17:23 1997
From: hinsen@ibs.ibs.fr (Konrad Hinsen)
Date: Mon, 30 Jun 1997 17:17:23 +0200
Subject: [MATRIX-SIG] reverse of take?
In-Reply-To: <199706301443.KAA23238@dante.mh.lucent.com>
(arw@dante.mh.lucent.com)
Message-ID: <199706301517.RAA28804@lmspc1.ibs.fr>
> I don't know what issues are here (licensing, whatever) but I vote
> that arrayfnsmodule be added as to the standard Numeric distribution
> for this capability alone, ala the Fourier stuff. It's pretty important to
> my mind.
Fine, then get and use it. I wouldn't put it into the standard
distribution before a definite syntax for list indexing has been
agreed on and implemented.
--
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr
Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28
Institut de Biologie Structurale | Fax: +33-4.76.88.54.94
41, av. des Martyrs | Deutsch/Esperanto/English/
38027 Grenoble Cedex 1, France | Nederlands/Francais
-------------------------------------------------------------------------------
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From arw@dante.mh.lucent.com Mon Jun 30 17:49:52 1997
From: arw@dante.mh.lucent.com (Aaron Watters)
Date: Mon, 30 Jun 1997 12:49:52 -0400
Subject: [MATRIX-SIG] reverse of take?
Message-ID: <199706301646.MAA01562@dante.mh.lucent.com>
As was mentioned, I don't believe syntax is
an issue -- once you have the Python builtin
functionality you can grow your own syntax
using the __getattr__ and __setattr__ hooks
at the Python level. What we need is a nice
builtin, which I think Zane has already
implemented.
-- Aaron Watters
----------
> From: Konrad Hinsen <hinsen@ibs.ibs.fr>
> To: arw@mh.lucent.com
> Cc: motteler@laura.llnl.gov; matrix-sig@python.org
> Subject: Re: [MATRIX-SIG] reverse of take?
> Date: Monday, June 30, 1997 11:17 AM
>
> > I don't know what issues are here (licensing, whatever) but I vote
> > that arrayfnsmodule be added as to the standard Numeric distribution
> > for this capability alone, ala the Fourier stuff. It's pretty important to
> > my mind.
>
> Fine, then get and use it. I wouldn't put it into the standard
> distribution before a definite syntax for list indexing has been
> agreed on and implemented.
> --
> -------------------------------------------------------------------------------
> Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr
> Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28
> Institut de Biologie Structurale | Fax: +33-4.76.88.54.94
> 41, av. des Martyrs | Deutsch/Esperanto/English/
> 38027 Grenoble Cedex 1, France | Nederlands/Francais
> -------------------------------------------------------------------------------
>
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From hinsen@ibs.ibs.fr Mon Jun 30 18:08:35 1997
From: hinsen@ibs.ibs.fr (Konrad Hinsen)
Date: Mon, 30 Jun 1997 19:08:35 +0200
Subject: [MATRIX-SIG] reverse of take?
In-Reply-To: <199706301646.MAA01562@dante.mh.lucent.com>
(arw@dante.mh.lucent.com)
Message-ID: <199706301708.TAA29271@lmspc1.ibs.fr>
> As was mentioned, I don't believe syntax is
> an issue -- once you have the Python builtin
> functionality you can grow your own syntax
> using the __getattr__ and __setattr__ hooks
> at the Python level. What we need is a nice
Not for arrays, which are C types! And these arrays are typically used
directly, without a Python wrapper. So getting the syntax right means
modifying arrayobject.c and putting the new code into the right module
etc. Probably not much work, but it has to be done, and should be
synchronized with possible changes that Jim has applied to the same
module recently.
Konrad.
--
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr
Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28
Institut de Biologie Structurale | Fax: +33-4.76.88.54.94
41, av. des Martyrs | Deutsch/Esperanto/English/
38027 Grenoble Cedex 1, France | Nederlands/Francais
-------------------------------------------------------------------------------
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From arw@dante.mh.lucent.com Mon Jun 30 19:45:53 1997
From: arw@dante.mh.lucent.com (Aaron Watters)
Date: Mon, 30 Jun 1997 14:45:53 -0400
Subject: [MATRIX-SIG] reverse of take?
Message-ID: <199706301843.OAA03287@dante.mh.lucent.com>
> Not for arrays, which are C types! And these arrays are typically used
> directly, without a Python wrapper. So getting the syntax right means
> modifying arrayobject.c and putting the new code into the right module
> etc. Probably not much work, but it has to be done, and should be
> synchronized with possible changes that Jim has applied to the same
> module recently.
okey dokey. (sorry 2 beers at lunch).
I propose that arraytype[arraytype] automatically be interpreted
as (1) gather as a value (2) scatter as an l-value. I apologize if
this is obviously dumb, incompatible with existing usage, or
already proposed (I'm new, remember, don't hurt me!) :).
-- Aaron Watters
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________
From motteler@laura.llnl.gov Mon Jun 30 20:32:41 1997
From: motteler@laura.llnl.gov (Zane C. Motteler)
Date: Mon, 30 Jun 1997 12:32:41 -0700 (PDT)
Subject: [MATRIX-SIG] reverse of take?
In-Reply-To: <199706301843.OAA03287@dante.mh.lucent.com>
Message-ID: <Pine.HPP.3.95.970630123043.25973A-100000@laura.llnl.gov>
On Mon, 30 Jun 1997, Aaron Watters wrote:
>okey dokey. (sorry 2 beers at lunch).
>I propose that arraytype[arraytype] automatically be interpreted
>as (1) gather as a value (2) scatter as an l-value. I apologize if
>this is obviously dumb, incompatible with existing usage, or
>already proposed (I'm new, remember, don't hurt me!) :).
Aaron,
I agree with you and I'm not even new. My "scatter" is not so
general, however. With a little work it could be. Now all I need
is the time.
Zane
-----------------------------------------------------------------------------
Zane C. Motteler, Ph. D. ___________ALSO:_________________
Computer Scientist | Professor Emeritus of Computer|
Lawrence Livermore National Laboratory | Science and Engineering |
P O Box 808, L-472 | California Polytechnic State |
Livermore, CA 94551-9900 | University, San Luis Obispo |
510/423-2143 ---------------------------------
FAX 510/423-9969
zcm@llnl.gov or motteler@laura.llnl.gov or zmottel@calpoly.edu
_______________
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________