From alawhead@vcn.bc.ca Fri Aug 7 06:19:28 1998
From: alawhead@vcn.bc.ca (Alexander Lawhead)
Date: Thu, 6 Aug 1998 22:19:28 -0700 (PDT)
Subject: [Matrix-SIG] 1-d smoother?
Message-ID:
Has anyone implemented any form of 1-d smoother using NumPy (loess,
smoothing spline, kernel smoother etc)? I have need of such a beast and
would rather be lazy and use someone else's code than do it myself! :)
Thanks in advance!
Alexander
From janne@avocado.pc.helsinki.fi Sat Aug 8 22:22:09 1998
From: janne@avocado.pc.helsinki.fi (Janne Sinkkonen)
Date: 09 Aug 1998 00:22:09 +0300
Subject: [Matrix-SIG] Numerical Python in Alpha Linux, is it ok?
Message-ID:
Just to make sure I will not have problems with the most important tool:
is the combination of NumPy, Gist and AXP Linux working flawlessly, or
should I expect some kind of problems?
--
Janne
From ffjhl@aurora.uaf.edu Sat Aug 8 22:42:03 1998
From: ffjhl@aurora.uaf.edu (Jonah Lee)
Date: Sat, 8 Aug 1998 13:42:03 -0800 (AKDT)
Subject: [Matrix-SIG] Numerical Python in Alpha Linux, is it ok?
In-Reply-To:
Message-ID:
On 9 Aug 1998, Janne Sinkkonen wrote:
>
> Just to make sure I will not have problems with the most important tool:
> is the combination of NumPy, Gist and AXP Linux working flawlessly, or
> should I expect some kind of problems?
I have NumPy compiled on AXP Linux just fine. I have not installed Gist on
that system but there should not be any problems.
>
> --
> Janne
>
> _______________________________________________
> Matrix-SIG maillist - Matrix-SIG@python.org
> http://www.python.org/mailman/listinfo/matrix-sig
>
>
Regards,
Jonah
From mhagger@blizzard.harvard.edu Tue Aug 11 01:02:49 1998
From: mhagger@blizzard.harvard.edu (Michael Haggerty)
Date: Mon, 10 Aug 1998 20:02:49 -0400
Subject: [Matrix-SIG] repeated transposes
Message-ID: <9808110002.AA22223@tsunami.harvard.edu>
Hi,
There is a feature of the Numeric module that is somewhat
inconvenient, and can adversely affect performance. Namely, that
Numeric doesn't realize that a matrix can always be transposed without
copying data, and that a matrix that has been transposed twice is back
in `contiguous' storage format.
>>> M = array([[1,2],[3,4]])
>>> Numeric.transpose(Numeric.transpose(M)).iscontiguous()
0
Why is this case important? Often Numeric is used to interface to
large C or Fortran matrices. But Fortran stores matrices column-wise,
whereas Numeric stores them (by default) row-wise. No problem:
whenever you receive a matrix from Fortran, you transpose() it; and
before you pass any matrix back to Fortran, you transpose() it again.
But since Numeric isn't fastidious about keeping track of when a
matrix is stored contiguously, the double transpose causes the matrix
to be copied two extra times (once on entry to the second transpose
operation, which copies it into the *wrong* order; a second time when
putting it back into Fortran order.
But there is no need for transpose ever to copy data--transposes can
always be done by adjusting dimensions and strides--and in fact, the
current algorithm is general enough to work with non-contiguous
matrices. The only trick is to get it to realize when it has just put
a matrix *back* into contiguous order and set the CONTIGUOUS bit back
on.
The following patch to multiarraymodule.c does that. With this patch,
>>> m = array([[1,2,3],[4,5,6]])
>>> m
array([[1, 2, 3],
[4, 5, 6]])
>>> n = transpose(m)
>>> n.iscontiguous()
0
>>> o = transpose(n)
>>> o
array([[1, 2, 3],
[4, 5, 6]])
>>> o.iscontiguous()
1
>>> o[1,0] = 0
>>> o
array([[1, 2, 3],
[0, 5, 6]])
>>> n
array([[1, 0],
[2, 5],
[3, 6]])
>>> m
array([[1, 2, 3],
[0, 5, 6]])
So you can see that now all three arrays refer to the same memory, and
that o is recognized to be contiguous.
There might be other Numeric algorithms that can be implemented in a
way that they can deal with noncontiguous data. This way, the copying
of large matrices can be avoided as much as possible. The function
provided in the patch below, `array_really_contiguous()', might be
useful for that purpose.
See patch below...
Yours,
Michael
--
Michael Haggerty
mhagger@blizzard.harvard.edu
========================= Patch ==============================
*** ../../../LLNLPython4/Numerical/Src/multiarraymodule.c Fri Jun 19 11:27:18 1998
--- multiarraymodule.c Mon Aug 10 19:10:04 1998
***************
*** 115,120 ****
--- 115,136 ----
return NULL;
}
+ /* Check whether the given array is stored contiguously (row-wise) in
+ memory. */
+ static int array_really_contiguous(PyArrayObject *ap) {
+ int sd;
+ int i;
+
+ sd = ap->descr->elsize;
+ for (i = ap->nd-1; i >= 0; --i) {
+ if (ap->dimensions[i] == 0) return 1; /* contiguous by definition */
+ if (ap->strides[i] != sd) return 0;
+ sd *= ap->dimensions[i];
+ }
+ return 1;
+ }
+
+ /* Changed to be able to deal with non-contiguous arrays. */
extern PyObject *PyArray_Transpose(PyArrayObject *ap, PyObject *op) {
long *axes, axis;
int i, n;
***************
*** 135,145 ****
--- 151,164 ----
permutation[i] = axis;
}
+ /* this allocates memory for dimensions and strides (but fills them
+ incorrectly), sets up descr, and points data at ap->data. */
ret = (PyArrayObject *)PyArray_FromDimsAndData(n, permutation,
ap->descr->type_num,
ap->data);
if (ret == NULL) goto fail;
+ /* point at true owner of memory: */
ret->base = (PyObject *)ap;
Py_INCREF(ap);
***************
*** 147,154 ****
ret->dimensions[i] = ap->dimensions[permutation[i]];
ret->strides[i] = ap->strides[permutation[i]];
}
! ret->flags &= ~CONTIGUOUS;
!
PyArray_Free(op, (char *)axes);
free(permutation);
return (PyObject *)ret;
--- 166,174 ----
ret->dimensions[i] = ap->dimensions[permutation[i]];
ret->strides[i] = ap->strides[permutation[i]];
}
! if (array_really_contiguous(ret)) ret->flags |= CONTIGUOUS;
! else ret->flags &= ~CONTIGUOUS;
!
PyArray_Free(op, (char *)axes);
free(permutation);
return (PyObject *)ret;
***************
*** 969,977 ****
PyArrayObject *a;
if (PyArg_ParseTuple(args, "OO", &a0, &shape) == NULL) return NULL;
! if ((a = (PyArrayObject *)PyArray_ContiguousFromObject(a0,
! PyArray_NOTYPE,0,0))
! ==NULL) return NULL;
ret = PyArray_Transpose(a, shape);
Py_DECREF(a);
--- 989,997 ----
PyArrayObject *a;
if (PyArg_ParseTuple(args, "OO", &a0, &shape) == NULL) return NULL;
! if ((a = (PyArrayObject *)PyArray_FromObject(a0,
! PyArray_NOTYPE,0,0))
! ==NULL) return NULL;
ret = PyArray_Transpose(a, shape);
Py_DECREF(a);
From kate@cptc.wisc.edu Wed Aug 12 06:55:51 1998
From: kate@cptc.wisc.edu (Kate Comer)
Date: Wed, 12 Aug 1998 00:55:51 -0500 (CDT)
Subject: [Matrix-SIG] returning multiple arrays from C
Message-ID:
I'm wrapping C around a set of FORTRAN spline subroutines that I want to
access from Python. Before I pull out the rest of my hair in frustration,
could someone please reveal the secret to returning more than one array
from a C extension back to python, and having the returned arrays actually
be different?
This is what I'm doing, with ~8 arrays instead of just y, dy, and yy
double *c_y, *c_yy, *c_dy;
PyObject *num_y, *num_yy, *num_dy;
PyArrayObject *arr_y, arr_yy, *num_dy;
DECLARE_INTEGER(m); \* these DECLARES *\
DECLARE_REAL_ARRAY(y, m); \* are for interface *\
DECLARE_REAL_ARRAY(yy,m); \* to FORTRAN spline *\
DECLARE_REAL_ARRAY(dy, m); \* subroutine *\
arr_y = (PyArrayObject *)PyArray_ContiguousFromObject(
num_y, PyArray_DOUBLE, 0, 0);
c_y = (double *)arr_y->data;
\* ...and the same thing two more times for the yy variables and dy
variables ...*\
for( i=0; i
(message from Kate Comer on Wed, 12 Aug 1998 00:55:51 -0500 (CDT))
References:
Message-ID: <199808121041.MAA28690@dirac.cnrs-orleans.fr>
> I'm wrapping C around a set of FORTRAN spline subroutines that I want to
> access from Python. Before I pull out the rest of my hair in frustration,
> could someone please reveal the secret to returning more than one array
> from a C extension back to python, and having the returned arrays actually
> be different?
No secret, it works as advertised. I suppose there is some
subtle bug in your code. There are certainly plenty of problems
in the example below, it shouldn't even compile!
> This is what I'm doing, with ~8 arrays instead of just y, dy, and yy
>
> double *c_y, *c_yy, *c_dy;
> PyObject *num_y, *num_yy, *num_dy;
> PyArrayObject *arr_y, arr_yy, *num_dy;
That should be
PyArrayObject *arr_y, *arr_yy, *arr_dy;
Right?
> DECLARE_INTEGER(m); \* these DECLARES *\
> DECLARE_REAL_ARRAY(y, m); \* are for interface *\
> DECLARE_REAL_ARRAY(yy,m); \* to FORTRAN spline *\
> DECLARE_REAL_ARRAY(dy, m); \* subroutine *\
No idea what that does...
> arr_y = (PyArrayObject *)PyArray_ContiguousFromObject(
> num_y, PyArray_DOUBLE, 0, 0);
> c_y = (double *)arr_y->data;
>
> \* ...and the same thing two more times for the yy variables and dy
> variables ...*\
Make sure that this section is correct; a typo anywhere could well cause
the problem you describe.
> for( i=0; i y[i] = *(c_x++); \* not real data, so they aren't in loop *\
> }
(That's c_y instead of c_x, right?)
Why do you have to copy the data? You could just pass c_y directly
to the Fortran routine. And again, any typo...
> return Py_BuildValue("OO", arr_yy, arr_dy);
Looks OK.
> Within Python I have
>
> y = array(whatever - it's 1d)
> m = len(y)
> dy = zeros(m, Float)
> yy = dy
There it is! You set yy equal to dy, which means that they refer
to the same array. In the C routine, you call PyArray_ContiguousFromObject,
but this routine does not create a new array if it finds that the original
argument is already contiguous. So the two arrays stay equal at all
times.
The obvious quick fix is
dy = zeros(m, Float)
yy = zeros(m, Float)
in Python. A more elegant way (in my opinion) is to create the
arrays dy and yy in the C code instead of passing them in from
Python. And you don't even have to pass in m, since it's just
the length of the array y.
--
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsen@cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.55.69
Rue Charles Sadron | Fax: +33-2.38.63.15.17
45071 Orleans Cedex 2 | Deutsch/Esperanto/English/
France | Nederlands/Francais
-------------------------------------------------------------------------------
From johann@leporello.Berkeley.EDU Fri Aug 21 00:06:42 1998
From: johann@leporello.Berkeley.EDU (Johann Hibschman)
Date: Thu, 20 Aug 1998 16:06:42 -0700 (PDT)
Subject: [Matrix-SIG] roll-my-own ufuncs
Message-ID:
Hi all,
Is there any additional documentation available for ufuncs? I have a few
functions which I regularly call, and it would be convenient if there were
expressed in python as ufuncs, but at first glance the ufuncmodule code is
rather impenetrable.
In general, is there a recommended method of passing a function to a
NumPy-aware function? I want to write a curve-fitting routine which would
work either on generic python functions or on wrapped C functions. I'd
like wrapped C functions to be used at C speeds, if possible. Any hints?
- Johann
From hauser@engr.uky.edu Wed Aug 26 20:18:31 1998
From: hauser@engr.uky.edu (Thomas Hauser)
Date: Wed, 26 Aug 1998 15:18:31 -0400
Subject: [Matrix-SIG] Missing EB.py for PyPDB
Message-ID: <199808261918.PAA03224@tomslinux.hauser.home>
Hi,
I'm trying to use the PyPDB package.
The PR.py imports EB
where is EB.py
Thanks
Thomas Hauser
From just@letterror.com Wed Aug 26 20:42:53 1998
From: just@letterror.com (Just van Rossum)
Date: Wed, 26 Aug 1998 21:42:53 +0200
Subject: [Matrix-SIG] Conway's Life with NumPy
Message-ID:
Today it occurred to me that it should be fairly trivial to implement
Conway's Life with NumPy. It was. It's a bit of nostalgia, since it was
one of the first programs I ever typed over from a book on my first
computer: a Sinclair ZX81...
Anyway, I thought I'd contribute this as a demo program, since it's
fairly trivial, and shows some basic NumPy features. It also shows why
Rich Comparisons are a Good Thing...
It would be nice to have a Tk based version of this, to make it more
attractive, but my Tk skills are still non-existant.
I still feel like a total NumPy newbie, so if anyone has suggestions
on how to do things differently, I'd be very interested.
Just
--- Life.py ---
"""A Python implementation of Conway's Life
Michael Wohlgenannt writes this about Life:
"For a single cell there are two distinct states, dead and alive. A living
cell dies of boredom if there are less than two living neighbours (all in
all there are eight). A dead cell gets back to life again if there are
exactly three living neighbours. The last rule is that a living cell dies
if there are more than three living neighbours.
A lot of cell configurations that can be constructed show a peculiar and
amusing behaviour. Some stay as they are, dead or alive, some oscillate,
some even propagate."
"""
# 26 august 1998, Just van Rossum
from Numeric import *
import sys
import string
def life(cells):
# convert cells to bytes for speed
cells = cells.astype(Int8)
# calculate how many neibors each cell has
neighbors = shift_left(shift_up(cells))
neighbors = neighbors + shift_up(cells)
neighbors = neighbors + shift_right(shift_up(cells))
neighbors = neighbors + shift_left(cells)
neighbors = neighbors + shift_right(cells)
neighbors = neighbors + shift_left(shift_down(cells))
neighbors = neighbors + shift_down(cells)
neighbors = neighbors + shift_right(shift_down(cells))
# apply the "Life" rules (see module doc string)
newcells = logical_or(
logical_and(
cells,
logical_and(
less_equal(2, neighbors),
less_equal(neighbors, 3)
)
),
equal(neighbors, 3)
)
# If I understood it correctly, with "rich comparisons"
# the above could look like this:
#
# newcells = cell and (2 <= neighbors <= 3) or neighbors == 3
#
# Now, wouldn't that be nice...
return newcells
def shift_up(cells):
return concatenate((cells[1:], cells[:1]))
def shift_down(cells):
return concatenate((cells[-1:], cells[:-1]))
def shift_left(cells):
return transpose(shift_up(transpose(cells)))
def shift_right(cells):
return transpose(shift_down(transpose(cells)))
def randomcells(width, height, chance = 5):
from whrandom import randint
cells = zeros((height, width), Int8)
_range = range
# fill with noise
for y in _range(height):
for x in _range(width):
cells[y][x] = randint(0, chance) == 0
return cells
def printcells(cells):
x, y = cells.shape
thing = "+" + y * "-" + "+"
lines = [thing]
for i in range(x):
list = map(lambda x: " O"[x], cells[i])
lines.append("|"+string.join(list, "")+"|")
lines.append(thing)
print string.join(lines, "\n")
if __name__ == "__main__":
import time
width = 20
height = 10
cells = randomcells(width, height)
while 1:
printcells(cells)
time.sleep(0.1)
cells = life(cells)
From Paul F. Dubois"
Your program is now LLNLDistribution/Numerical/Demo/life.py. Look for it in
our next release, and thanks. A version that uses Tk for display, as
suggested, would be welcome. Anyone?
From amullhau@ix.netcom.com Wed Aug 26 21:13:10 1998
From: amullhau@ix.netcom.com (Andrew P. Mullhaupt)
Date: Wed, 26 Aug 1998 16:13:10 -0400
Subject: [Matrix-SIG] Conway's Life with NumPy
Message-ID: <04d201bdd12d$f3303c00$f2615ecf@amullhau.ix.netcom.com>
-----Original Message-----
From: Just van Rossum
To: matrix-sig@python.org
Date: Wednesday, August 26, 1998 3:44 PM
Subject: [Matrix-SIG] Conway's Life with NumPy
>Today it occurred to me that it should be fairly trivial to implement
>Conway's Life with NumPy. It was.>
>I still feel like a total NumPy newbie, so if anyone has suggestions
>on how to do things differently, I'd be very interested.
You might find that the expressive power of Numerical Python makes it easy
to implement Gosper's recursive version which is orders of magnitude more
efficient in time and space for sparse cases, which is where you end up with
many interesting life patterns.
Later,
Andrew Mullhaupt
From aaron@cs.rutgers.edu Thu Aug 27 13:53:25 1998
From: aaron@cs.rutgers.edu (Aaron Watters)
Date: Thu, 27 Aug 1998 08:53:25 -0400
Subject: [Matrix-SIG] Conway's Life with NumPy
References: <04d201bdd12d$f3303c00$f2615ecf@amullhau.ix.netcom.com>
Message-ID: <35E556C5.EF75B4B1@cs.rutgers.edu>
Please expand. It seems to me that you need missing
"rich indexing" functionality to implement sparse life, but
perhaps I'm missing something important (very likely).
Please inform. -- Aaron Watters
Andrew P. Mullhaupt wrote:
> -----Original Message-----
> From: Just van Rossum
> To: matrix-sig@python.org
> Date: Wednesday, August 26, 1998 3:44 PM
> Subject: [Matrix-SIG] Conway's Life with NumPy
>
> >Today it occurred to me that it should be fairly trivial to implement
> >Conway's Life with NumPy. It was.>
> >I still feel like a total NumPy newbie, so if anyone has suggestions
> >on how to do things differently, I'd be very interested.
>
> You might find that the expressive power of Numerical Python makes it easy
> to implement Gosper's recursive version which is orders of magnitude more
> efficient in time and space for sparse cases, which is where you end up with
> many interesting life patterns.
>
> Later,
> Andrew Mullhaupt
>
> _______________________________________________
> Matrix-SIG maillist - Matrix-SIG@python.org
> http://www.python.org/mailman/listinfo/matrix-sig
From Paul F. Dubois"
Everybody,
Prompted by a message from Aaron, I think it is time to revisit this
question. What do you think I should do about the N announcements of
numerical-related Python packages? N is getting large.
Clearly I can't maintain and test all this stuff. Even maintaining a list of
links is fraught with peril. I wonder if some of this FAQ technology that
Guido has might be a better tool to let people keep their own
announcements/software up to date? I know some people have talked about me
creating some big Linnean classification for a library (or we could, for
example, use the NAG one, except that most people don't know what it is).
But I'm sure I can't manage that level of effort. We are extremely
short-handed at the moment.
I'd appreciate any comments on what to do. Or rather, how I can accomplish
something useful without doing much.
Paul
-----Original Message-----
From: Aaron Watters
To: Paul F. Dubois
Date: Thursday, August 27, 1998 5:57 AM
Subject: Re: [Matrix-SIG] Conway's Life with NumPy
>I would be delighted if you'd put a link to the pysimplex
>stuff somewhere in there too. (a copy might not be a good
>idea, since i may get around to improving it at any time.
>
>http://www.pythonpros.com/arw/pysimpex
>
> -- Aaron Watters
>
>Paul F. Dubois wrote:
>
>> Your program is now LLNLDistribution/Numerical/Demo/life.py. Look for it
in
>> our next release, and thanks. A version that uses Tk for display, as
>> suggested, would be welcome. Anyone?
>>
>> _______________________________________________
>> Matrix-SIG maillist - Matrix-SIG@python.org
>> http://www.python.org/mailman/listinfo/matrix-sig
>
>
>
>
From hinsen@cnrs-orleans.fr Mon Aug 31 15:20:14 1998
From: hinsen@cnrs-orleans.fr (Konrad Hinsen)
Date: Mon, 31 Aug 1998 16:20:14 +0200
Subject: [Matrix-SIG] Conway's Life with NumPy
In-Reply-To: <001001bdd48e$57d89c20$52120f80@dubois1-1.llnl.gov>
References: <001001bdd48e$57d89c20$52120f80@dubois1-1.llnl.gov>
Message-ID: <199808311420.QAA23048@dirac.cnrs-orleans.fr>
> Clearly I can't maintain and test all this stuff. Even maintaining a list of
> links is fraught with peril. I wonder if some of this FAQ technology that
> Guido has might be a better tool to let people keep their own
> announcements/software up to date? I know some people have talked about me
I think that's worth a try. Maybe we can even convince one of the
FAQ Wizard wizards to set up one for NumPy. Then you'd just
have to add a pointer to your Web page and let others do the work.
Konrad.
--
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsen@cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.55.69
Rue Charles Sadron | Fax: +33-2.38.63.15.17
45071 Orleans Cedex 2 | Deutsch/Esperanto/English/
France | Nederlands/Francais
-------------------------------------------------------------------------------
From da@skivs.ski.org Mon Aug 31 17:30:39 1998
From: da@skivs.ski.org (David Ascher)
Date: Mon, 31 Aug 1998 09:30:39 -0700 (PDT)
Subject: [Matrix-SIG] Conway's Life with NumPy
In-Reply-To: <199808311420.QAA23048@dirac.cnrs-orleans.fr>
Message-ID:
> I think that's worth a try. Maybe we can even convince one of the
> FAQ Wizard wizards to set up one for NumPy. Then you'd just
> have to add a pointer to your Web page and let others do the work.
It's in the works -- I'll announce the URL as soon as Ken and I finish
getting the scientific computing alpha topic guide (which will happen once
I get back stateside and clean it up a little).
For management of numpy software, IMHO the right solution is to piggy back
on the Grove system -- alas, I don't think it's quite ready yet. I'll ask
Eric Raymond for his feeling on the ETA of that project and let y'all
know.
--david
From Paul F. Dubois"
I just rebuilt Python, Numerical, and C++ with MS Visual C++ 6.0 on an NT
box and everything seemed to go well. They automatically convert the project
files that David's tool puts out to a new format. I didn't do extensive
testing. The compile seemed much faster.