From n8gray at caltech.edu  Thu Aug  2 20:18:13 2001
From: n8gray at caltech.edu (Nathaniel Gray)
Date: Thu, 2 Aug 2001 17:18:13 -0700
Subject: [Numpy-discussion] A savespace bug, perhaps?
Message-ID: <01080217181302.01319@DHCP-134-182>

Hi everybody,

I'm using Python 2.1.1 and Numpy 20.1.0  This seems to be a problem:

Python 2.1.1 (#2, Jul 31 2001, 14:10:42)                                     
[GCC 2.96 20000731 (Linux-Mandrake 8.0 2.96-0.48mdk)] on linux2
Type "copyright", "credits" or "license" for more information.
>>> from Numeric import *
>>> from MLab import zeros
>>> joe = zeros(3,'l',1)
>>> fred = array([1,2,3],'l')
>>> joe += fred
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: return array has incorrect type

This fails whether or not fred has the spacesaver flag set.

This, however, works fine:
>>> joe = zeros(3, 'l')
>>> fred = array([1,2,3], 'l')
>>> joe += fred
>>> joe
array([1, 2, 3])

Am I doing something wrong or is this a bug?

Please CC to me since I'm not subscribed to the list.
Thanks a lot!
-Nathan

-- 
Nathaniel Gray

California Institute of Technology
Computation and Neural Systems
--



From oliphant.travis at ieee.org  Thu Aug  2 15:13:48 2001
From: oliphant.travis at ieee.org (Travis Oliphant)
Date: Thu, 2 Aug 2001 19:13:48 +0000
Subject: [Numpy-discussion] A savespace bug, perhaps?
In-Reply-To: <01080217181302.01319@DHCP-134-182>
References: <01080217181302.01319@DHCP-134-182>
Message-ID: <01080219134800.17512@travis>

On Fri, 03 Aug 2001, Nathaniel Gray wrote:
> Hi everybody,
>
> I'm using Python 2.1.1 and Numpy 20.1.0  This seems to be a problem:
>
> Python 2.1.1 (#2, Jul 31 2001, 14:10:42)
> [GCC 2.96 20000731 (Linux-Mandrake 8.0 2.96-0.48mdk)] on linux2
> Type "copyright", "credits" or "license" for more information.
>
> >>> from Numeric import *
> >>> from MLab import zeros
> >>> joe = zeros(3,'l',1)
> >>> fred = array([1,2,3],'l')
> >>> joe += fred
>
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
> TypeError: return array has incorrect type
>
> This fails whether or not fred has the spacesaver flag set.
>
> This, however, works fine:
> >>> joe = zeros(3, 'l')
> >>> fred = array([1,2,3], 'l')
> >>> joe += fred
> >>> joe
>
> array([1, 2, 3])
>
> Am I doing something wrong or is this a bug?
>
> Please CC to me since I'm not subscribed to the list.
> Thanks a lot!
> -Nathan

Thanks for the bug report, Nathan. 

It was indeed a bug, which I hope has now been squashed in the latest CVS 
version of Numeric.  

Thanks for your help.

Travis




From europax at home.com  Fri Aug  3 18:11:12 2001
From: europax at home.com (Rob)
Date: Fri, 03 Aug 2001 15:11:12 -0700
Subject: [Numpy-discussion] new to list
Message-ID: <3B6B2180.FBDCDBF9@home.com>

Hello,

I'm new to the list so I thought I'd introduce myself.  I use Numpy at
work for some Python programs that interface with Agilent and/or Anritsu
microwave equipment, using the GPIB bus.

However, my main interest lately is porting some EM codes into Numeric
Python.  I've had varied success.  See my web site
www.members.home.net/europax.  Right now I'm trying to Numpy-ize this
FEM-MOM code.  I've profiled it using the Python profile class.  Using
MOM for a dipole takes 5 minutes of cpu :) :)  Interestingly, some of
the worst routines are simple ones that really have little to do with
matrix math.
Anyway, I'm having fun.  Its wonderful to be able to focus on algorithms
rather than wading through arcane programming syntax !!

Rob.



From gblee at mail.paichai.ac.kr  Mon Aug  6 22:45:03 2001
From: gblee at mail.paichai.ac.kr (=?ks_c_5601-1987?B?wMyx1LrA?=)
Date: Tue, 7 Aug 2001 11:45:03 +0900
Subject: [Numpy-discussion] NumTut
Message-ID: <000a01c11eea$f5e09040$7386facb@paichai.ac.kr>

Dear who may be concerned,

I want to install the package NumTut. But I can't find the directory of source distribution. Please let me know how to get the directory odf souce distribution and the subdirectory Demo.

Thanks

Gyoy-Bong Lee from Korea
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20010807/766674bd/attachment.html>

From dubois1 at llnl.gov  Tue Aug  7 13:20:19 2001
From: dubois1 at llnl.gov (Paul F. Dubois)
Date: Tue, 7 Aug 2001 10:20:19 -0700
Subject: [Numpy-discussion] swig and Numeric
Message-ID: <01080710213100.17772@almanac>

Does someone have an example of using SWIG to connect Numeric arrays to C code
that expects double* arguments?



From hungjunglu at yahoo.com  Tue Aug  7 15:50:04 2001
From: hungjunglu at yahoo.com (Hung Jung Lu)
Date: Tue, 7 Aug 2001 12:50:04 -0700 (PDT)
Subject: [Numpy-discussion] fast constructor for arrays from byte data (unpickling?)
Message-ID: <20010807195004.19068.qmail@web12606.mail.yahoo.com>

Hi,

Arrays have a method called tostring() which generates
the binary data. Is there an inverse function to that?
That is, generating an array from the binary data
string?

For large matrices, pickling/unpickling is a bit too
much overhead (data stored as ASCII character strings
instead of binary data strings.) I know, I am talking
about a factor 4 here. But there is a big difference
between 1 minute loading time and 4 minute loading
time.

I would imagine this is a very common problem/request
for people working with large matrices. And I am sure
hacking the C code to provide another fast constructor
for arrays from binary strings won't be too hard. The
questions is: has anyone already tried it? Is it
already there?

(For the kludge masters: one kludge is of course to
store the binary data on disk, then use cStringIO to
build the pickled file and then unpickle from the
cStringIO. Speed is probably OK since the pickled file
lives on RAM. But that's a kludge. :) )

regards,

Hung Jung


__________________________________________________
Do You Yahoo!?
Make international calls for as low as $.04/minute with Yahoo! Messenger
http://phonecard.yahoo.com/



From kern at caltech.edu  Tue Aug  7 15:59:37 2001
From: kern at caltech.edu (Robert Kern)
Date: Tue, 7 Aug 2001 12:59:37 -0700
Subject: [Numpy-discussion] fast constructor for arrays from byte data (unpickling?)
In-Reply-To: <20010807195004.19068.qmail@web12606.mail.yahoo.com>
References: <20010807195004.19068.qmail@web12606.mail.yahoo.com>
Message-ID: <20010807125937.E22660@myrddin.caltech.edu>

On Tue, Aug 07, 2001 at 12:50:04PM -0700, Hung Jung Lu wrote:
> Hi,
> 
> Arrays have a method called tostring() which generates
> the binary data. Is there an inverse function to that?
> That is, generating an array from the binary data
> string?

Numeric.fromstring

HTH.

-- 
Robert Kern
kern at caltech.edu

"In the fields of hell where the grass grows high
 Are the graves of dreams allowed to die."
  -- Richard Harter



From chrishbarker at home.net  Tue Aug  7 16:26:57 2001
From: chrishbarker at home.net (Chris Barker)
Date: Tue, 07 Aug 2001 13:26:57 -0700
Subject: [Numpy-discussion] fast constructor for arrays from byte data 
 (unpickling?)
References: <20010807195004.19068.qmail@web12606.mail.yahoo.com>
Message-ID: <3B704F10.9457A928@home.net>

Hung Jung Lu wrote:
> Arrays have a method called tostring() which generates
> the binary data. Is there an inverse function to that?
> That is, generating an array from the binary data
> string?

As you suspected, there is such a thing, and it is called (surprise):
fromstring(). It is a function, rather than a method, as it is a
constructor for an array. To get data from a file, you have to get it
into a string first, so I use:

M = fromstring(file.read(),Float)

This makes a rank -1 array. You might have to reshape it.

Note that you end up creating a Python string of the data first, and
then a NumPy array from that. This doesn't really cost that much, but it
can be an issue with huge data sets. I wish there was a fromfile()
function. I may get around to writing it one day.

-Chris







-- 
Christopher Barker,
Ph.D.                                                           
ChrisHBarker at home.net                 ---           ---           ---
http://members.home.net/barkerlohmann ---@@       -----@@       -----@@
                                   ------@@@     ------@@@     ------@@@
Oil Spill Modeling                ------   @    ------   @   ------   @
Water Resources Engineering       -------      ---------     --------    
Coastal and Fluvial Hydrodynamics --------------------------------------
------------------------------------------------------------------------



From hungjunglu at yahoo.com  Tue Aug  7 16:08:54 2001
From: hungjunglu at yahoo.com (Hung Jung Lu)
Date: Tue, 7 Aug 2001 13:08:54 -0700 (PDT)
Subject: [Numpy-discussion] fast constructor for arrays from byte data (unpickling?)
In-Reply-To: <20010807125937.E22660@myrddin.caltech.edu>
Message-ID: <20010807200854.20400.qmail@web12601.mail.yahoo.com>

> > That is, generating an array from the binary data
> > string?
> 
> Numeric.fromstring

Ahhh! Now that seems so obvious. :) (Banging my head.)

thanks!

Hung Jung


__________________________________________________
Do You Yahoo!?
Make international calls for as low as $.04/minute with Yahoo! Messenger
http://phonecard.yahoo.com/



From paul at pfdubois.com  Tue Aug  7 15:56:02 2001
From: paul at pfdubois.com (Paul F. Dubois)
Date: Tue, 7 Aug 2001 12:56:02 -0700
Subject: [Numpy-discussion] fast constructor for arrays from byte data (unpickling?)
In-Reply-To: <20010807195004.19068.qmail@web12606.mail.yahoo.com>
Message-ID: <ADEOIFHFONCLEEPKCACCGELPCLAA.paul@pfdubois.com>

See fromstring() 
>>> import Numeric
>>> x=Numeric.arange(10)
>>> s=x.tostring()
>>> y = Numeric.fromstring(s)
>>> y
array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
>>> 

-----Original Message-----
From: numpy-discussion-admin at lists.sourceforge.net
[mailto:numpy-discussion-admin at lists.sourceforge.net]On Behalf Of Hung
Jung Lu
Sent: Tuesday, August 07, 2001 12:50 PM
To: numpy-discussion at lists.sourceforge.net
Subject: [Numpy-discussion] fast constructor for arrays from byte data
(unpickling?)


Hi,

Arrays have a method called tostring() which generates
the binary data. Is there an inverse function to that?
That is, generating an array from the binary data
string?

For large matrices, pickling/unpickling is a bit too
much overhead (data stored as ASCII character strings
instead of binary data strings.) I know, I am talking
about a factor 4 here. But there is a big difference
between 1 minute loading time and 4 minute loading
time.

I would imagine this is a very common problem/request
for people working with large matrices. And I am sure
hacking the C code to provide another fast constructor
for arrays from binary strings won't be too hard. The
questions is: has anyone already tried it? Is it
already there?

(For the kludge masters: one kludge is of course to
store the binary data on disk, then use cStringIO to
build the pickled file and then unpickle from the
cStringIO. Speed is probably OK since the pickled file
lives on RAM. But that's a kludge. :) )

regards,

Hung Jung


__________________________________________________
Do You Yahoo!?
Make international calls for as low as $.04/minute with Yahoo! Messenger
http://phonecard.yahoo.com/

_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
http://lists.sourceforge.net/lists/listinfo/numpy-discussion




From kern at caltech.edu  Tue Aug  7 16:17:23 2001
From: kern at caltech.edu (Robert Kern)
Date: Tue, 7 Aug 2001 13:17:23 -0700
Subject: [Numpy-discussion] fast constructor for arrays from byte data (unpickling?)
In-Reply-To: <3B704F10.9457A928@home.net>
References: <20010807195004.19068.qmail@web12606.mail.yahoo.com> <3B704F10.9457A928@home.net>
Message-ID: <20010807131723.F22660@myrddin.caltech.edu>

On Tue, Aug 07, 2001 at 01:26:57PM -0700, Chris Barker wrote:

[snip]

> Note that you end up creating a Python string of the data first, and
> then a NumPy array from that. This doesn't really cost that much, but it
> can be an issue with huge data sets. I wish there was a fromfile()
> function. I may get around to writing it one day.

With Python 2.0 or greater, one can use mmap'ed files as arguments to
fromstring.

> -Chris

-- 
Robert Kern
kern at caltech.edu

"In the fields of hell where the grass grows high
 Are the graves of dreams allowed to die."
  -- Richard Harter



From chrishbarker at home.net  Tue Aug  7 18:02:01 2001
From: chrishbarker at home.net (Chris Barker)
Date: Tue, 07 Aug 2001 15:02:01 -0700
Subject: [Numpy-discussion] fast constructor for arrays from byte data 
 (unpickling?)
References: <20010807195004.19068.qmail@web12606.mail.yahoo.com> <3B704F10.9457A928@home.net> <20010807131723.F22660@myrddin.caltech.edu>
Message-ID: <3B706559.597944ED@home.net>

Robert Kern wrote:
> On Tue, Aug 07, 2001 at 01:26:57PM -0700, Chris Barker wrote:
> > I wish there was a fromfile() function.
> 
> With Python 2.0 or greater, one can use mmap'ed files as arguments to
> fromstring.

Can you give me an example of how to use it? I can not get it to work at
all, the docs are pretty sketchy. I can't even get a mmap's file to be
created properly on linux. I havn't tried Windows yet, but I'll need it
to work there too!

Also, mmap does not seem to be supported on the Mac, which is where I am
having memory problems (due to the Mac's lousy memeory management). I'll
ask about it on the MacPython list.

-Chris



-- 
Christopher Barker,
Ph.D.                                                           
ChrisHBarker at home.net                 ---           ---           ---
http://members.home.net/barkerlohmann ---@@       -----@@       -----@@
                                   ------@@@     ------@@@     ------@@@
Oil Spill Modeling                ------   @    ------   @   ------   @
Water Resources Engineering       -------      ---------     --------    
Coastal and Fluvial Hydrodynamics --------------------------------------
------------------------------------------------------------------------



From kern at caltech.edu  Tue Aug  7 17:54:36 2001
From: kern at caltech.edu (Robert Kern)
Date: Tue, 7 Aug 2001 14:54:36 -0700
Subject: [Numpy-discussion] fast constructor for arrays from byte data (unpickling?)
In-Reply-To: <3B706559.597944ED@home.net>
References: <20010807195004.19068.qmail@web12606.mail.yahoo.com> <3B704F10.9457A928@home.net> <20010807131723.F22660@myrddin.caltech.edu> <3B706559.597944ED@home.net>
Message-ID: <20010807145436.G22660@myrddin.caltech.edu>

On Tue, Aug 07, 2001 at 03:02:01PM -0700, Chris Barker wrote:
> Robert Kern wrote:
> > On Tue, Aug 07, 2001 at 01:26:57PM -0700, Chris Barker wrote:
> > > I wish there was a fromfile() function.
> > 
> > With Python 2.0 or greater, one can use mmap'ed files as arguments to
> > fromstring.
> 
> Can you give me an example of how to use it? I can not get it to work at
> all, the docs are pretty sketchy. I can't even get a mmap's file to be
> created properly on linux. I havn't tried Windows yet, but I'll need it
> to work there too!

Yeah, it took me some fiddling, too, before I got it to work. The Windows call
to mmap has slightly different semantics in that second argument, but I think 
that if you want to snarf the whole file, then what I do below should work as
well. The Windows API lets you use 0 to default to the whole file, but that's
not portable.

Python 2.0.1 (#0, Jul  3 2001, 12:36:30) 
[GCC 2.95.4 20010629 (Debian prerelease)] on linux2
Type "copyright", "credits" or "license" for more information.
>>> import Numeric
>>> import mmap
>>> f = open('earth.dat', 'r+')
>>> f.seek(0, 2)  # seek to EOF
>>> size = f.tell()
>>> f.seek(0)  # rewind; possibly not necessary
>>> m = mmap.mmap(f.fileno(), size)
>>> a = Numeric.fromstring(m)
>>> size
548240
>>> a.shape
(137060,)
>>> size / 4
137060
>>> 

> Also, mmap does not seem to be supported on the Mac, which is where I am
> having memory problems (due to the Mac's lousy memeory management). I'll
> ask about it on the MacPython list.

You're probably SOL, there, but I'll wish you good luck anyways.

if-wishes-were-mmap-implementations-ly yours,

-- 
Robert Kern
kern at caltech.edu

"In the fields of hell where the grass grows high
 Are the graves of dreams allowed to die."
  -- Richard Harter



From chrishbarker at home.net  Tue Aug  7 19:33:30 2001
From: chrishbarker at home.net (Chris Barker)
Date: Tue, 07 Aug 2001 16:33:30 -0700
Subject: [Numpy-discussion] fast constructor for arrays from byte data 
 (unpickling?)
References: <20010807195004.19068.qmail@web12606.mail.yahoo.com> <3B704F10.9457A928@home.net> <20010807131723.F22660@myrddin.caltech.edu> <3B706559.597944ED@home.net> <20010807145436.G22660@myrddin.caltech.edu>
Message-ID: <3B707ACA.B5FE8E55@home.net>

Robert Kern wrote:

> Yeah, it took me some fiddling, too, before I got it to work. The Windows call
> to mmap has slightly different semantics in that second argument, but I think
> that if you want to snarf the whole file, then what I do below should work as
> well. The Windows API lets you use 0 to default to the whole file, but that's
> not portable.

[code snipped]

no, the "0 argument does not seem to work on Unix. It seems odd to not
have a way to just snarf the whole file automatically, but so be it

Thanks, that works, but now I am wondering: what I want is a fast and
memory efficient way to get the contents of a fjile into a NujmPy array,
this sure doesn't look any better than:

a = fromstring(file.read()))

thanks anyway,

-Chris


-- 
Christopher Barker,
Ph.D.                                                           
ChrisHBarker at home.net                 ---           ---           ---
http://members.home.net/barkerlohmann ---@@       -----@@       -----@@
                                   ------@@@     ------@@@     ------@@@
Oil Spill Modeling                ------   @    ------   @   ------   @
Water Resources Engineering       -------      ---------     --------    
Coastal and Fluvial Hydrodynamics --------------------------------------
------------------------------------------------------------------------



From kern at caltech.edu  Tue Aug  7 19:28:18 2001
From: kern at caltech.edu (Robert Kern)
Date: Tue, 7 Aug 2001 16:28:18 -0700
Subject: [Numpy-discussion] fast constructor for arrays from byte data (unpickling?)
In-Reply-To: <3B707ACA.B5FE8E55@home.net>
References: <20010807195004.19068.qmail@web12606.mail.yahoo.com> <3B704F10.9457A928@home.net> <20010807131723.F22660@myrddin.caltech.edu> <3B706559.597944ED@home.net> <20010807145436.G22660@myrddin.caltech.edu> <3B707ACA.B5FE8E55@home.net>
Message-ID: <20010807162818.H22660@myrddin.caltech.edu>

On Tue, Aug 07, 2001 at 04:33:30PM -0700, Chris Barker wrote:

[snip]

> Thanks, that works, but now I am wondering: what I want is a fast and
> memory efficient way to get the contents of a fjile into a NujmPy array,
> this sure doesn't look any better than:
> 
> a = fromstring(file.read()))

Depends on how large the file is. file.read() creates a temporary string the
size of the file. That string isn't freed until fromstring() finishes and
returns the array object. For a brief time, both the string and the array have
duplicates of the same data taking up space in memory.

I don't know the details of mmap, so it's certainly possible that the only way
that fromstring knows how to access the data is to pull all of it into memory 
first, thus recreating the problem. Alas.

> thanks anyway,
> 
> -Chris

-- 
Robert Kern
kern at caltech.edu

"In the fields of hell where the grass grows high
 Are the graves of dreams allowed to die."
  -- Richard Harter



From serge at rohan.sdsu.edu  Wed Aug  8 00:38:30 2001
From: serge at rohan.sdsu.edu (Serge Rey)
Date: Wed, 8 Aug 2001 14:38:30 +1000
Subject: [Numpy-discussion] permutation and array subsets
Message-ID: <20010808143830.E314@typhoon.sdsu.edu>

i'm trying to use the results of permutation (from RandomArray.py) to
reorder the rows of an array:

>>> r=arange(20)
>>> id=permutation(20)
>>> y=r[id]
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  IndexError: invalid index
  >>> 

i can use a for loop to permutate the ordering of r, but that has a
speed hit associated with it.  can anyone offer a suggestion for a way
to do this more efficiently? 

thanks,
serge
-- 
Sergio J. Rey	http://typhoon.sdsu.edu/rey.html



From ransom at cfa.harvard.edu  Wed Aug  8 00:46:41 2001
From: ransom at cfa.harvard.edu (Scott Ransom)
Date: Wed, 08 Aug 2001 00:46:41 -0400
Subject: [Numpy-discussion] permutation and array subsets
References: <20010808143830.E314@typhoon.sdsu.edu>
Message-ID: <3B70C431.D1F833C6@cfa.harvard.edu>

Serge Rey wrote:
>
> i'm trying to use the results of permutation (from RandomArray.py) to
> reorder the rows of an array:
> 
> >>> r=arange(20)
> >>> id=permutation(20)
> >>> y=r[id]

Try:

y = take(r, permutation(len(r)))

Scott

-- 
Scott M. Ransom                   Address:  Harvard-Smithsonian CfA
Phone:  (617) 496-7908                      60 Garden St.  MS 10 
email:  ransom at cfa.harvard.edu              Cambridge, MA  02138
GPG Fingerprint: 06A9 9553 78BE 16DB 407B  FFCA 9BFA B6FF FFD3 2989



From ransom at cfa.harvard.edu  Wed Aug  8 00:54:49 2001
From: ransom at cfa.harvard.edu (Scott Ransom)
Date: Wed, 08 Aug 2001 00:54:49 -0400
Subject: [Numpy-discussion] swig and Numeric
References: <01080710213100.17772@almanac>
Message-ID: <3B70C619.4FE32847@cfa.harvard.edu>

"Paul F. Dubois" wrote:
> 
> Does someone have an example of using SWIG to connect Numeric arrays to C code
> that expects double* arguments?

Hi Paul,

I have a fairly comprehensive set of typedefs (including 1-D and 2-D
array input and output) for interfacing Numeric arrays here:

ftp://cfa-ftp.harvard.edu/pub/ransom/numpy.i

As an example, say you have a function with a prototype:

int foo(double *array, int len_array);

You would simply make a ".i" file with the following:

======================
%module foo
%include numpy.i 

%apply double* IN_1D_DOUBLE { double *array };
int bar(double *array, int len_array);

=======================

Run swig on it, compile it, and import it.

Hope this helps,

Scott

-- 
Scott M. Ransom                   Address:  Harvard-Smithsonian CfA
Phone:  (617) 496-7908                      60 Garden St.  MS 10 
email:  ransom at cfa.harvard.edu              Cambridge, MA  02138
GPG Fingerprint: 06A9 9553 78BE 16DB 407B  FFCA 9BFA B6FF FFD3 2989



From chrishbarker at home.net  Wed Aug  8 16:37:02 2001
From: chrishbarker at home.net (Chris Barker)
Date: Wed, 08 Aug 2001 13:37:02 -0700
Subject: [Numpy-discussion] fast constructor for arrays from byte data 
 (unpickling?)
References: <20010807195004.19068.qmail@web12606.mail.yahoo.com> <3B704F10.9457A928@home.net> <20010807131723.F22660@myrddin.caltech.edu> <3B706559.597944ED@home.net> <20010807145436.G22660@myrddin.caltech.edu> <3B707ACA.B5FE8E55@home.net> <20010807162818.H22660@myrddin.caltech.edu>
Message-ID: <3B71A2EE.EC3BBE46@home.net>

Robert Kern wrote:

> > Thanks, that works, but now I am wondering: what I want is a fast and
> > memory efficient way to get the contents of a file into a NujmPy array,
> > this sure doesn't look any better than:
> >
> > a = fromstring(file.read()))
> 
> Depends on how large the file is. file.read() creates a temporary string the
> size of the file. That string isn't freed until fromstring() finishes and
> returns the array object. For a brief time, both the string and the array have
> duplicates of the same data taking up space in memory.

Exactly. Also is the memory used for hte string guarranteed to be freed
right away? I have no idea how Python internals work.

Anyway, that's why I want a "fromfile()" method, like the one inthe
library array module.
 
> I don't know the details of mmap, so it's certainly possible that the only way
> that fromstring knows how to access the data is to pull all of it into memory
> first, thus recreating the problem. Alas.

I don't understand mmap at all. From the name, it sounds like the entire
contents of the file is mapped into memory, so the memory would get used
as soon as you set it up. If anyone knows, I'd like to hear...


-Chris





-- 
Christopher Barker,
Ph.D.                                                           
ChrisHBarker at home.net                 ---           ---           ---
http://members.home.net/barkerlohmann ---@@       -----@@       -----@@
                                   ------@@@     ------@@@     ------@@@
Oil Spill Modeling                ------   @    ------   @   ------   @
Water Resources Engineering       -------      ---------     --------    
Coastal and Fluvial Hydrodynamics --------------------------------------
------------------------------------------------------------------------



From phil at geog.ubc.ca  Wed Aug  8 16:56:56 2001
From: phil at geog.ubc.ca (Phil Austin)
Date: Wed, 8 Aug 2001 13:56:56 -0700 (PDT)
Subject: [Numpy-discussion] mmap (was: fast constructor for arrays from byte data)
Message-ID: <15217.42904.327270.469067@curlew.geog.ubc.ca>

Chris Barker writes:

> I don't understand mmap at all. From the name, it sounds like the entire
> contents of the file is mapped into memory, so the memory would get used
> as soon as you set it up. If anyone knows, I'd like to hear...

This might be more than you need to know, but there's a 
copy of the article:

"Why aren't you using mmap() yet?"
by Kevin Sheehan at:

http://thrush.eos.ubc.ca/users/phil/mmap/mmap.pdf (note reversed page
order)

and a Solaris/SGI email debate in mhonarc format:

http://thrush.eos.ubc.ca/users/phil/mmap/threads.html

bottom line: you really need a vm implementation that supports
madvise() to get the maximum benefit from mmap.  (Linux 2.4
might have madvise, I haven't tried it out yet -- 2.2 didn't).

The way I used to use mmap was through a numpy array class that was
initiated from a pointer to a mmap'd file.  This made reading/writing
to the file as simple as indexing into the numpy array.  I haven't
been back to reimplement that class on linux since I left solaris,
though.  You can also get some of the benefits of mmap via netcdf,
which will use mmap if it exists.

Regards, Phil



From kern at caltech.edu  Wed Aug  8 17:23:10 2001
From: kern at caltech.edu (Robert Kern)
Date: Wed, 8 Aug 2001 14:23:10 -0700
Subject: [Numpy-discussion] mmap (was: fast constructor for arrays from byte data)
In-Reply-To: <15217.42904.327270.469067@curlew.geog.ubc.ca>
References: <15217.42904.327270.469067@curlew.geog.ubc.ca>
Message-ID: <20010808142310.A518@myrddin.caltech.edu>

On Wed, Aug 08, 2001 at 01:56:56PM -0700, Phil Austin wrote:

[snip pointers to info]

> bottom line: you really need a vm implementation that supports
> madvise() to get the maximum benefit from mmap.  (Linux 2.4
> might have madvise, I haven't tried it out yet -- 2.2 didn't).

FWIW, there appears to be a madvise(2) system call in Linux 2.4.7 (in 
mm/filemap.c for those of you with kernel source trees lying around), but I
can't find it in the headers anywhere. Sigh. I don't know if madvise is supposed
to be internal or not, though.

-- 
Robert Kern
kern at caltech.edu

"In the fields of hell where the grass grows high
 Are the graves of dreams allowed to die."
  -- Richard Harter



From kern at caltech.edu  Wed Aug  8 17:24:14 2001
From: kern at caltech.edu (Robert Kern)
Date: Wed, 8 Aug 2001 14:24:14 -0700
Subject: [Numpy-discussion] fast constructor for arrays from byte data (unpickling?)
In-Reply-To: <3B71A2EE.EC3BBE46@home.net>
References: <20010807195004.19068.qmail@web12606.mail.yahoo.com> <3B704F10.9457A928@home.net> <20010807131723.F22660@myrddin.caltech.edu> <3B706559.597944ED@home.net> <20010807145436.G22660@myrddin.caltech.edu> <3B707ACA.B5FE8E55@home.net> <20010807162818.H22660@myrddin.caltech.edu> <3B71A2EE.EC3BBE46@home.net>
Message-ID: <20010808142414.B518@myrddin.caltech.edu>

On Wed, Aug 08, 2001 at 01:37:02PM -0700, Chris Barker wrote:
> Robert Kern wrote:
> 
> > > Thanks, that works, but now I am wondering: what I want is a fast and
> > > memory efficient way to get the contents of a file into a NujmPy array,
> > > this sure doesn't look any better than:
> > >
> > > a = fromstring(file.read()))
> > 
> > Depends on how large the file is. file.read() creates a temporary string the
> > size of the file. That string isn't freed until fromstring() finishes and
> > returns the array object. For a brief time, both the string and the array have
> > duplicates of the same data taking up space in memory.
> 
> Exactly. Also is the memory used for hte string guarranteed to be freed
> right away? I have no idea how Python internals work.

Once fromstring returns, the string's refcount is 0 and should be freed just
about right away. I'm not sure, but I don't think the new gc will affect that.

> Anyway, that's why I want a "fromfile()" method, like the one inthe
> library array module.

Yes, I agree, it would be nice to have.

> > I don't know the details of mmap, so it's certainly possible that the only way
> > that fromstring knows how to access the data is to pull all of it into memory
> > first, thus recreating the problem. Alas.
> 
> I don't understand mmap at all. From the name, it sounds like the entire
> contents of the file is mapped into memory, so the memory would get used
> as soon as you set it up. If anyone knows, I'd like to hear...

Performing a very cursory, very non-scientific study, I created a 110M file,
mmap'ed it, then made a string from some of it. I'm using 64MB of RAM, 128MB
swap partition on a Linux 2.4.7 machine. According to top(1), the memory use 
didn't jump up until I made the string. Also, given that the call to mmap
returned almost instantaneously, I'd say that mmap doesn't pull the whole file
into memory when the object is created (one could just read the file for that).

I don't know what test to perform to see whether the fromstring() constructor
uses double memory, but my guess would be that memcpy won't pull in the whole
file before copying. OTOH, the accessed portions of the mmap'ed file may be kept
in memory.

Does anyone know the details on mmap? I'm shooting in the dark, here.
<reads Phil Austin's e-mail>  Ooh, nice.

> -Chris

-- 
Robert Kern
kern at caltech.edu

"In the fields of hell where the grass grows high
 Are the graves of dreams allowed to die."
  -- Richard Harter



From tariq_rashid at lineone.net  Wed Aug  8 18:17:48 2001
From: tariq_rashid at lineone.net (Tariq Rashid)
Date: Wed, 8 Aug 2001 23:17:48 +0100
Subject: [Numpy-discussion] swig and C/C++
In-Reply-To: <E15UCBk-0004VJ-00@usw-sf-list1.sourceforge.net>
Message-ID: <NFBBLBJKHCHBHGGGHJDOAEMOCAAA.tariq_rashid@lineone.net>

in fact ... even simpler ... can anyone point me to an example of passing
Numeric arrays to and from C ... the swig examples are fine for integers /
floats ... but as i'm a new to this stuff I need a bit if guidance with
Numeric arrays ...

any ideas / pointers would be very gratefully receieved... i'm working on
wavelet stuff which i want to give back to the Numeric community....

thanks

tariq		(tariq_rashid at lineone.net)

----------------------------

Today's Topics:

   1. NumTut (=?ks_c_5601-1987?B?wMyx1LrA?=)
   2. swig and Numeric (Paul F. Dubois)


From dubois1 at llnl.gov  Tue Aug  7 13:20:19 2001
From: dubois1 at llnl.gov (Paul F. Dubois)
Date: Tue, 7 Aug 2001 10:20:19 -0700
Subject: [Numpy-discussion] swig and Numeric
Message-ID: <mailman.3.1490332291.18468.numpy-discussion@python.org>

Does someone have an example of using SWIG to connect Numeric arrays to C=
 code
that expects double* arguments?





From just at letterror.com  Fri Aug 10 03:29:42 2001
From: just at letterror.com (Just van Rossum)
Date: Fri, 10 Aug 2001 09:29:42 +0200
Subject: [Numpy-discussion] NumPy installation instructions for Windows
Message-ID: <20010810092947-r01010700-1f9414d2-0910-010c@10.0.0.21>

I develop a package that depends on NumPy, and I would like to add a line or two
to my documentation about how to install NumPy under Windows -- just the basic
set of modules, not the extensions.

I understand users should grab the .exe archive. The FAQ only mentions "The .exe
files are  Windows installers of the sort that you are used to". Does this mean
everything is explained and done by the installer itself, or is there some
additional action the user should perform? Thanks!

(I have no Windows box here so I can't see for myself...)

Just



From paul at pfdubois.com  Fri Aug 10 12:04:11 2001
From: paul at pfdubois.com (Paul F. Dubois)
Date: Fri, 10 Aug 2001 09:04:11 -0700
Subject: [Numpy-discussion] NumPy installation instructions for Windows
In-Reply-To: <20010810092947-r01010700-1f9414d2-0910-010c@10.0.0.21>
Message-ID: <ADEOIFHFONCLEEPKCACCEEOBCLAA.paul@pfdubois.com>

The windows installer does everything required to import Numeric.

-----Original Message-----
From: numpy-discussion-admin at lists.sourceforge.net
[mailto:numpy-discussion-admin at lists.sourceforge.net]On Behalf Of Just
van Rossum
Sent: Friday, August 10, 2001 12:30 AM
To: numpy-discussion at lists.sourceforge.net
Subject: [Numpy-discussion] NumPy installation instructions for Windows


I develop a package that depends on NumPy, and I would like to add a line or
two
to my documentation about how to install NumPy under Windows -- just the
basic
set of modules, not the extensions.

I understand users should grab the .exe archive. The FAQ only mentions "The
.exe
files are  Windows installers of the sort that you are used to". Does this
mean
everything is explained and done by the installer itself, or is there some
additional action the user should perform? Thanks!

(I have no Windows box here so I can't see for myself...)

Just

_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
http://lists.sourceforge.net/lists/listinfo/numpy-discussion




From edcjones at erols.com  Fri Aug 10 17:16:04 2001
From: edcjones at erols.com (Edward C. Jones)
Date: Fri, 10 Aug 2001 17:16:04 -0400
Subject: [Numpy-discussion] SWIG / Numeric core dump
Message-ID: <3B744F14.9080104@erols.com>

The following Python / SWIG / Numeric program dumps core.
I use RedHat Linux 7.1 on a PC. Python 2.1, Numeric 20.1.0,
SWIG 1.1 build 883, gcc 2.96. What is the problem?

The output looks like:

> doit.py
called import_array
Calling PyArray_FromDims
Segmentation fault (core dumped)
>

The program files are:
----------------------------------------------
doit.py:

#! /usr/bin/python

import Numeric, spam

spam.funct()
print 'spam.funct completed'

----------------------------------------------
spam.h:

#include "Python.h"
#include "/usr/include/python2.1/Numeric/arrayobject.h"

void funct(void);

----------------------------------------------
spam.i

%module spam

%{
#include "spam.h"
%}  

%init %{
    import_array();
    printf("called import_array\n");
%}

void funct(void);

----------------------------------------------
spammodule.c

#include "spam.h"

void funct(void)
{
    PyArrayObject *pao;
    int dims[2];

    dims[0] = 100;
    dims[1] = 200;
    printf("Calling PyArray_FromDims\n");
    pao = (PyArrayObject*) PyArray_FromDims(2, dims, PyArray_UBYTE);
    printf("Completed PyArray_FromDims\n");
    Py_DECREF(pao);
}

----------------------------------------------
swigit:

swig -python spam.i
gcc -c -Wall spammodule.c spam_wrap.c -I/usr/include/python2.1
ld -shared spammodule.o spam_wrap.o -o spam.so





From vanandel at atd.ucar.edu  Fri Aug 10 17:47:12 2001
From: vanandel at atd.ucar.edu (Joe Van Andel)
Date: Fri, 10 Aug 2001 15:47:12 -0600
Subject: [Numpy-discussion] SWIG / Numeric core dump
References: <3B744F14.9080104@erols.com>
Message-ID: <3B745660.476E49EE@atd.ucar.edu>

The best way of finding core dumps with a Python extension is to compile
the extension source with '-g', and then follow these steps.
(You may also have to compile the Numeric Extensions with '-g', to trace
into them).

% gdb /usr/bin/python2.1
(gdb) br _PyImport_LoadDynamicModule 
(gdb) cont   # repeat until your extension is loaded


(gdb) finish # to load your extension
(gdb) br wrap_myfunction
(gdb) disable 1   # don't want to break for more modules being loaded
(gdb) continue


Note: you can't set a breakpoint in your code until your module has been
loaded.


-- 
Joe VanAndel  	          
National Center for Atmospheric Research
http://www.atd.ucar.edu/~vanandel/
Internet: vanandel at ucar.edu



From edcjones at erols.com  Sat Aug 11 21:56:22 2001
From: edcjones at erols.com (Edward C. Jones)
Date: Sat, 11 Aug 2001 21:56:22 -0400
Subject: [Numpy-discussion] SWIG + Numeric 20.1.0 = core dump: the fix
Message-ID: <3B75E246.4030505@erols.com>

If
    Numeric and SWIG are used together in creating a Python extension
  and
    there are C/C++ files as part of the extension
then
  Numeric 20.1.0 will probably break your code.

This happens because changes were made to arrayobject.h between
Numeric 20.0.0 and 20.1.0.

The changes that need to be made in your code are explained
by Konrad Hinsen:
(http://www.geocrawler.com/archives/3/1329/2001/5/0/5718231/)

     FROM: Konrad Hinsen
     DATE: 05/09/2001 09:12:25
     SUBJECT:  [Numpy-discussion] Modified header files

     Recently we had a discussion about how to use NumPy arrays from
     extension modules with multiple source files, on various
     platforms.  The modified header files that are attached to this
     message provide a (hopefully!) general solution. In fact, I
     propose to make them part of the official distribution, unless
     there are objections.

     If used like before, these header files give exactly the same
     result as the ones in NumPy 20.0.0. However, they permit to define
     the name of the C API pointer array and make it globally visible.
     Under the condition that the chosen name is unique, this should
     not create problems under any platform, no matter if static or
     dynamic linking is used.

     To use NumPy features in multiple source file extension modules,
     you have to write

        #define PY_ARRAY_UNIQUE_SYMBOL PyArrayXXX
        #include "Numeric/arrayobject.h"

     in the main source file (the one that contains the init function)
     and
     
        #define PY_ARRAY_UNIQUE_SYMBOL PyArrayXXX
        #define NO_IMPORT_ARRAY
        #include "Numeric/arrayobject.h"

     in all the others. The symbol you choose instead of PyArrayXXX
     should contain both the name of the imported module (array) and
     the name of the importing module (whatever your module is called)
     in order to be unique with a reasonably high probability.

     The same applies to the Ufunc module, just replace "array" by
     "ufunc" in the example. I have also applied the "static"
     correction to the Ufunc header file, there is no reason not to do
     it.

     Konrad.
     -- 
     -----------------------------------------------------------------------
     Konrad Hinsen                            | E-Mail:
     hinsen at NO-SPAM.cnrs-orleans.fr
     Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
     Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
     45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
     France                                   | Nederlands/Francais
     -----------------------------------------------------------------------

I do this by modifying one header file that is included in all
the ".c" and ".i" files. In the following example the header file
is "spam.h".

------------------------------------------------------------
doit.py:

#! /usr/bin/python

import Numeric, spam

spam.funct()
print 'spam.funct completed'
------------------------------------------------------------
spam.h:

#include "Python.h"
#define PY_ARRAY_UNIQUE_SYMBOL Py_Array_API_spam
#ifndef SWIG_FILE_WITH_INIT
  #define NO_IMPORT_ARRAY
#endif

#include "/usr/include/python2.1/Numeric/arrayobject.h"
void funct(void);
------------------------------------------------------------
spam.i:

%module spam

%{
#define SWIG_FILE_WITH_INIT
#include "spam.h"
%}  

%init %{
import_array();
printf("called import_array\n");
%}

void funct(void);
------------------------------------------------------------
spammodule.c

#include "spam.h"

void funct(void)
{
    PyArrayObject *pao;
    int dims[2];

    dims[0] = 100;
    dims[1] = 200;
    printf("Calling PyArray_FromDims\n");
    pao = (PyArrayObject*) PyArray_FromDims(2, dims, PyArray_UBYTE);
    printf("Completed PyArray_FromDims\n");
    Py_DECREF(pao);
}
------------------------------------------------------------
compile script:

swig -python spam.i
gcc -c -Wall spammodule.c spam_wrap.c -I/usr/include/python2.1
ld -shared spammodule.o spam_wrap.o -o spam.so
------------------------------------------------------------

I use RedHat 7.1 Linux on a PC, Python 2.1, Numeric 20.1.0,
SWIG 1.1 build 883, and gcc 2.96. Anyone is welcome to use
the above example in documentation.





From huaiyu_zhu at yahoo.com  Fri Aug 17 02:30:20 2001
From: huaiyu_zhu at yahoo.com (Huaiyu Zhu)
Date: Thu, 16 Aug 2001 23:30:20 -0700 (PDT)
Subject: [Numpy-discussion] Subarray with with arbitrary index?
Message-ID: <Pine.LNX.4.10.10108162312270.1077-100000@mars.localdomain>

Hi,

Is it possible to assign to a subarray with arbitrary index?

Suppose I have three arrays

a = arange(8)
b = array([2, 3, 5])
c = arange(3)+100

I want a function f, such that calling  f(a, b, c) would change a to

[0 1 100 101 4 102 6 7]

This would be useful in certain applications that would otherwise
require sparse matrices.   



Huaiyu Zhu




From jjl at pobox.com  Fri Aug 17 09:36:26 2001
From: jjl at pobox.com (John J. Lee)
Date: Fri, 17 Aug 2001 14:36:26 +0100 (BST)
Subject: [Numpy-discussion] Subarray with with arbitrary index?
In-Reply-To: <Pine.LNX.4.10.10108162312270.1077-100000@mars.localdomain>
Message-ID: <Pine.SOL.4.30.0108171427230.15545-100000@mimosa.csv.warwick.ac.uk>

On Thu, 16 Aug 2001, Huaiyu Zhu wrote:

> Hi,
>
> Is it possible to assign to a subarray with arbitrary index?
>
> Suppose I have three arrays
>
> a = arange(8)
> b = array([2, 3, 5])
> c = arange(3)+100
>
> I want a function f, such that calling  f(a, b, c) would change a to
>
> [0 1 100 101 4 102 6 7]

f = Numeric.put
f(a, b, c)

put used to be in Python, but it's been in C since some release 17.x.x, I
think.

I have a sinking feeling that I must have missed something (no interpreter
here to check it works)...

BTW, a week ago I noticed that I had reinvented the wheel in rewriting, in
an uglier and less efficient form, Numeric.allclose (hope I got the name
right).  As far as I can see, it isn't listed in the manual.  Did I miss
it?  All it would need is the docstring copying over.


John




From paul at pfdubois.com  Fri Aug 17 11:23:28 2001
From: paul at pfdubois.com (Paul F. Dubois)
Date: Fri, 17 Aug 2001 08:23:28 -0700
Subject: [Numpy-discussion] Subarray with with arbitrary index?
In-Reply-To: <Pine.SOL.4.30.0108171427230.15545-100000@mimosa.csv.warwick.ac.uk>
Message-ID: <ADEOIFHFONCLEEPKCACCKEBNCMAA.paul@pfdubois.com>

John is right:
>>> a=Numeric.arange(8)
>>> b=Numeric.array([2,3,5])
>>> c=Numeric.arange(3)+100
>>> Numeric.put(a,b,c)
>>> print a
[  0   1 100 101   4 102   6   7]

Thanks for pointing out that I had left allclose out of the Numeric part of
the manual. I did it in the MA part and then forgot. I'm fixing it now. BTW:
There are changenotes at source forge that are sometimes ahead of the
manual.

-----Original Message-----
From: numpy-discussion-admin at lists.sourceforge.net
[mailto:numpy-discussion-admin at lists.sourceforge.net]On Behalf Of John
J. Lee
Sent: Friday, August 17, 2001 6:36 AM
To: Huaiyu Zhu
Cc: numpy-discussion at lists.sourceforge.net
Subject: Re: [Numpy-discussion] Subarray with with arbitrary index?


On Thu, 16 Aug 2001, Huaiyu Zhu wrote:

> Hi,
>
> Is it possible to assign to a subarray with arbitrary index?
>
> Suppose I have three arrays
>
> a = arange(8)
> b = array([2, 3, 5])
> c = arange(3)+100
>
> I want a function f, such that calling  f(a, b, c) would change a to
>
> [0 1 100 101 4 102 6 7]

f = Numeric.put
f(a, b, c)

put used to be in Python, but it's been in C since some release 17.x.x, I
think.

I have a sinking feeling that I must have missed something (no interpreter
here to check it works)...

BTW, a week ago I noticed that I had reinvented the wheel in rewriting, in
an uglier and less efficient form, Numeric.allclose (hope I got the name
right).  As far as I can see, it isn't listed in the manual.  Did I miss
it?  All it would need is the docstring copying over.


John


_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
http://lists.sourceforge.net/lists/listinfo/numpy-discussion




From paul at pfdubois.com  Sat Aug 18 12:29:23 2001
From: paul at pfdubois.com (Paul F. Dubois)
Date: Sat, 18 Aug 2001 09:29:23 -0700
Subject: [Numpy-discussion] Subarray with with arbitrary index?
In-Reply-To: <Pine.LNX.4.10.10108172206320.972-100000@mars.localdomain>
Message-ID: <000001c12802$f12a2040$0301a8c0@plstn1.sfba.home.com>

I'm afraid that you have to do put using one-d indices. But you do *not*
have to try to ravel the source. I.e., the first arg is just the name of
the array.
>>> from Numeric import *
>>> x=array([[1,2,3],[4,5,6]])
>>> x
array([[1, 2, 3],
       [4, 5, 6]])
>>> put(x,[0,4],[100,200])
>>> x
array([[100,   2,   3],
       [  4, 200,   6]])
>>>
-----Original Message-----
From: Huaiyu Zhu [mailto:huaiyu_zhu at yahoo.com] 
Sent: Friday, August 17, 2001 11:36 PM
To: Paul F. Dubois
Cc: John J. Lee
Subject: RE: [Numpy-discussion] Subarray with with arbitrary index?

Thanks, John and Paul.  That is what I was looking for.

It did not occur to me to look for verbs put and take, rather than words
line sub- index, slice and so on.  Maybe puting some of these words in
the manual could help people doing a search?

Now that this made the most costly part of my program about seven times
faster, other problems become more prominent.  One of such question is:
How do we do it on more than one axis?

Suppose a is a 2d array. Then put(a[1,:], b, c) works, but put(a[:,1],
b,
c) complains about the first argument not a continuous array.  Doing
transpose does not help.  So do I guess it right that this is
implemented
only in the representation of a linear array?

If so, there would be no hope of using put(a, ([2, 4], [1,2]), v) or
even
more exotic ones like using += on an arbitray subgrid?

Regards,

Huaiyu

On Fri, 17 Aug 2001, Paul F. Dubois wrote:

> John is right:
> >>> a=Numeric.arange(8)
> >>> b=Numeric.array([2,3,5])
> >>> c=Numeric.arange(3)+100
> >>> Numeric.put(a,b,c)
> >>> print a
> [  0   1 100 101   4 102   6   7]
> 
> Thanks for pointing out that I had left allclose out of the Numeric
part of
> the manual. I did it in the MA part and then forgot. I'm fixing it
now. BTW:
> There are changenotes at source forge that are sometimes ahead of the
> manual.
> 
> -----Original Message-----
> From: numpy-discussion-admin at lists.sourceforge.net
> [mailto:numpy-discussion-admin at lists.sourceforge.net]On Behalf Of John
> J. Lee
> Sent: Friday, August 17, 2001 6:36 AM
> To: Huaiyu Zhu
> Cc: numpy-discussion at lists.sourceforge.net
> Subject: Re: [Numpy-discussion] Subarray with with arbitrary index?
> 
> 
> On Thu, 16 Aug 2001, Huaiyu Zhu wrote:
> 
> > Hi,
> >
> > Is it possible to assign to a subarray with arbitrary index?
> >
> > Suppose I have three arrays
> >
> > a = arange(8)
> > b = array([2, 3, 5])
> > c = arange(3)+100
> >
> > I want a function f, such that calling  f(a, b, c) would change a to
> >
> > [0 1 100 101 4 102 6 7]
> 
> f = Numeric.put
> f(a, b, c)
> 
> put used to be in Python, but it's been in C since some release
17.x.x, I
> think.
> 
> I have a sinking feeling that I must have missed something (no
interpreter
> here to check it works)...
> 
> BTW, a week ago I noticed that I had reinvented the wheel in
rewriting, in
> an uglier and less efficient form, Numeric.allclose (hope I got the
name
> right).  As far as I can see, it isn't listed in the manual.  Did I
miss
> it?  All it would need is the docstring copying over.
> 
> 
> John
> 
> 
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> http://lists.sourceforge.net/lists/listinfo/numpy-discussion
> 





From paul at pfdubois.com  Sat Aug 18 12:33:08 2001
From: paul at pfdubois.com (Paul F. Dubois)
Date: Sat, 18 Aug 2001 09:33:08 -0700
Subject: [Numpy-discussion] MA patch submitter, please upload file
Message-ID: <000101c12803$76f70e40$0301a8c0@plstn1.sfba.home.com>

Would the person who submitted the patch concerning MA binary ops,
please upload the file to the patch, or email it to me? You have
identified a real problem whose answer is not obvious so I'd like to see
your idea of how to fix it.
 
If you log in to SourceForge before submitting bugs and patches then I
can know who you are to thank you or make followup inquiries like this
one.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20010818/76388cce/attachment.html>

From ehagemann at home.com  Sat Aug 18 20:21:27 2001
From: ehagemann at home.com (Eric Hagemann)
Date: Sat, 18 Aug 2001 20:21:27 -0400
Subject: [Numpy-discussion] For the next round on the manual
Message-ID: <006801c12844$e384e870$6401a8c0@cc911046a>

Type-o in the manual

    Check the definition for the 'convolve' function.  Manual tells of mode=0 as default when code and experiment point to mode=2 being default

    No biggie -- but it might be nice to correct if anybody is editing the manual any time soon

Cheers
Eric

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20010818/92c6568d/attachment.html>

From karshi.hasanov at utoronto.ca  Sun Aug 19 14:17:36 2001
From: karshi.hasanov at utoronto.ca (Karshi Hasanov)
Date: Sun, 19 Aug 2001 14:17:36 -0400
Subject: [Numpy-discussion] Matlab_NumPy_Array?
Message-ID: <01081914173600.00854@cr322577-a>

Hi all,

 I  don't understand how to conver arrays from Matlab to Numpy. Here what I 
did: 1) Converted Matlab A(i,j,k) to 'A.dat' binary file
      2) file_open=open('~/A.dat', 'rb')
      3) read_file=file_open.read()
      4) A_string=fromstring(read_file, 'f')
      5) A=reshape(A_string, (i,j,k))
where i,j,k are size of A[i,j,k]

I am not getting the right. 
What is the best/right way of convering arrays?
Thanks



From gball at cfa.harvard.edu  Wed Aug 22 19:04:09 2001
From: gball at cfa.harvard.edu (Greg Ball)
Date: Wed, 22 Aug 2001 19:04:09 -0400 (EDT)
Subject: [Numpy-discussion] Matlab_NumPy_Array?
In-Reply-To: <01081914173600.00854@cr322577-a>
Message-ID: <Pine.OSF.4.10.10108221901390.8371-100000@surya.harvard.edu>

Try replacing step 4 with

A_string=fromstring(read_file, 'd')

'f' is for 32 bit floats, 'd' is for 64 bit floats.
Matlab uses 64 bit (double) floats.

HTH

--Greg Ball



On Sun, 19 Aug 2001, Karshi Hasanov wrote:

> 
> Hi all,
> 
>  I  don't understand how to conver arrays from Matlab to Numpy. Here what I 
> did: 1) Converted Matlab A(i,j,k) to 'A.dat' binary file
>       2) file_open=open('~/A.dat', 'rb')
>       3) read_file=file_open.read()
>       4) A_string=fromstring(read_file, 'f')
>       5) A=reshape(A_string, (i,j,k))
> where i,j,k are size of A[i,j,k]
> 
> I am not getting the right. 
> What is the best/right way of convering arrays?
> Thanks
> 
> 
> 
> 




From jsw at cdc.noaa.gov  Thu Aug 23 11:22:17 2001
From: jsw at cdc.noaa.gov (Jeff Whitaker)
Date: Thu, 23 Aug 2001 09:22:17 -0600 (MDT)
Subject: [Numpy-discussion] patch to speed up LinearAlgebra module
Message-ID: <Pine.OSX.4.33.0108230915100.23509-100000@crdmac10.cdc.noaa.gov>

I've uploaded a patch to LinearAlgebra.py and lapack_litemodule.c that
speeds up some computations (svd, linear least-squares, and hermitian
eigenanalysis) by 3-5 times when linking with external (preferably
optimized) blas and lapack libs.  The patch replaces calls to dgesvd,
dgelss and dsyev with the newer, faster lapack routines dgesdd, dgelsd and
dsyevd.  Here are my timing results before and after (G4 350mhz powermac,
optimized ATLAS blas lib and lapack 3.0 from netlib.org):

Before:

svd of 1000X1000 matrix takes 609.19
generalized inverse of 1000X1000 matrix takes 744.36
linear least-squares solution of 1000X1000 matrix takes
451.68
eigenvectors of 1000X1000 symmetric matrix takes 210.08

After:

svd of 1000X1000 matrix takes 142.55
generalized inverse of 1000X1000 matrix takes 237.08
linear least-squares solution of 1000X1000 matrix takes
81.91
eigenvectors of 1000X1000 symmetric matrix takes 56.23


Note that since these newer lapack routines are not in lapack_lite, you
must link external blas and lapack3 libs.

-Jeff

-- 
Jeffrey S. Whitaker         Phone  : (303)497-6313
Meteorologist               FAX    : (303)497-6449
NOAA/OAR/CDC  R/CDC1        Email  : jsw at cdc.noaa.gov
325 Broadway                Web    : www.cdc.noaa.gov/~jsw
Boulder, CO, USA 80303-3328 Office : Skaggs Research Cntr 1D-124




From chrishbarker at home.net  Thu Aug 23 12:42:14 2001
From: chrishbarker at home.net (Chris Barker)
Date: Thu, 23 Aug 2001 09:42:14 -0700
Subject: [Numpy-discussion] C API questions and tricks.
References: <000001c12802$f12a2040$0301a8c0@plstn1.sfba.home.com>
Message-ID: <3B853266.A5B36D8F@home.net>

Hi all,

I recently discovered the "clip()" function, and thought it was just
what I'd been wanting, because I need to do that a lot, and it provided
a nice notation, and I was hoping speed improvement over a couple of
where() statements.

I did some experiments, and discovered that it was, in fact, slower than
the where statements, and that the fastest way to do it is to use two
putmask() calls. (note: this changes the array in place, rather than
creating a new one)

The reason it is not faster is because it is written in Python, calling
choose(), similarly to how where() does. I decided that I could use a
fast version, so I set out to write one in C, resulting in the following
questions:

A) How do I loop through all the elements of a discontiguous array of
arbitraty dimensions? If I knew the rank ahead of time, I could just
nest some for loops and use the strides[] values. Not knowing before
hand, it seems that I should be able to do some nesting of loops using
nd and dimensions[], but I can't seem to work it out. Someone must have
come up with a nifty way to do this. Is there an existing macro or
function to do it?

B) How can I write a function that can act on any of the NumPy data
types? I currently have it written to only work with contiguous Float
arrays, which is what i need at the moment, but I'd much rather have one
for the general case.

C) I'd also like any feedback on other elements of my code (enclosed
with this email). A few comments on what the function (I've called it
fastclip) is supposed to do:

"""
fastclip(A,min,max)

changes the array, A, in place, so that all the elements less than min
are replaced by min, and all the elements greater that max are replaced
by max.

min and max can be either scalars, or anything that can be converted to
an array with the same number of elements as A (using
PyArray_ContiguousFromObject() ). If min and/or max is an array, than
the coresponding elements are used. This allows, among other things, a
way to clip to just a min or max value by calling it as:
fastclip(A,A,max) or fastclip(A,min,A).

"""

I wrote a little test script to benchmark the function, and it is much
faster that the alternatives that I have thought of:

#!/usr/bin/env python
# testing speed of where vs clip vs fastclip

from Numeric import *
from RandomArray import uniform
from NumericExtras import fastclip
import time

n = 5000

a = uniform(0,100,(n,))
b = uniform(0,100,(n,))
c = uniform(0,100,(n,))

min = 20.0
max = 80.0

print "n = %i"%(n,)

start = time.clock()
for i in range(100):
    a = clip(a,min,max)
print "clip took %f seconds"%(time.clock()-start)

start = time.clock()
for i in range(100):
    putmask(a,a < min,min)
    putmask(a,a > max,max)
print "putmask took %f seconds"%(time.clock()-start)

start = time.clock()
for i in range(100):
    fastclip(a,min,max)
print "fastclip took %f seconds"%(time.clock()-start)

Here are some results:

n = 50
clip took 0.020000 seconds
putmask took 0.050000 seconds
fastclip took 0.010000 seconds

n = 5000
clip took 0.300000 seconds
putmask took 0.230000 seconds
fastclip took 0.030000 seconds

As expected the large the array is, the better the improvement.

I'd love to here any feedbackyou can give me: I've new to writing Python
extensions, using the  Numeric API, and C itself, for that matter.

-thanks, Chris



















-- 
Christopher Barker,
Ph.D.                                                           
ChrisHBarker at home.net                 ---           ---           ---
http://members.home.net/barkerlohmann ---@@       -----@@       -----@@
                                   ------@@@     ------@@@     ------@@@
Oil Spill Modeling                ------   @    ------   @   ------   @
Water Resources Engineering       -------      ---------     --------    
Coastal and Fluvial Hydrodynamics --------------------------------------
------------------------------------------------------------------------
-------------- next part --------------

#include "Python.h"
#include "arrayobject.h"

// This define is to accomidate the different ways shared libs are built (see the bottom of the file)

#define _LINUX // this shouild be one of:  _LINUX, _WIN32, or _MACOS

// These are little macros I use to access array values for specific rank arrays
#define ARRAYVAL0(aType,a) ( *(aType *)(a->data))	
#define ARRAYVAL1(aType,a,i) ( *(aType *)(a->data + (i)*a->strides[0]))	
#define ARRAYVAL2(aType,a,i,j) ( *(aType *)(a->data + (i)*a->strides[0] + (j)*a->strides[1]))	
#define ARRAYVAL3(aType,a,i,j,k) ( *(aType *)(a->data + (i)*a->strides[0] + (j)*a->strides[1] + (k)*a->strides[2]))	



// A function that computes the total number of elements in and array 
// Does this exist already?
int TotalNumberOfElements(PyArrayObject *Array)
{
  int total = 1;
  short i;
  
  for (i = 0; i < Array->nd; i++)
    {
      total *= Array->dimensions[i];
    }
  return total;
}

// This is the fastclip code:
// called from Python as: fastclip(Array,min,max)
// min, max can be scalar or the same size as Array.
// At the moment, Array has to be contiguous, and it only works with arrays of type Float.
static PyObject * NumericExtras_fastclip(PyObject *self, PyObject *args)
{
  
  int NumArray, elementsize, i;

  PyObject *InMin;
  PyObject *InMax;

  PyArrayObject *Array;
  PyArrayObject *Min;
  PyArrayObject *Max;

  //printf("I'm starting\n");

  if (!PyArg_ParseTuple(args, "O!OO", 
						&PyArray_Type, &Array,
					    &InMin,
						&InMax))
  {
    return NULL;
  }  
  
  // check types of input

  // check type of Array
  if (!PyArray_ISCONTIGUOUS(Array)){
	PyErr_SetString (PyExc_ValueError,
					 "a must be contiguous");
	return NULL;
  }
                            
  if (Array->descr->type_num != PyArray_DOUBLE){
	PyErr_SetString (PyExc_ValueError,
					 "a must be a NumPy array of type Float");
	return NULL;
  }

  // convert min and max to arrays:
  // if they don't convert, the input values are not valid
  Min = (PyArrayObject *) PyArray_ContiguousFromObject(InMin, PyArray_DOUBLE, 0, 0);
  if (Min == NULL){
    PyErr_SetString (PyExc_ValueError,
					 "min must be either a scalar or the same size as a");
    return NULL;
  }
  Max = (PyArrayObject *) PyArray_ContiguousFromObject(InMax, PyArray_DOUBLE, 0, 0);
  if (Max == NULL){
    PyErr_SetString (PyExc_ValueError,
					 "max must be either a scalar or the same size as a");
    Py_DECREF(Min);
    return NULL;
  }


  // This seems pretty kludgy to have the four cases, but it was the first
  // thing that came to mind that I was sure would work, and would accomidate
  // either array or scalar arguments for min and max. 
  NumArray = TotalNumberOfElements(Array);
  elementsize = sizeof(double);

  if (TotalNumberOfElements(Min) == 1 && TotalNumberOfElements(Max) == 1){ // both limits are scalar
    for (i = 0; i < NumArray; i++) { // loop over each element
      if (*(double *)(Array->data + i*elementsize) > ARRAYVAL0(double,Max)){
         *(double *)(Array->data + i*elementsize) = ARRAYVAL0(double,Max);
          }
      else if (*(double *)(Array->data + i*elementsize) < ARRAYVAL0(double,Min)){
         *(double *)(Array->data + i*elementsize) = ARRAYVAL0(double,Min);
      }
    }
  }
  else if (TotalNumberOfElements(Min) == 1 && TotalNumberOfElements(Max) == NumArray){ // Min is scalar
    for (i = 0; i < NumArray; i++) { // loop over each element
      if (*(double *)(Array->data + i*elementsize) > *(double *)(Max->data + i*elementsize)){
         *(double *)(Array->data + i*elementsize) = *(double *)(Max->data + i*elementsize) ;
          }
      else if (*(double *)(Array->data + i*elementsize) < ARRAYVAL0(double,Min)){
         *(double *)(Array->data + i*elementsize) = ARRAYVAL0(double,Min);
      }
    }
  }    
  else if (TotalNumberOfElements(Max) == 1 && TotalNumberOfElements(Min) == NumArray){ // Max is scalar
    for (i = 0; i < NumArray; i++) { // loop over each element
      if (*(double *)(Array->data + i*elementsize) > ARRAYVAL0(double,Max)){
         *(double *)(Array->data + i*elementsize) = ARRAYVAL0(double,Max);
          }
      else if (*(double *)(Array->data + i*elementsize) < *(double *)(Min->data + i*elementsize)){
         *(double *)(Array->data + i*elementsize) = *(double *)(Min->data + i*elementsize) ;
      }
    }
  }    
  else if (TotalNumberOfElements(Min) == NumArray && TotalNumberOfElements(Max) == NumArray){ // Neither is scalar
    for (i = 0; i < NumArray; i++) { // loop over each element
      if (*(double *)(Array->data + i*elementsize) > *(double *)(Max->data + i*elementsize)){
         *(double *)(Array->data + i*elementsize) = *(double *)(Max->data + i*elementsize) ;
          }
      else if (*(double *)(Array->data + i*elementsize) < *(double *)(Min->data + i*elementsize)){
         *(double *)(Array->data + i*elementsize) = *(double *)(Min->data + i*elementsize) ;
      }
    }    
  }
  else { // The size of Min and/or Max don't match Array
    	PyErr_SetString (PyExc_ValueError,
					 "min and max must be either scalar or the same number of elements as a");
        Py_DECREF(Min);
        Py_DECREF(Max);
        return NULL;
  }

  return Py_None;
}
  

	
static PyMethodDef NumericExtrasMethods[] = {
  {"fastclip", NumericExtras_fastclip, METH_VARARGS},
  {NULL, NULL} /* Sentinel */
};


#if defined _WIN32
  void _declspec(dllexport)
#elif defined _LINUX
  void 
#elif defined _MACOS
  void
#else
  #error There is no valid platform defined (should be in platfrom.h and be one of: _WIN32,_LINUX,_MACOS)
#endif

initNumericExtras()

{
  (void) Py_InitModule("NumericExtras", NumericExtrasMethods);
  import_array()
}





















From paul at pfdubois.com  Thu Aug 23 13:41:40 2001
From: paul at pfdubois.com (Paul F. Dubois)
Date: Thu, 23 Aug 2001 10:41:40 -0700
Subject: [Numpy-discussion] For the next round on the manual
In-Reply-To: <006801c12844$e384e870$6401a8c0@cc911046a>
Message-ID: <000101c12bfa$de43fae0$0301a8c0@plstn1.sfba.home.com>

Fixed in my manual source, thank you. 
 
-----Original Message-----
From: numpy-discussion-admin at lists.sourceforge.net
[mailto:numpy-discussion-admin at lists.sourceforge.net] On Behalf Of Eric
Hagemann
Sent: Saturday, August 18, 2001 5:21 PM
To: Numpy-discussion at lists.sourceforge.net
Subject: [Numpy-discussion] For the next round on the manual
 
Type-o in the manual
 
    Check the definition for the 'convolve' function.  Manual tells of
mode=0 as default when code and experiment point to mode=2 being default
 
    No biggie -- but it might be nice to correct if anybody is editing
the manual any time soon
 
Cheers
Eric
 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20010823/51079634/attachment.html>

From chrishbarker at home.net  Thu Aug 23 15:19:33 2001
From: chrishbarker at home.net (Chris Barker)
Date: Thu, 23 Aug 2001 12:19:33 -0700
Subject: [Numpy-discussion] For the next round on the manual
References: <000101c12bfa$de43fae0$0301a8c0@plstn1.sfba.home.com>
Message-ID: <3B855745.2FE519D6@home.net>

While you're at it, I found what appears to be another small typo:

In the API section, description of the ArrayObject structure, is the
line:

a.shape = (dimensions[0]...dimensions[nd])


shouldn't this be dimensions[nd-1] ?


-Chris


-- 
Christopher Barker,
Ph.D.                                                           
ChrisHBarker at home.net                 ---           ---           ---
http://members.home.net/barkerlohmann ---@@       -----@@       -----@@
                                   ------@@@     ------@@@     ------@@@
Oil Spill Modeling                ------   @    ------   @   ------   @
Water Resources Engineering       -------      ---------     --------    
Coastal and Fluvial Hydrodynamics --------------------------------------
------------------------------------------------------------------------



From rob at hooft.net  Fri Aug 24 02:49:40 2001
From: rob at hooft.net (Rob W. W. Hooft)
Date: Fri, 24 Aug 2001 08:49:40 +0200
Subject: [Numpy-discussion] C API questions and tricks.
In-Reply-To: <3B853266.A5B36D8F@home.net>
References: <000001c12802$f12a2040$0301a8c0@plstn1.sfba.home.com>
	<3B853266.A5B36D8F@home.net>
Message-ID: <15237.63748.700849.145167@temoleh.chem.uu.nl>

>>>>> "CB" == Chris Barker <chrishbarker at home.net> writes:

I wrote my own specialized clip function too, only for 'f' and 'i'
types, and only for contiguous arrays....

 CB> A) How do I loop through all the elements of a discontiguous
 CB> array of arbitraty dimensions? If I knew the rank ahead of time,
 CB> I could just nest some for loops and use the strides[]
 CB> values. Not knowing before hand, it seems that I should be able
 CB> to do some nesting of loops using nd and dimensions[], but I
 CB> can't seem to work it out. Someone must have come up with a nifty
 CB> way to do this. Is there an existing macro or function to do it?

Never done this in C for numpy yet, but I normally program this
along the following lines:

n=[2,3,2,3]
x=[0,0,0,0]
while 1:
    print x
    i=len(n)-1
    while i>=0 and x[i]+1==n[i]:
       x[i]=0
       i=i-1
    if i<0:
       break
    x[i]=x[i]+1

It is always going to be slower than the straight loop for contiguous
arrays. So if you want to do it properly, you'd have to check whether
the array is contiguous, and if it is, use the fast way.

Regards,

Rob Hooft

-- 
=====   rob at hooft.net                 http://www.hooft.net/people/rob/  =====
=====   R&D, Bruker Nonius BV, Delft  http://www.nonius.nl/             =====
===== PGPid 0xFA19277D ================================= Use Linux! =========



From paul at pfdubois.com  Fri Aug 24 14:45:43 2001
From: paul at pfdubois.com (Paul F. Dubois)
Date: Fri, 24 Aug 2001 11:45:43 -0700
Subject: [Numpy-discussion] manual update
Message-ID: <000201c12ccc$fc388e80$0301a8c0@plstn1.sfba.home.com>

I have updated the Numeric manual on the web site. I added some to the
MLab section to reflect the axis arguments that were added some time ago
but never reflected in the manual. The manual also documents a change to
the behavior of the attribute "masked" in package MA. The document
corresponds to CVS now rather than the last release. I hope to release
Numeric-20.2 sometime soon but want more exercise of the MA and MLab
changes first.
 
I changed MLab.mean to return a floating point result when the argument
is integer. I'm sorry if this breaks anything but the prior behavior has
no mathematical meaning and then, worse, was being used by std to get
the wrong standard deviation too. I changed std to work for axis > 0, as
it was previously broken.
 
I don't own Matlab and so maintaining a package that emulates something
I don't have makes me nervous. I wish one of the other developers with
more knowledge in this area would pitch in.
 
Thanks to Chris Barker and others who sent in typos and suggestions.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20010824/8b3578c9/attachment.html>

From europax at home.com  Fri Aug 24 16:14:52 2001
From: europax at home.com (Rob)
Date: Fri, 24 Aug 2001 13:14:52 -0700
Subject: [Numpy-discussion] manual update
References: <000201c12ccc$fc388e80$0301a8c0@plstn1.sfba.home.com>
Message-ID: <3B86B5BC.4AF0CB65@home.com>

Re Matlab,

I've never used it before, but I was very suprised to see that its
arrays work much like Numpy.  I have some Matlab FDTD code here and the
arrays are sliced instead of indexed.  I might be able to rope a few
more people at work into Python based on that :)    Rob.



From edcjones at erols.com  Sat Aug 25 11:16:33 2001
From: edcjones at erols.com (Edward C. Jones)
Date: Sat, 25 Aug 2001 11:16:33 -0400
Subject: [Numpy-discussion] HTML manual display problems in Mozilla
Message-ID: <3B87C151.4040108@erols.com>

Mozilla .0.9.3 is not correctly displaying the HTML version of the 
manual. The problem appears to be in footnotes. Netscape 4.76 had 
similar problems with the previous version of the manual. I use RedHat 
7.1 Linux on a PC.

Thanks,
Ed Jones





From europax at home.com  Sun Aug 26 11:38:56 2001
From: europax at home.com (Rob)
Date: Sun, 26 Aug 2001 08:38:56 -0700
Subject: [Numpy-discussion] Can this function by Numpy-ized?
Message-ID: <3B89180F.CD83EA10@home.com>

I finally got my FEM EM code working.  I profiled it and this function
uses up a big hunk of time.  It performs gaussian integration over a
triangle.  I am trying to figure out how to slice the arrays so as to
push it down into the C level.  Does anyone have any ideas?  Thanks, 
Rob.

ps. it looks to be intractible to me.  Maybe I need to look at writing a
C extension.  I've never done that before.


##***************************************************************************
##Prototype:  void  ComputeGaussQuadPoint(int QuadPoint, int
*a.TrngleNode,
##                                            double *SrcPointCol) 
##Description: To compute the coordinates of 7-point Gauss nodes of 
##             a triangular patch. 
##Input value: 
##    int QuadPoint  --- node index, it can be from 0 to 6. 
##    int *a.TrngleNode  ---  the three nodes of a tringular patch. 
##    double *SrcPointCol --- where to store the results 
##Return value: none 
##Global value used: a.NodeCord, a.Qpnt 
##Global value modified:    none 
##Subroutines called:    none 
##Note: Not very sure.           
****************************************************************************
def ComputeGaussQuadPoint(QuadPoint,TrngleNode,a):
    
    SrcPointCol=zeros((3),Float)
    
    SrcPointCol[0] =   a.Qpnt[QuadPoint,0]*a.NodeCord[TrngleNode[0],0]\
                     + a.Qpnt[QuadPoint,1]*a.NodeCord[TrngleNode[1],0]\
                     + a.Qpnt[QuadPoint,2]*a.NodeCord[TrngleNode[2],0]

    SrcPointCol[1] =   a.Qpnt[QuadPoint,0]*a.NodeCord[TrngleNode[0],1]\
                     + a.Qpnt[QuadPoint,1]*a.NodeCord[TrngleNode[1],1]\
                     + a.Qpnt[QuadPoint,2]*a.NodeCord[TrngleNode[2],1]

    SrcPointCol[2] =   a.Qpnt[QuadPoint,0]*a.NodeCord[TrngleNode[0],2]\
                     + a.Qpnt[QuadPoint,1]*a.NodeCord[TrngleNode[1],2]\
                     + a.Qpnt[QuadPoint,2]*a.NodeCord[TrngleNode[2],2]
                                     

    return SrcPointCol
  





-- 
The Numeric Python EM Project

www.members.home.net/europax



From jjl at pobox.com  Sun Aug 26 13:27:08 2001
From: jjl at pobox.com (John J. Lee)
Date: Sun, 26 Aug 2001 18:27:08 +0100 (BST)
Subject: [Numpy-discussion] Can this function by Numpy-ized?
In-Reply-To: <3B89180F.CD83EA10@home.com>
Message-ID: <Pine.SOL.4.30.0108261755160.10385-100000@mimosa.csv.warwick.ac.uk>

On Sun, 26 Aug 2001, Rob wrote:

> I finally got my FEM EM code working.  I profiled it and this function
> uses up a big hunk of time.  It performs gaussian integration over a
> triangle.  I am trying to figure out how to slice the arrays so as to
> push it down into the C level.  Does anyone have any ideas?  Thanks,
> Rob.
>
> ps. it looks to be intractible to me.  Maybe I need to look at writing a
> C extension.  I've never done that before.
[...]

I'm not a good enough ufunc / array function hacker to come up with
anything, but I doubt it, and I wonder how much time it would save anyway,
given the size of the arrays involved.  From the look of your comments
before the function, it looks like a) you're a C programmer, not quite
comfortable with Python yet, or b) you wrote it in C first, then moved it
into Python.  If the latter, you might want to try wrapping that C
function with SWIG, though to be honest I'm not sure the overhead of
learning to use SWIG is any less than for writing a C extension manually
(but less error-prone I expect, and less work if you have a lot of C to
wrap).  I think the Scipy project has some typemaps for passing Numeric
arrays; if not, I've seen some collections of SWIG typemaps for Numeric
somewhere...

BTW, the best place to put those comments is in a docstring.  Here is a
slightly more Pythonically-formatted version (I must be bored today):

def ComputeGaussQuadPoint(QuadPoint, a):
    """Return coordinates of 7-point Gauss nodes of a triangular patch.

    QuadPoint: node index, from 0 to 6
    a: triangular patch?
    """
    SrcPointCol=zeros((3),Float)
    tn = a.TrngleNode  # the three nodes of the triangular patch
    SrcPointCol[0] = (a.Qpnt[QuadPoint,0]*a.NodeCord[tn[0],0] +
                      a.Qpnt[QuadPoint,1]*a.NodeCord[tn[1],0] +
                      a.Qpnt[QuadPoint,2]*a.NodeCord[tn[2],0])

    SrcPointCol[1] = (a.Qpnt[QuadPoint,0]*a.NodeCord[tn[0],1] +
                      a.Qpnt[QuadPoint,1]*a.NodeCord[tn[1],1] +
                      a.Qpnt[QuadPoint,2]*a.NodeCord[tn[2],1])

    SrcPointCol[2] = (a.Qpnt[QuadPoint,0]*a.NodeCord[tn[0],2] +
                      a.Qpnt[QuadPoint,1]*a.NodeCord[tn[1],2] +
                      a.Qpnt[QuadPoint,2]*a.NodeCord[tn[2],2])

    return SrcPointCol


John




From paul at pfdubois.com  Sun Aug 26 13:32:28 2001
From: paul at pfdubois.com (Paul F. Dubois)
Date: Sun, 26 Aug 2001 10:32:28 -0700
Subject: [Numpy-discussion] Can this function by Numpy-ized?
In-Reply-To: <3B89180F.CD83EA10@home.com>
Message-ID: <000001c12e55$144a5c40$0301a8c0@plstn1.sfba.home.com>

def cgqp(QuadPoint, TrngleNode, a):
    s = a.Qpnt[Quadpoint,:]
    c = Numeric.take(a.NodeCord, TrngleNode)
    return Numeric.add.reduce(s * c, axis=1)

This may or may not be right. The data structures I would have to set up
to test it are too much for Sunday morning.

-----Original Message-----
From: numpy-discussion-admin at lists.sourceforge.net
[mailto:numpy-discussion-admin at lists.sourceforge.net] On Behalf Of Rob
Sent: Sunday, August 26, 2001 8:39 AM
To: numpy-discussion at lists.sourceforge.net.
Subject: [Numpy-discussion] Can this function by Numpy-ized?

I finally got my FEM EM code working.  I profiled it and this function
uses up a big hunk of time.  It performs gaussian integration over a
triangle.  I am trying to figure out how to slice the arrays so as to
push it down into the C level.  Does anyone have any ideas?  Thanks, 
Rob.

ps. it looks to be intractible to me.  Maybe I need to look at writing a
C extension.  I've never done that before.


##**********************************************************************
*****
##Prototype:  void  ComputeGaussQuadPoint(int QuadPoint, int
*a.TrngleNode,
##                                            double *SrcPointCol) 
##Description: To compute the coordinates of 7-point Gauss nodes of 
##             a triangular patch. 
##Input value: 
##    int QuadPoint  --- node index, it can be from 0 to 6. 
##    int *a.TrngleNode  ---  the three nodes of a tringular patch. 
##    double *SrcPointCol --- where to store the results 
##Return value: none 
##Global value used: a.NodeCord, a.Qpnt 
##Global value modified:    none 
##Subroutines called:    none 
##Note: Not very sure.           
************************************************************************
****
def ComputeGaussQuadPoint(QuadPoint,TrngleNode,a):
    
    SrcPointCol=zeros((3),Float)
    
    SrcPointCol[0] =   a.Qpnt[QuadPoint,0]*a.NodeCord[TrngleNode[0],0]\
                     + a.Qpnt[QuadPoint,1]*a.NodeCord[TrngleNode[1],0]\
                     + a.Qpnt[QuadPoint,2]*a.NodeCord[TrngleNode[2],0]

    SrcPointCol[1] =   a.Qpnt[QuadPoint,0]*a.NodeCord[TrngleNode[0],1]\
                     + a.Qpnt[QuadPoint,1]*a.NodeCord[TrngleNode[1],1]\
                     + a.Qpnt[QuadPoint,2]*a.NodeCord[TrngleNode[2],1]

    SrcPointCol[2] =   a.Qpnt[QuadPoint,0]*a.NodeCord[TrngleNode[0],2]\
                     + a.Qpnt[QuadPoint,1]*a.NodeCord[TrngleNode[1],2]\
                     + a.Qpnt[QuadPoint,2]*a.NodeCord[TrngleNode[2],2]
                                     

    return SrcPointCol
  





-- 
The Numeric Python EM Project

www.members.home.net/europax

_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
http://lists.sourceforge.net/lists/listinfo/numpy-discussion




From europax at home.com  Sun Aug 26 16:17:06 2001
From: europax at home.com (Rob)
Date: Sun, 26 Aug 2001 13:17:06 -0700
Subject: [Numpy-discussion] Can this function by Numpy-ized?
References: <000001c12e55$144a5c40$0301a8c0@plstn1.sfba.home.com>
Message-ID: <3B895942.F527D958@home.com>

Thanks Paul,  I will try it out.  That at least gives me some
direction.  I've been pouring over the Numpy manual, but there are so
many different functions!  I would like to try to write an extension
that includes some of these FEM routines, but thats for later.  

BTW, at work I have to use Windows NT, but I've grown to love the
calldll/DynWin extension.  You can even interact with the windows kernel
with that one.   Rob.


"Paul F. Dubois" wrote:
> 
> def cgqp(QuadPoint, TrngleNode, a):
>     s = a.Qpnt[Quadpoint,:]
>     c = Numeric.take(a.NodeCord, TrngleNode)
>     return Numeric.add.reduce(s * c, axis=1)
> 
> This may or may not be right. The data structures I would have to set up
> to test it are too much for Sunday morning.
> 
> -----Original Message-----
> From: numpy-discussion-admin at lists.sourceforge.net
> [mailto:numpy-discussion-admin at lists.sourceforge.net] On Behalf Of Rob
> Sent: Sunday, August 26, 2001 8:39 AM
> To: numpy-discussion at lists.sourceforge.net.
> Subject: [Numpy-discussion] Can this function by Numpy-ized?
> 
> I finally got my FEM EM code working.  I profiled it and this function
> uses up a big hunk of time.  It performs gaussian integration over a
> triangle.  I am trying to figure out how to slice the arrays so as to
> push it down into the C level.  Does anyone have any ideas?  Thanks,
> Rob.
> 
> ps. it looks to be intractible to me.  Maybe I need to look at writing a
> C extension.  I've never done that before.
> 
> ##**********************************************************************
> *****
> ##Prototype:  void  ComputeGaussQuadPoint(int QuadPoint, int
> *a.TrngleNode,
> ##                                            double *SrcPointCol)
> ##Description: To compute the coordinates of 7-point Gauss nodes of
> ##             a triangular patch.
> ##Input value:
> ##    int QuadPoint  --- node index, it can be from 0 to 6.
> ##    int *a.TrngleNode  ---  the three nodes of a tringular patch.
> ##    double *SrcPointCol --- where to store the results
> ##Return value: none
> ##Global value used: a.NodeCord, a.Qpnt
> ##Global value modified:    none
> ##Subroutines called:    none
> ##Note: Not very sure.
> ************************************************************************
> ****
> def ComputeGaussQuadPoint(QuadPoint,TrngleNode,a):
> 
>     SrcPointCol=zeros((3),Float)
> 
>     SrcPointCol[0] =   a.Qpnt[QuadPoint,0]*a.NodeCord[TrngleNode[0],0]\
>                      + a.Qpnt[QuadPoint,1]*a.NodeCord[TrngleNode[1],0]\
>                      + a.Qpnt[QuadPoint,2]*a.NodeCord[TrngleNode[2],0]
> 
>     SrcPointCol[1] =   a.Qpnt[QuadPoint,0]*a.NodeCord[TrngleNode[0],1]\
>                      + a.Qpnt[QuadPoint,1]*a.NodeCord[TrngleNode[1],1]\
>                      + a.Qpnt[QuadPoint,2]*a.NodeCord[TrngleNode[2],1]
> 
>     SrcPointCol[2] =   a.Qpnt[QuadPoint,0]*a.NodeCord[TrngleNode[0],2]\
>                      + a.Qpnt[QuadPoint,1]*a.NodeCord[TrngleNode[1],2]\
>                      + a.Qpnt[QuadPoint,2]*a.NodeCord[TrngleNode[2],2]
> 
> 
>     return SrcPointCol
> 
> 
> --
> The Numeric Python EM Project
> 
> www.members.home.net/europax
> 
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> http://lists.sourceforge.net/lists/listinfo/numpy-discussion
> 
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> http://lists.sourceforge.net/lists/listinfo/numpy-discussion

-- 
The Numeric Python EM Project

www.members.home.net/europax



From europax at home.com  Sun Aug 26 18:18:46 2001
From: europax at home.com (Rob)
Date: Sun, 26 Aug 2001 15:18:46 -0700
Subject: [Numpy-discussion] Can this function by Numpy-ized?
References: <000001c12e55$144a5c40$0301a8c0@plstn1.sfba.home.com>
Message-ID: <3B8975C6.5EE83A96@home.com>

The good news is that your routine reduces execution time by 30%.  The
bad news is that the wrong numbers are coming out of it.  I'm trying
some otner permutations of "take" and "add.reduce" in the function to
see if I can get it.  One method that worked was splitting up "c" into
c1,c2,c3, such that:

c1=Numeric.take(a.NodeCord[:,0],TrngleNode)
etc

and then using  Numeric.add.reduce(s*c1,1)
etc

This gives the right results, but is slower than plain Python.  I'll
keep at it.  Thanks again.


"Paul F. Dubois" wrote:
> 
> def cgqp(QuadPoint, TrngleNode, a):
>     s = a.Qpnt[Quadpoint,:]
>     c = Numeric.take(a.NodeCord, TrngleNode)
>     return Numeric.add.reduce(s * c, axis=1)
> 
> This may or may not be right. The data structures I would have to set up
> to test it are too much for Sunday morning.
> 
> -----Original Message-----
> From: numpy-discussion-admin at lists.sourceforge.net
> [mailto:numpy-discussion-admin at lists.sourceforge.net] On Behalf Of Rob
> Sent: Sunday, August 26, 2001 8:39 AM
> To: numpy-discussion at lists.sourceforge.net.
> Subject: [Numpy-discussion] Can this function by Numpy-ized?
> 
> I finally got my FEM EM code working.  I profiled it and this function
> uses up a big hunk of time.  It performs gaussian integration over a
> triangle.  I am trying to figure out how to slice the arrays so as to
> push it down into the C level.  Does anyone have any ideas?  Thanks,
> Rob.
> 
> ps. it looks to be intractible to me.  Maybe I need to look at writing a
> C extension.  I've never done that before.
> 
> ##**********************************************************************
> *****
> ##Prototype:  void  ComputeGaussQuadPoint(int QuadPoint, int
> *a.TrngleNode,
> ##                                            double *SrcPointCol)
> ##Description: To compute the coordinates of 7-point Gauss nodes of
> ##             a triangular patch.
> ##Input value:
> ##    int QuadPoint  --- node index, it can be from 0 to 6.
> ##    int *a.TrngleNode  ---  the three nodes of a tringular patch.
> ##    double *SrcPointCol --- where to store the results
> ##Return value: none
> ##Global value used: a.NodeCord, a.Qpnt
> ##Global value modified:    none
> ##Subroutines called:    none
> ##Note: Not very sure.
> ************************************************************************
> ****
> def ComputeGaussQuadPoint(QuadPoint,TrngleNode,a):
> 
>     SrcPointCol=zeros((3),Float)
> 
>     SrcPointCol[0] =   a.Qpnt[QuadPoint,0]*a.NodeCord[TrngleNode[0],0]\
>                      + a.Qpnt[QuadPoint,1]*a.NodeCord[TrngleNode[1],0]\
>                      + a.Qpnt[QuadPoint,2]*a.NodeCord[TrngleNode[2],0]
> 
>     SrcPointCol[1] =   a.Qpnt[QuadPoint,0]*a.NodeCord[TrngleNode[0],1]\
>                      + a.Qpnt[QuadPoint,1]*a.NodeCord[TrngleNode[1],1]\
>                      + a.Qpnt[QuadPoint,2]*a.NodeCord[TrngleNode[2],1]
> 
>     SrcPointCol[2] =   a.Qpnt[QuadPoint,0]*a.NodeCord[TrngleNode[0],2]\
>                      + a.Qpnt[QuadPoint,1]*a.NodeCord[TrngleNode[1],2]\
>                      + a.Qpnt[QuadPoint,2]*a.NodeCord[TrngleNode[2],2]
> 
> 
>     return SrcPointCol
> 
> 
> --
> The Numeric Python EM Project
> 
> www.members.home.net/europax
> 
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> http://lists.sourceforge.net/lists/listinfo/numpy-discussion

-- 
The Numeric Python EM Project

www.members.home.net/europax



From europax at home.com  Mon Aug 27 10:49:22 2001
From: europax at home.com (Rob)
Date: Mon, 27 Aug 2001 07:49:22 -0700
Subject: [Numpy-discussion] Gaussian Quadrature routine Numpy-ization :)
Message-ID: <3B8A5DF2.F6AC7487@home.com>

I finally got it to work, but the Numpy-ized version runs slower than
the plain Python one.  I think that I can transpose the NodeCord matrix
once in the program and feed that in, rather than the scratch matrix
that is generated here.  Evidently transpose is not very fast even for
smal matrices. Here is my test program:


from Numeric import *



Qpnt=array(([20,21,22],[23,24,25],[26,27,28]))
NodeCord=array(([1,2,3],[4,5,6],[7,8,9]))
TrngleNode=array((1,2,0))


#the original routine
def ComputeGaussQuadPoint(QuadPoint,TrngleNode,Qpnt,NodeCord):
    
    SrcPointCol=zeros((3))
    
    
    SrcPointCol[0] =   Qpnt[QuadPoint,0]*NodeCord[TrngleNode[0],0]\
                     + Qpnt[QuadPoint,1]*NodeCord[TrngleNode[1],0]\
                     + Qpnt[QuadPoint,2]*NodeCord[TrngleNode[2],0]

    SrcPointCol[1] =   Qpnt[QuadPoint,0]*NodeCord[TrngleNode[0],1]\
                     + Qpnt[QuadPoint,1]*NodeCord[TrngleNode[1],1]\
                     + Qpnt[QuadPoint,2]*NodeCord[TrngleNode[2],1]

    SrcPointCol[2] =   Qpnt[QuadPoint,0]*NodeCord[TrngleNode[0],2]\
                     + Qpnt[QuadPoint,1]*NodeCord[TrngleNode[1],2]\
                     + Qpnt[QuadPoint,2]*NodeCord[TrngleNode[2],2]

    return SrcPointCol


#the yet-to-be-faster routine

def Quad(QuadPoint, TrngleNode, Qpnt,NodeCord):
    s = Qpnt[QuadPoint,:]
    c= take(NodeCord, TrngleNode)
    SrcPointCol= add.reduce(s *
transpose(c),1)                               

    return SrcPointCol
  
QuadPoint=1

print "The Correct:"
print ComputeGaussQuadPoint(QuadPoint,TrngleNode,Qpnt,NodeCord)

print "The New"
print  Quad(QuadPoint,TrngleNode,Qpnt,NodeCord)





-- 
The Numeric Python EM Project

www.members.home.net/europax



From tim.hochberg at ieee.org  Mon Aug 27 12:10:01 2001
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Mon, 27 Aug 2001 09:10:01 -0700
Subject: [Numpy-discussion] Gaussian Quadrature routine Numpy-ization :)
References: <3B8A5DF2.F6AC7487@home.com>
Message-ID: <010b01c12f12$b9f6ca80$87740918@cx781526b>

Hi Rob,

It looks like you are trying to use the function below to integrate over a
single triangle. I'm almost certain that you will _not_ be able to get this
to run fast; there's just too much overhead per triangle from all the
function calls and what not. Instead you need to structure things so that
you do more work per call. One way is to pass a list of triangles and get
back a list of answers. This way you spread your area over many triangles.
Another possibility, assuming that you need to evaluate the integral for all
seven possible values of QPoint each time, is to compute the answer for all
values of QPoint at once so that you reduce the overhead per triangle by
seven. For example (minimally tested):

Qpnt=reshape(arange(7*3), (7,3))
# Other code from other messages elided....

def Quads(TrngleNode, Qpnt, NodeCord):
     c = take(NodeCord, TrngleNode)
     SrcPointCols= add.reduce(Qpnt[...,NewAxis] *  c[NewAxis,],1)
     return SrcPointCols

# Initialize stuff to minimize effects of timing....
from time import clock
q1 = [None]*7
q2 = [None]*7
rng = range(7)

# Time the three methods....

t0 = clock()
for i in rng:
    q1[i] = ComputeGaussQuadPoint(i,TrngleNode,Qpnt,NodeCord)
t1 = clock()
for i in rng:
    q2[i] = Quad(i,TrngleNode,Qpnt,NodeCord)
t2 = clock()
q3 = Quads(TrngleNode, Qpnt, NodeCord)
t3 = clock()

# And the results...

print "Times (us):", (t1-t0)*1e6, (t2-t1)*1e6, (t3-t2)*1e6
for i in range(7):
    print "The Answers:", q1[i], q2[i], q3[i]


If this code is still a bottleneck, you could do both (compute the values
for all nodes and all values of QPoint at once, but this may be enough to
move the bottleneck elsewhere; by my timing it's over 7 times as fast.

-tim

[snip]

> Evidently transpose is not very fast even for
> smal matrices.

Actually, transpose should be fast, and should look progressively faster for
larger matrices. Typically all that happens is that the strides are changed.

----- Original Message -----
From: "Rob" <europax at home.com>
To: <numpy-discussion at lists.sourceforge.net>
Sent: Monday, August 27, 2001 7:49 AM
Subject: [Numpy-discussion] Gaussian Quadrature routine Numpy-ization :)


> I finally got it to work, but the Numpy-ized version runs slower than
> the plain Python one.  I think that I can transpose the NodeCord matrix
> once in the program and feed that in, rather than the scratch matrix
> that is generated here.  Evidently transpose is not very fast even for
> smal matrices. Here is my test program:
>
>
> from Numeric import *
>
>
>
> Qpnt=array(([20,21,22],[23,24,25],[26,27,28]))
> NodeCord=array(([1,2,3],[4,5,6],[7,8,9]))
> TrngleNode=array((1,2,0))
>
>
> #the original routine
> def ComputeGaussQuadPoint(QuadPoint,TrngleNode,Qpnt,NodeCord):
>
>     SrcPointCol=zeros((3))
>
>
>     SrcPointCol[0] =   Qpnt[QuadPoint,0]*NodeCord[TrngleNode[0],0]\
>                      + Qpnt[QuadPoint,1]*NodeCord[TrngleNode[1],0]\
>                      + Qpnt[QuadPoint,2]*NodeCord[TrngleNode[2],0]
>
>     SrcPointCol[1] =   Qpnt[QuadPoint,0]*NodeCord[TrngleNode[0],1]\
>                      + Qpnt[QuadPoint,1]*NodeCord[TrngleNode[1],1]\
>                      + Qpnt[QuadPoint,2]*NodeCord[TrngleNode[2],1]
>
>     SrcPointCol[2] =   Qpnt[QuadPoint,0]*NodeCord[TrngleNode[0],2]\
>                      + Qpnt[QuadPoint,1]*NodeCord[TrngleNode[1],2]\
>                      + Qpnt[QuadPoint,2]*NodeCord[TrngleNode[2],2]
>
>     return SrcPointCol
>
>
> #the yet-to-be-faster routine
>
> def Quad(QuadPoint, TrngleNode, Qpnt,NodeCord):
>     s = Qpnt[QuadPoint,:]
>     c= take(NodeCord, TrngleNode)
>     SrcPointCol= add.reduce(s *
> transpose(c),1)
>
>     return SrcPointCol
>
> QuadPoint=1
>
> print "The Correct:"
> print ComputeGaussQuadPoint(QuadPoint,TrngleNode,Qpnt,NodeCord)
>
> print "The New"
> print  Quad(QuadPoint,TrngleNode,Qpnt,NodeCord)
>
>
>
>
>
> --
> The Numeric Python EM Project
>
> www.members.home.net/europax
>
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> http://lists.sourceforge.net/lists/listinfo/numpy-discussion




From chrishbarker at home.net  Mon Aug 27 16:17:41 2001
From: chrishbarker at home.net (Chris Barker)
Date: Mon, 27 Aug 2001 13:17:41 -0700
Subject: [Numpy-discussion] manual update
References: <000201c12ccc$fc388e80$0301a8c0@plstn1.sfba.home.com>
Message-ID: <3B8AAAE5.81E90762@home.net>

Paul F. Dubois wrote:

>I don't own Matlab and so maintaining a package that emulates something
>I don't have makes me nervous. I wish one of the other developers with
>more knowledge in this area would pitch in.


I'm not much of a NumPy developer, but I do own MATLAB, and was once a
heavy user of it (now I use Python a lot more, or course). I'm not sure
what I can contibute, but I'd be glad to test and comment.

-Chris




-- 
Christopher Barker,
Ph.D.                                                           
ChrisHBarker at home.net                 ---           ---           ---
http://members.home.net/barkerlohmann ---@@       -----@@       -----@@
                                   ------@@@     ------@@@     ------@@@
Oil Spill Modeling                ------   @    ------   @   ------   @
Water Resources Engineering       -------      ---------     --------    
Coastal and Fluvial Hydrodynamics --------------------------------------
------------------------------------------------------------------------



From europax at home.com  Mon Aug 27 20:40:30 2001
From: europax at home.com (Rob)
Date: Mon, 27 Aug 2001 17:40:30 -0700
Subject: [Numpy-discussion] Gaussian Quadrature routine Numpy-ization :)
References: <3B8A5DF2.F6AC7487@home.com> <010b01c12f12$b9f6ca80$87740918@cx781526b>
Message-ID: <3B8AE87E.E416C138@home.com>

Hi Tim,  Yes this is part of a FEM-MOM hybrid EM simulator.  It was
originally written in C, so I've had to deal with the C way of doing
things.  I'd like to consolidate many of these types of operations so
that the program becomes streamlined and easier to understand- "The
Python Way" TM.  :)

I will try your method to see how it works.  By contrast, I have a FDTD
simulator which is a true speed demon with Numpy.  But it was organized
in a way that I would easily use slicing to time march the 6 scalar
Maxwell's equations.   

In any case I'm having fun.  I do this at home.  At work I have Ansoft
Serenade and Agilent ADS at my disposal.

Rob.

Tim Hochberg wrote:
> 
> Hi Rob,
> 
> It looks like you are trying to use the function below to integrate over a
> single triangle. I'm almost certain that you will _not_ be able to get this
> to run fast; there's just too much overhead per triangle from all the
> function calls and what not. Instead you need to structure things so that
> you do more work per call. One way is to pass a list of triangles and get
> back a list of answers. This way you spread your area over many triangles.
> Another possibility, assuming that you need to evaluate the integral for all
> seven possible values of QPoint each time, is to compute the answer for all
> values of QPoint at once so that you reduce the overhead per triangle by
> seven. For example (minimally tested):
> 
> Qpnt=reshape(arange(7*3), (7,3))
> # Other code from other messages elided....
> 
> def Quads(TrngleNode, Qpnt, NodeCord):
>      c = take(NodeCord, TrngleNode)
>      SrcPointCols= add.reduce(Qpnt[...,NewAxis] *  c[NewAxis,],1)
>      return SrcPointCols
> 
> # Initialize stuff to minimize effects of timing....
> from time import clock
> q1 = [None]*7
> q2 = [None]*7
> rng = range(7)
> 
> # Time the three methods....
> 
> t0 = clock()
> for i in rng:
>     q1[i] = ComputeGaussQuadPoint(i,TrngleNode,Qpnt,NodeCord)
> t1 = clock()
> for i in rng:
>     q2[i] = Quad(i,TrngleNode,Qpnt,NodeCord)
> t2 = clock()
> q3 = Quads(TrngleNode, Qpnt, NodeCord)
> t3 = clock()
> 
> # And the results...
> 
> print "Times (us):", (t1-t0)*1e6, (t2-t1)*1e6, (t3-t2)*1e6
> for i in range(7):
>     print "The Answers:", q1[i], q2[i], q3[i]
> 
> If this code is still a bottleneck, you could do both (compute the values
> for all nodes and all values of QPoint at once, but this may be enough to
> move the bottleneck elsewhere; by my timing it's over 7 times as fast.
> 
> -tim
> 
> [snip]
> 
> > Evidently transpose is not very fast even for
> > smal matrices.
> 
> Actually, transpose should be fast, and should look progressively faster for
> larger matrices. Typically all that happens is that the strides are changed.
> 
> ----- Original Message -----
> From: "Rob" <europax at home.com>
> To: <numpy-discussion at lists.sourceforge.net>
> Sent: Monday, August 27, 2001 7:49 AM
> Subject: [Numpy-discussion] Gaussian Quadrature routine Numpy-ization :)
> 
> > I finally got it to work, but the Numpy-ized version runs slower than
> > the plain Python one.  I think that I can transpose the NodeCord matrix
> > once in the program and feed that in, rather than the scratch matrix
> > that is generated here.  Evidently transpose is not very fast even for
> > smal matrices. Here is my test program:
> >
> >
> > from Numeric import *
> >
> >
> >
> > Qpnt=array(([20,21,22],[23,24,25],[26,27,28]))
> > NodeCord=array(([1,2,3],[4,5,6],[7,8,9]))
> > TrngleNode=array((1,2,0))
> >
> >
> > #the original routine
> > def ComputeGaussQuadPoint(QuadPoint,TrngleNode,Qpnt,NodeCord):
> >
> >     SrcPointCol=zeros((3))
> >
> >
> >     SrcPointCol[0] =   Qpnt[QuadPoint,0]*NodeCord[TrngleNode[0],0]\
> >                      + Qpnt[QuadPoint,1]*NodeCord[TrngleNode[1],0]\
> >                      + Qpnt[QuadPoint,2]*NodeCord[TrngleNode[2],0]
> >
> >     SrcPointCol[1] =   Qpnt[QuadPoint,0]*NodeCord[TrngleNode[0],1]\
> >                      + Qpnt[QuadPoint,1]*NodeCord[TrngleNode[1],1]\
> >                      + Qpnt[QuadPoint,2]*NodeCord[TrngleNode[2],1]
> >
> >     SrcPointCol[2] =   Qpnt[QuadPoint,0]*NodeCord[TrngleNode[0],2]\
> >                      + Qpnt[QuadPoint,1]*NodeCord[TrngleNode[1],2]\
> >                      + Qpnt[QuadPoint,2]*NodeCord[TrngleNode[2],2]
> >
> >     return SrcPointCol
> >
> >
> > #the yet-to-be-faster routine
> >
> > def Quad(QuadPoint, TrngleNode, Qpnt,NodeCord):
> >     s = Qpnt[QuadPoint,:]
> >     c= take(NodeCord, TrngleNode)
> >     SrcPointCol= add.reduce(s *
> > transpose(c),1)
> >
> >     return SrcPointCol
> >
> > QuadPoint=1
> >
> > print "The Correct:"
> > print ComputeGaussQuadPoint(QuadPoint,TrngleNode,Qpnt,NodeCord)
> >
> > print "The New"
> > print  Quad(QuadPoint,TrngleNode,Qpnt,NodeCord)
> >
> >
> >
> >
> >
> > --
> > The Numeric Python EM Project
> >
> > www.members.home.net/europax
> >
> > _______________________________________________
> > Numpy-discussion mailing list
> > Numpy-discussion at lists.sourceforge.net
> > http://lists.sourceforge.net/lists/listinfo/numpy-discussion
> 
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> http://lists.sourceforge.net/lists/listinfo/numpy-discussion

-- 
The Numeric Python EM Project

www.members.home.net/europax



From europax at home.com  Tue Aug 28 00:34:56 2001
From: europax at home.com (Rob)
Date: Mon, 27 Aug 2001 21:34:56 -0700
Subject: [Numpy-discussion] Gaussian Quadrature routine Numpy-ization :)
References: <3B8A5DF2.F6AC7487@home.com> <010b01c12f12$b9f6ca80$87740918@cx781526b>
Message-ID: <3B8B1F70.B2807C42@home.com>

Tim, I used your routine, computing all of the triangles at once and
then feeding the resulting array back into the needy functions.  I
shaved 30% off the execution time!!  There is another routine which
computes correction terms which can use the "take()" method.  I'll work
on that tomorrow.  Thanks alot!

Rob.

ps. I'll have to study the Numpy manual to figure out how your routine
works.


Tim Hochberg wrote:
> 
> Hi Rob,
> 
> It looks like you are trying to use the function below to integrate over a
> single triangle. I'm almost certain that you will _not_ be able to get this
> to run fast; there's just too much overhead per triangle from all the
> function calls and what not. Instead you need to structure things so that
> you do more work per call. One way is to pass a list of triangles and get
> back a list of answers. This way you spread your area over many triangles.
> Another possibility, assuming that you need to evaluate the integral for all
> seven possible values of QPoint each time, is to compute the answer for all
> values of QPoint at once so that you reduce the overhead per triangle by
> seven. For example (minimally tested):
> 
> Qpnt=reshape(arange(7*3), (7,3))
> # Other code from other messages elided....
> 
> def Quads(TrngleNode, Qpnt, NodeCord):
>      c = take(NodeCord, TrngleNode)
>      SrcPointCols= add.reduce(Qpnt[...,NewAxis] *  c[NewAxis,],1)
>      return SrcPointCols
> 
> # Initialize stuff to minimize effects of timing....
> from time import clock
> q1 = [None]*7
> q2 = [None]*7
> rng = range(7)
> 
> # Time the three methods....
> 
> t0 = clock()
> for i in rng:
>     q1[i] = ComputeGaussQuadPoint(i,TrngleNode,Qpnt,NodeCord)
> t1 = clock()
> for i in rng:
>     q2[i] = Quad(i,TrngleNode,Qpnt,NodeCord)
> t2 = clock()
> q3 = Quads(TrngleNode, Qpnt, NodeCord)
> t3 = clock()
> 
> # And the results...
> 
> print "Times (us):", (t1-t0)*1e6, (t2-t1)*1e6, (t3-t2)*1e6
> for i in range(7):
>     print "The Answers:", q1[i], q2[i], q3[i]
> 
> If this code is still a bottleneck, you could do both (compute the values
> for all nodes and all values of QPoint at once, but this may be enough to
> move the bottleneck elsewhere; by my timing it's over 7 times as fast.
> 
> -tim
> 
> [snip]
> 
> > Evidently transpose is not very fast even for
> > smal matrices.
> 
> Actually, transpose should be fast, and should look progressively faster for
> larger matrices. Typically all that happens is that the strides are changed.
> 
> ----- Original Message -----
> From: "Rob" <europax at home.com>
> To: <numpy-discussion at lists.sourceforge.net>
> Sent: Monday, August 27, 2001 7:49 AM
> Subject: [Numpy-discussion] Gaussian Quadrature routine Numpy-ization :)
> 
> > I finally got it to work, but the Numpy-ized version runs slower than
> > the plain Python one.  I think that I can transpose the NodeCord matrix
> > once in the program and feed that in, rather than the scratch matrix
> > that is generated here.  Evidently transpose is not very fast even for
> > smal matrices. Here is my test program:
> >
> >
> > from Numeric import *
> >
> >
> >
> > Qpnt=array(([20,21,22],[23,24,25],[26,27,28]))
> > NodeCord=array(([1,2,3],[4,5,6],[7,8,9]))
> > TrngleNode=array((1,2,0))
> >
> >
> > #the original routine
> > def ComputeGaussQuadPoint(QuadPoint,TrngleNode,Qpnt,NodeCord):
> >
> >     SrcPointCol=zeros((3))
> >
> >
> >     SrcPointCol[0] =   Qpnt[QuadPoint,0]*NodeCord[TrngleNode[0],0]\
> >                      + Qpnt[QuadPoint,1]*NodeCord[TrngleNode[1],0]\
> >                      + Qpnt[QuadPoint,2]*NodeCord[TrngleNode[2],0]
> >
> >     SrcPointCol[1] =   Qpnt[QuadPoint,0]*NodeCord[TrngleNode[0],1]\
> >                      + Qpnt[QuadPoint,1]*NodeCord[TrngleNode[1],1]\
> >                      + Qpnt[QuadPoint,2]*NodeCord[TrngleNode[2],1]
> >
> >     SrcPointCol[2] =   Qpnt[QuadPoint,0]*NodeCord[TrngleNode[0],2]\
> >                      + Qpnt[QuadPoint,1]*NodeCord[TrngleNode[1],2]\
> >                      + Qpnt[QuadPoint,2]*NodeCord[TrngleNode[2],2]
> >
> >     return SrcPointCol
> >
> >
> > #the yet-to-be-faster routine
> >
> > def Quad(QuadPoint, TrngleNode, Qpnt,NodeCord):
> >     s = Qpnt[QuadPoint,:]
> >     c= take(NodeCord, TrngleNode)
> >     SrcPointCol= add.reduce(s *
> > transpose(c),1)
> >
> >     return SrcPointCol
> >
> > QuadPoint=1
> >
> > print "The Correct:"
> > print ComputeGaussQuadPoint(QuadPoint,TrngleNode,Qpnt,NodeCord)
> >
> > print "The New"
> > print  Quad(QuadPoint,TrngleNode,Qpnt,NodeCord)
> >
> >
> >
> >
> >
> > --
> > The Numeric Python EM Project
> >
> > www.members.home.net/europax
> >
> > _______________________________________________
> > Numpy-discussion mailing list
> > Numpy-discussion at lists.sourceforge.net
> > http://lists.sourceforge.net/lists/listinfo/numpy-discussion

-- 
The Numeric Python EM Project

www.members.home.net/europax



From a.culhane at ucc.ie  Tue Aug 28 12:22:05 2001
From: a.culhane at ucc.ie (Culhane, Aedin)
Date: Tue, 28 Aug 2001 17:22:05 +0100
Subject: [Numpy-discussion] Newbie Installation problems
Message-ID: <F64493091FAC4D4DAC46261FEC1967994796A1@xch4.ucc.ie>

Dear Numpy Discussion List Users,
I would really appreciate if someone could help me install numerical python
on Win98. I have downloaded Numerical Python twice now (older version and
20.1.0) but I am unable to get python to import numeric. I am running
python2.1.

I need it to be able to calculate basis statistics (mean, std dev, etc).
Will numerical python do this easily or should I write my own scripts?
Thanks a million for your help,
Aedin Culhane




From cgw at alum.mit.edu  Tue Aug 28 12:39:35 2001
From: cgw at alum.mit.edu (Charles G Waldman)
Date: Tue, 28 Aug 2001 11:39:35 -0500
Subject: [Numpy-discussion] Newbie Installation problems
In-Reply-To: <F64493091FAC4D4DAC46261FEC1967994796A1@xch4.ucc.ie>
References: <F64493091FAC4D4DAC46261FEC1967994796A1@xch4.ucc.ie>
Message-ID: <15243.51527.186135.887241@nyx.dyndns.org>

Culhane, Aedin writes:
 > Dear Numpy Discussion List Users,
 > I would really appreciate if someone could help me install numerical python
 > on Win98. I have downloaded Numerical Python twice now (older version and
 > 20.1.0) but I am unable to get python to import numeric. I am running
 > python2.1.

I'm sure a lot of folks on the list would be glad to help, but without
a little more detail it's hard to say much.  What errors are you
seeing?  How did you install Python?  What directory is it installed
into?  How did you try to install Numpy?

 > I need it to be able to calculate basis statistics (mean, std dev, etc).
 > Will numerical python do this easily or should I write my own scripts?
 > Thanks a million for your help,
 > Aedin Culhane

I think there are some statistics modules floating around on the Net,
but mean and standard deviation are one-liners in NumPy:


from Numeric import *

def mean(a): 
    return float(sum(a))/len(a)

def std_dev(a): 
    return sqrt(mean((a-mean(a))**2))





From chrishbarker at home.net  Tue Aug 28 13:40:10 2001
From: chrishbarker at home.net (Chris Barker)
Date: Tue, 28 Aug 2001 10:40:10 -0700
Subject: [Numpy-discussion] Newbie Installation problems
References: <F64493091FAC4D4DAC46261FEC1967994796A1@xch4.ucc.ie> <15243.51527.186135.887241@nyx.dyndns.org>
Message-ID: <3B8BD77A.1942DD31@home.net>

Charles G Waldman wrote:

> I think there are some statistics modules floating around on the Net,
> but mean and standard deviation are one-liners in NumPy:
> 
> from Numeric import *
> 
> def mean(a):
>     return float(sum(a))/len(a)
> 
> def std_dev(a):
>     return sqrt(mean((a-mean(a))**2))

or, for the unbiased estimate:

def std_dev(a):
    return sqrt(sum((a-mean(a))**2)/(len(a)-1))

You can also find these in Mlab.py

-Chris

-- 
Christopher Barker,
Ph.D.                                                           
ChrisHBarker at home.net                 ---           ---           ---
http://members.home.net/barkerlohmann ---@@       -----@@       -----@@
                                   ------@@@     ------@@@     ------@@@
Oil Spill Modeling                ------   @    ------   @   ------   @
Water Resources Engineering       -------      ---------     --------    
Coastal and Fluvial Hydrodynamics --------------------------------------
------------------------------------------------------------------------



From chrishbarker at home.net  Tue Aug 28 14:51:54 2001
From: chrishbarker at home.net (Chris Barker)
Date: Tue, 28 Aug 2001 11:51:54 -0700
Subject: [Numpy-discussion] MLab.py status: do you use it?
References: <000701c12f3b$c0605840$0301a8c0@plstn1.sfba.home.com>
Message-ID: <3B8BE84A.6CF21320@home.net>

Hi all,

Paul recently requested help with testing and debugging MLab.py, and I
offered to help out. He then sent me this note:

"Paul F. Dubois" wrote:
> MLab.py started out as a package that was supposed to give numpy a
> Matlab look and feel. So useful questions are: are there really useful
> things that should be added?

I can't answer these questions myself. Interestingly enough, despite the
fact that I came to NumPy having used MATLAB heavily for five years (and
one dissertation), I have made very little use of MLab.py. I'm wondering
if anyone is, indeed, using it.

As for trying to give numpy a MATLAB look and feel, I question the
usefulness of that. NumPy looks and feels a little different than
MATLAB, and, frankly, I like NumPy's feel better for the most part (now
that rich comparisons have been added, anyway). The one thing that
MATLAB has that NumPy doesn't, that I really miss, is list indexing.
Having to use put and take is so much less elegant! I'm sure this isn't
the least bit trivial to add, however. (is it even possible, given
Python's idea of slicing?)

Anyway, what I do think MATLAB does provide, and MLab.py should, is a
whole bunch of utility functions for various common manipulations.

> Do we really have what we have correct?
 
> Most of all, these is no test routine for it. If you could make one
> following the model of the test.py in directory Test, it would be great.
> The idea is to have something that does not require the runner of the
> test to know anything. It just runs silently unless something is wrong.

I certainly could write up a test routine. I will start work on that. In
the meantime, before I put too much effort into it, I'd really like some
feedback on what people want MLab.py to be, if they see using it at all.

As I said above, I'm not sure trying to give NumPy a MATLAB feel is a
useful goal, so what is left is having all those handy functions.
Perhaps we could shift away from MLab.py, and turn it into Utilities.py,
which could use MATLAB (and other user experience) as guidance as to
what utilities to include.

I'd really like folks' feedback on this before I put too much work into
something no one uses, or will use.

-Chris



-- 
Christopher Barker,
Ph.D.                                                           
ChrisHBarker at home.net                 ---           ---           ---
http://members.home.net/barkerlohmann ---@@       -----@@       -----@@
                                   ------@@@     ------@@@     ------@@@
Oil Spill Modeling                ------   @    ------   @   ------   @
Water Resources Engineering       -------      ---------     --------    
Coastal and Fluvial Hydrodynamics --------------------------------------
------------------------------------------------------------------------



From paul at pfdubois.com  Tue Aug 28 15:41:07 2001
From: paul at pfdubois.com (Paul F. Dubois)
Date: Tue, 28 Aug 2001 12:41:07 -0700
Subject: [Numpy-discussion] Newbie Installation problems
In-Reply-To: <15243.51527.186135.887241@nyx.dyndns.org>
Message-ID: <000001c12ff9$7b503220$0301a8c0@plstn1.sfba.home.com>

Install Python first. From the Numerical Python download site, get the
.exe file (not the zip) and run it. It is a conventional Setup-like
program. Get .exe files for the optional packages if desired and run
time too.

Functions mean and std are available in module MLab.py (note the capital
L).

Python 2.2a2 (#22, Aug 22 2001, 01:24:03) [MSC 32 bit (Intel)] on win32
Type "copyright", "credits" or "license" for more information.
IDLE 0.8 -- press F1 for help
>>> import MLab
>>> d = [1,2,4,9,12,24]
>>> MLab.mean(d)
8.6666666666666661
>>> MLab.std(d)
8.6178110136313997
>>>
-----Original Message-----
From: numpy-discussion-admin at lists.sourceforge.net
[mailto:numpy-discussion-admin at lists.sourceforge.net] On Behalf Of
Charles G Waldman
Sent: Tuesday, August 28, 2001 9:40 AM
To: Culhane, Aedin
Cc: numpy-discussion at lists.sourceforge.net
Subject: [Numpy-discussion] Newbie Installation problems

Culhane, Aedin writes:
 > Dear Numpy Discussion List Users,
 > I would really appreciate if someone could help me install numerical
python
 > on Win98. I have downloaded Numerical Python twice now (older version
and
 > 20.1.0) but I am unable to get python to import numeric. I am running
 > python2.1.

I'm sure a lot of folks on the list would be glad to help, but without
a little more detail it's hard to say much.  What errors are you
seeing?  How did you install Python?  What directory is it installed
into?  How did you try to install Numpy?

 > I need it to be able to calculate basis statistics (mean, std dev,
etc).
 > Will numerical python do this easily or should I write my own
scripts?
 > Thanks a million for your help,
 > Aedin Culhane

I think there are some statistics modules floating around on the Net,
but mean and standard deviation are one-liners in NumPy:


from Numeric import *

def mean(a): 
    return float(sum(a))/len(a)

def std_dev(a): 
    return sqrt(mean((a-mean(a))**2))



_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
http://lists.sourceforge.net/lists/listinfo/numpy-discussion




From nodwell at physics.ubc.ca  Tue Aug 28 19:24:22 2001
From: nodwell at physics.ubc.ca (Eric Nodwell)
Date: Tue, 28 Aug 2001 16:24:22 -0700
Subject: [Numpy-discussion] one-offset arrays
Message-ID: <20010828162422.A560@holler.physics.ubc.ca>

Does anyone have a simple method of using one-offset arrays?
Specifically, can I define an array "a" so that a[1] refers to the
first element?

Without one-offset indexing, it seems to me that Python is minimally
useful for numerical computations. Many, perhaps the majority, of
numerical algorithms are one-indexed. Matlab for example is one-based
for this reason.  In fact it seems strange to me that a "high-level"
language like Python should use zero-offset lists.






From serge at rohan.sdsu.edu  Tue Aug 28 20:18:34 2001
From: serge at rohan.sdsu.edu (Serge Rey)
Date: Wed, 29 Aug 2001 10:18:34 +1000
Subject: [Numpy-discussion] Re: MLab.py status: do you use it?
In-Reply-To: <E15boFC-0000dU-00@usw-sf-list1.sourceforge.net>; from numpy-discussion-request@lists.sourceforge.net on Tue, Aug 28, 2001 at 12:09:50PM -0700
References: <E15boFC-0000dU-00@usw-sf-list1.sourceforge.net>
Message-ID: <20010829101834.A657@typhoon.sdsu.edu>

On Tue, Aug 28, 2001 at 12:09:50PM -0700, numpy-discussion-request at lists.sourceforge.net wrote:

-snip- 
> I certainly could write up a test routine. I will start work on that. In
> the meantime, before I put too much effort into it, I'd really like some
> feedback on what people want MLab.py to be, if they see using it at all.
> 
> As I said above, I'm not sure trying to give NumPy a MATLAB feel is a
> useful goal, so what is left is having all those handy functions.
> Perhaps we could shift away from MLab.py, and turn it into Utilities.py,
> which could use MATLAB (and other user experience) as guidance as to
> what utilities to include.
> 
> I'd really like folks' feedback on this before I put too much work into
> something no one uses, or will use.

i use MLab.py fairly often but see it more as a collection of useful
functions. personally, i agree with moving it into Utilities.py, as that
is in line with how i use it.

otoh, MLab.py does provide a nice entry point for folks coming from
Matlab into numpy. so, perhaps any eventual move to Utilities.py could
be accompanied with some documentation to remind users of those
linkages.

i'd offer to help with the testing, but my copy of matlab is back in the
states (i'm in australia on sabbatical) so i'd have to go from memory
(dangerous). let me know if i can help you.

cheers,
serge

-- 
Sergio J. Rey	http://typhoon.sdsu.edu/rey.html
"NT is secure.... as long as you don't remove the shrink wrap." - G. Myers



From oliphant.travis at ieee.org  Tue Aug 28 15:42:17 2001
From: oliphant.travis at ieee.org (Travis Oliphant)
Date: Tue, 28 Aug 2001 19:42:17 +0000
Subject: [Numpy-discussion] one-offset arrays
In-Reply-To: <20010828162422.A560@holler.physics.ubc.ca>
References: <20010828162422.A560@holler.physics.ubc.ca>
Message-ID: <01082819421703.09959@travis>

On Tue, 28 Aug 2001, Eric Nodwell wrote:
> Does anyone have a simple method of using one-offset arrays?
> Specifically, can I define an array "a" so that a[1] refers to the
> first element?
>

Inherit from the UserArray and redefine slicing to your hearts content.

> Without one-offset indexing, it seems to me that Python is minimally
> useful for numerical computations. Many, perhaps the majority, of
> numerical algorithms are one-indexed. Matlab for example is one-based
> for this reason.  In fact it seems strange to me that a "high-level"
> language like Python should use zero-offset lists.
>

Wow, that is a pretty condescending-sounding statement, though I'm sure you 
didn't mean it that way.  Python is indeed being used for quite serious 
numerical computations.  I have been using Python for quite some time for 
Numerical work and find it's zero-based indexing combined with the 
leave-last-one-out slicing notation to be much more convenient.  

It is different from MATLAB but that hardly makes it less useful as I suspect 
you'd agree after trying it out for awhile.

What is your application domain that so requires 1-based indexing.

Travis E. Oliphant, Ph.D.
Assistant Professor
Brigham Young University
Provo, UT 84602
(801) 378-3108
oliphant.travis at ieee.org



From oliphant.travis at ieee.org  Tue Aug 28 16:09:30 2001
From: oliphant.travis at ieee.org (Travis Oliphant)
Date: Tue, 28 Aug 2001 20:09:30 +0000
Subject: [Numpy-discussion] MLab.py status: do you use it?
In-Reply-To: <3B8BE84A.6CF21320@home.net>
References: <000701c12f3b$c0605840$0301a8c0@plstn1.sfba.home.com> <3B8BE84A.6CF21320@home.net>
Message-ID: <01082820093006.09959@travis>

On Tue, 28 Aug 2001, Chris Barker wrote:
> Hi all,
>
> Paul recently requested help with testing and debugging MLab.py, and I
> offered to help out. He then sent me this note:
>
> "Paul F. Dubois" wrote:
> > MLab.py started out as a package that was supposed to give numpy a
> > Matlab look and feel. So useful questions are: are there really useful
> > things that should be added?

I use many of the functions in MLab regularly.  I have no problem renaming it 
to Utility.py or even incorporating it back into some other module.

scipy uses MLab currently has well.

>
> I can't answer these questions myself. Interestingly enough, despite the
> fact that I came to NumPy having used MATLAB heavily for five years (and
> one dissertation), I have made very little use of MLab.py. I'm wondering
> if anyone is, indeed, using it.

Yes, we are...

>
> As for trying to give numpy a MATLAB look and feel, I question the
> usefulness of that. NumPy looks and feels a little different than
> MATLAB, and, frankly, I like NumPy's feel better for the most part (now
> that rich comparisons have been added, anyway). The one thing that
> MATLAB has that NumPy doesn't, that I really miss, is list indexing.
> Having to use put and take is so much less elegant! I'm sure this isn't
> the least bit trivial to add, however. (is it even possible, given
> Python's idea of slicing?)

I don't think look and feel is the right word --- but increasing the number 
of easily accesible utility routines is what the goal of SciPy is all about.

List indexing is difficult to add to the C-code due to the complexity of that 
particular subroutine.  Both Paul and I have looked at the code to try and 
add this feature but neither one of us had the time to make it work right.  I 
think it would be possible if we could ever agree on what list-indexing 
should do exactly for multidimensional arrays.    

Python's idea of slicing is actually quite general.  It takes 
a[obj1, obj2, obj3]  and essentially calls  the (C-equivalent) of __getitem__ 
(the mapping interface) with the tuple (obj1, obj2, obj3).  Any objects which 
look like (3:10:2)  are translated to a slice-object.  So, the objects could 
easily be indices into the array or a mask array of ones and zeros. 

I seem to remember that there were some "difficult-to-decide" cases but I 
can't recall them at the moment.

>
> Anyway, what I do think MATLAB does provide, and MLab.py should, is a
> whole bunch of utility functions for various common manipulations.
>
> > Do we really have what we have correct?

I'm pretty confident in it (that's not a life-staking claim...)
Most of my utility-development work is taking place in SciPy now.  Bugs will 
get fixed in MLab.py since Scipy uses it.

> >
> > Most of all, these is no test routine for it. If you could make one
> > following the model of the test.py in directory Test, it would be great.
> > The idea is to have something that does not require the runner of the
> > test to know anything. It just runs silently unless something is wrong.
>
> I certainly could write up a test routine. I will start work on that. In
> the meantime, before I put too much effort into it, I'd really like some
> feedback on what people want MLab.py to be, if they see using it at all.
>

It's a good idea.

> As I said above, I'm not sure trying to give NumPy a MATLAB feel is a
> useful goal, so what is left is having all those handy functions.
> Perhaps we could shift away from MLab.py, and turn it into Utilities.py,
> which could use MATLAB (and other user experience) as guidance as to
> what utilities to include.

I've got some fun suggestions for utility functions which I've just been 
storing away.  Some of them are put into SciPy.





From europax at home.com  Tue Aug 28 22:54:01 2001
From: europax at home.com (Rob)
Date: Tue, 28 Aug 2001 19:54:01 -0700
Subject: [Numpy-discussion] invert() with complex arrays
Message-ID: <3B8C5949.1F9C0531@home.com>

Is there any way to get invert() to work with complex arrays?

Thanks,  Rob.

-- 
The Numeric Python EM Project

www.members.home.net/europax



From milgram at cgpp.com  Tue Aug 28 23:28:34 2001
From: milgram at cgpp.com (J. Milgram)
Date: Tue, 28 Aug 2001 23:28:34 -0400
Subject: [Numpy-discussion] cephes-1.3 install problem
In-Reply-To: Your message of "Tue, 28 Aug 2001 20:09:30 -0000."
             <01082820093006.09959@travis> 
Message-ID: <200108290328.f7T3SYP00391@cgpp.com>


Hi,

this is surely a FAQ so I apologize in advance, but:

I'm trying to install cephes-1.3 and getting this:

cc -O2 -march=pentium -I/usr/local/include/python2.0 -I/usr/local/include/python2.0/Numeric    -c amos_wrappers.c -o amos_wrappers.o
In file included from cephes/mconf.h:162,
                 from amos_wrappers.h:11,
                 from amos_wrappers.c:8: 
cephes/protos.h:67: parse error before `sizeof'
cephes/protos.h:68: parse error before `sizeof'
cephes/protos.h:69: parse error before `sizeof'
make: *** [amos_wrappers.o] Error 1

Am installing with:

Python 2.0 (#14, Aug  2 2001, 10:06:36) 
[GCC egcs-2.91.66 19990314/Linux (egcs-1.1.2 release)] on linux2

and have already installed:

Numeric-20.1.0

grateful for any help ... I did search a bit and it seems to be a
known problem but I couldn't figure out the answer. Surely something
simple.

thanks

Judah

Judah Milgram        milgram at cgpp.com
P.O. Box 8376, Langley Park, MD 20787
(301) 422-4626 (-3047 fax)





From nodwell at physics.ubc.ca  Wed Aug 29 01:00:50 2001
From: nodwell at physics.ubc.ca (Eric Nodwell)
Date: Tue, 28 Aug 2001 22:00:50 -0700
Subject: [Numpy-discussion] one-offset arrays
Message-ID: <20010828220046.A4161@holler.physics.ubc.ca>

>> Does anyone have a simple method of using one-offset arrays?
>> Specifically, can I define an array "a" so that a[1] refers to the
>> first element?
>>

> Inherit from the UserArray and redefine slicing to your hearts content.

>> Without one-offset indexing, it seems to me that Python is minimally
>> useful for numerical computations. Many, perhaps the majority, of
>> numerical algorithms are one-indexed. Matlab for example is one-based
>> for this reason.  In fact it seems strange to me that a "high-level"
>> language like Python should use zero-offset lists.
>>

> Wow, that is a pretty condescending-sounding statement, though I'm sure you 
> didn't mean it that way.  Python is indeed being used for quite serious 
> numerical computations.  I have been using Python for quite some time for 
> Numerical work and find it's zero-based indexing combined with the 
> leave-last-one-out slicing notation to be much more convenient.  

Oops, what I really need is a wet-ware (i.e. brain) macro which
enforces the correct order of the pair (think,write,send)! The above
unconsidered comments arose from the sequence
(frustration,write,send,think,regret).  ;)

Obviously there is a lot of numerical work being done with python and
people are very happy with it. But for me I still think it would be
"minimally useful" without 1-indexed arrays. Here's why:

In my view, the most important reason to prefer 1-based indexing
versus 0-based indexing is compatibility.  For numerical work, some of
the languages which I use or have used are Matlab, Mathematica, Maple
and Fortran. These are all 1-indexed. (C is by nature 0-indexed
because it is so close to machine architecture, but with a little bit
of not-entirely-clean pointer manipulation, you can easily make
1-indexed arrays and matrices.) Obviously Python can't be compatible
with these languages in a strict sense, but like most people who do
some programming work, I've built up a library of my own commonly used
routines specific to my work; in general it's a trivial matter to
translate numerical routines from one language to the another if
translation is just a matter of substituting of one set of syntactical
symbols and function names for anther.  However it can be damn tricky
to convert 1-indexed code to 0-indexed code or visa versa without
introducing any errors- believe me!  (Yes it's possible to call nearly
any language from nearly any other language these days so in theory
you don't have to recode, but there are lots of reasons why often
recoding is the preferable route.)

The other reason for choosing 1-based indexing is to keep the code as
near to the standard notation as possible. This is one of the
advantages of using a high level language - it approximates the way
you think about things instead of the way the computer organizes
them. Of course, this can go either way depending on the quantity in
question: as a simple example a spatial vector (x,y,z) is
conventionally labelled 1,2,3 (1-indexed), but a relativistic
four-vector with time included (t,x,y,z) is conventionally labelled
0,1,2,3 (0-indexed). So ideally one would be able to choose the
indexing-type case-by-case. I'm sure that computer programmers will
argue vehemently that code which mixes both 0-indexed and 1-indexed
arrays is hard to understand and maintain, but for experts in a
particular field who are accustomed to certain ingrained notations, it
is the code which breaks the conventional notation which is most
error-prone. In my case, I'm dealing at the moment with crystal
structures with which are associated certain conventional sets of
vectors and tensors - all 1-indexed by convention. I find it a
delicate enough task to always get the correct vector or tensor
without having to remember that d[2] is actually d3. Defining d1,d2,d3
is not convenient because often the index itself needs to be
calculated.

I guess if I understood the reason for 0-indexed lists and tuples in
Python I would be happier. In normal, everyday usage, sets,
collections and lists are 1-indexed (the first item, the second item,
the third item, and so on). Python is otherwise such an elegant and
natural language. Why the ugly exception of making the user conform to
the underlying mechanism of an array being an address plus an offset?

All this is really neither here nor there, since this debate, at least
as far as Python is concerned, was probably settled 10 years ago and
I'm sure nobody wants to hear anything more about it at this point.
As you point out, I can define my own array type with inheritance. I
will also need my own range command and several other functions which
haven't occured to me yet. I was hoping that there would be a standard
module to implement this.


By the way, what is leave-last-one-out slicing? Is it
a[:-1]
or is it
a[0,...]
or is it something else?


Eric



From oliphant.travis at ieee.org  Tue Aug 28 19:30:54 2001
From: oliphant.travis at ieee.org (Travis Oliphant)
Date: Tue, 28 Aug 2001 23:30:54 +0000
Subject: [Numpy-discussion] one-offset arrays
In-Reply-To: <20010828220046.A4161@holler.physics.ubc.ca>
References: <20010828220046.A4161@holler.physics.ubc.ca>
Message-ID: <01082823305401.12637@travis>

> In my view, the most important reason to prefer 1-based indexing
> versus 0-based indexing is compatibility.  For numerical work, some of
> the languages which I use or have used are Matlab, Mathematica, Maple
> and Fortran. These are all 1-indexed. (C is by nature 0-indexed
> because it is so close to machine architecture, but with a little bit
> of not-entirely-clean pointer manipulation, you can easily make
> 1-indexed arrays and matrices.) Obviously Python can't be compatible
> with these languages in a strict sense, but like most people who do
> some programming work, I've built up a library of my own commonly used
> routines specific to my work; in general it's a trivial matter to
> translate numerical routines from one language to the another if
> translation is just a matter of substituting of one set of syntactical
> symbols and function names for anther.  However it can be damn tricky
> to convert 1-indexed code to 0-indexed code or visa versa without
> introducing any errors- believe me!  (Yes it's possible to call nearly
> any language from nearly any other language these days so in theory
> you don't have to recode, but there are lots of reasons why often
> recoding is the preferable route.)

You aren't the first to raise this issue.  I wouldn't mind if the user had 
the option, but then I again I tend to prefer the flag-for-every-feature 
approach which others who have more computing experience than me have said 
leads to problems due to the presence of many different ways to do things and 
unforseen interaction.s

I could definitely see the coding advantage in dealing with implementing 
algorithms that uses notation that is already 1-based.  I have come across 
this myself -- in fact just yesterday when I was coding up the Pade 
approximation to the matrix exponential using the pseudo-code algorithm given 
by Golub and Van Loan in their "Matrix Computations" book.

It seems to me like it would be a lot of work to add this feature back into 
the code now (there would be a million places to look for places where the 
code inherently assumes 0-based indexing).  It would also, as you mention, be 
inconsistent with Python.

A general approach would be to inherit from the UserArray for your codes and 
reimplement the __getitem__  and __getslice__ commands.  Your objects should 
still be able to be passed to many of the routines which expect arrays 
(because under the covers one of the first things the array_from_object 
C-code does is check to see if the  object has an __array__ method and calls 
it).  Note that this will not copy data around so there is minimal overhead.

But, you would have to take care to wrap the returned object back into an 
array_object.  (Maybe something could be done here...Hmmm.)

>
> By the way, what is leave-last-one-out slicing? Is it
> a[:-1]
> or is it
> a[0,...]
> or is it something else?

I meant the fact that a[3:6] returns elements a[3], a[4], a[5]  but NOT a[6]. 
I'm sorry for using my own poorly-worded term.  I can't remember what other 
Pythonistas call it.



From oliphant.travis at ieee.org  Tue Aug 28 19:38:17 2001
From: oliphant.travis at ieee.org (Travis Oliphant)
Date: Tue, 28 Aug 2001 23:38:17 +0000
Subject: [Numpy-discussion] one-offset arrays
In-Reply-To: <20010828220046.A4161@holler.physics.ubc.ca>
References: <20010828220046.A4161@holler.physics.ubc.ca>
Message-ID: <01082823381702.12637@travis>

Eric brought up the point of one-offset arrays which can be relatively easily 
created with UserArrays.

This led me to some thinking about why UserArrays are not more often used.

I think one of the biggest reasons is that most of the functions can take 
UserArrays but returned the basic array type upon completion.  So, you end up 
having to continually construct your UserArrays.

Are there other reasons people have thought of?

So, here's a suggestion:

Why don't we modify PyArray_Return to return an object of the same class as 
one of the arguments which was passed if the class defines an __array__ 
method. 

Which argument to pick and how this would be implemented without changing old 
code is an interesting question.

Assuming it were possible to cause PyArray_Return to do such a thing, would 
this be a bad idea?

Sincrely,

Tried-to-use-Matrix-objects-but-always-resort-to-dot(x,x)

Travis Oliphant



From hinsen at cnrs-orleans.fr  Wed Aug 29 05:38:16 2001
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Wed, 29 Aug 2001 11:38:16 +0200
Subject: [Numpy-discussion] invert() with complex arrays
In-Reply-To: <3B8C5949.1F9C0531@home.com> (message from Rob on Tue, 28 Aug
	2001 19:54:01 -0700)
References: <3B8C5949.1F9C0531@home.com>
Message-ID: <200108290938.f7T9cGe11697@chinon.cnrs-orleans.fr>

> Is there any way to get invert() to work with complex arrays?

If you mean LinearAlgebra.inverse(), it should work with complex arrays,
although I haven't checked that for a long time.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------



From hinsen at cnrs-orleans.fr  Wed Aug 29 06:01:10 2001
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Wed, 29 Aug 2001 12:01:10 +0200
Subject: [Numpy-discussion] one-offset arrays
In-Reply-To: <20010828220046.A4161@holler.physics.ubc.ca> (message from Eric
	Nodwell on Tue, 28 Aug 2001 22:00:50 -0700)
References: <20010828220046.A4161@holler.physics.ubc.ca>
Message-ID: <200108291001.f7TA1AE14088@chinon.cnrs-orleans.fr>

> versus 0-based indexing is compatibility.  For numerical work, some of
> the languages which I use or have used are Matlab, Mathematica, Maple
> and Fortran. These are all 1-indexed. (C is by nature 0-indexed
> because it is so close to machine architecture, but with a little bit

I don't think this is a matter if numerics vs. machine orientation, it
is just an historical accident. Having used 0-based and 1-based
languages extensively, I find both perfectly suitable for numerical
work, and in my experience 0-based indexing leads to shorter index
expressions in many cases.

> symbols and function names for anther.  However it can be damn tricky
> to convert 1-indexed code to 0-indexed code or visa versa without
> introducing any errors- believe me!  (Yes it's possible to call nearly

Having done so recently, I agree :-(

> The other reason for choosing 1-based indexing is to keep the code as
> near to the standard notation as possible. This is one of the

I wouldn't call 1-based indexing "standard" - it all depends on your
background.

> question: as a simple example a spatial vector (x,y,z) is
> conventionally labelled 1,2,3 (1-indexed), but a relativistic
> four-vector with time included (t,x,y,z) is conventionally labelled
> 0,1,2,3 (0-indexed). So ideally one would be able to choose the

So, when you implement classes that represent vectors, you should
stick to whatever notation is common in the application domain.

I see bare-bones arrays as we have them in NumPy as a low-level
building block for high-level classes, i.e. not as a data type one
should use directly in non-trivial application code. It's not just
indexing, also operations. For your four-vectors, you would want a
method that calculates the norm in the Einstein metric, for example.
If you don't have a special four-vector class but use arrays directly,
any length calculation will look messy. That's what OO design is good
for. In fact, with special four-vector classes and appropriate
methods, there will hardly be any explicit indices in the application
code at all!

> indexing-type case-by-case. I'm sure that computer programmers will
> argue vehemently that code which mixes both 0-indexed and 1-indexed
> arrays is hard to understand and maintain, but for experts in a

Yes, if they are supposed to be the same data type. If different
classes use different indexing schemes, everything remains clear,
as long as the interactions between the classes are well defined.

> error-prone. In my case, I'm dealing at the moment with crystal
> structures with which are associated certain conventional sets of
> vectors and tensors - all 1-indexed by convention. I find it a

Have a look at the vector and tensor classes in Scientific Python
(http://dirac.cnrs-orleans.fr/programs/scientific.html). Although they
are 0-based as well (because I got so used to that!), you might find
that the predefined methods cover most of your needs and that explicit
indices are hardly needed. But you are also free to modify the
__getitem__ and __setitem__ methods to make the classes 1-based.

> I guess if I understood the reason for 0-indexed lists and tuples in
> Python I would be happier. In normal, everyday usage, sets,

Python has a very consistent indexing scheme, which you can best
understand in the following way: imagine that the indices are pointers
to positions *between* the elements:

elements           a   b   c   d

index            0   1   2   3   4
negative index  -4  -3  -2  -1

Then all the slicing and indexing rules make a lot of sense, and
fulfill practically useful relations. For example, it is true that

     x[a:b] + x[b:c]  equals  x[a:c]

for any list/array x, which would not be the case with 1-based
indices. Adding indices is also more straightforward with base 0, all
those "i+j-1" you see so frequently in Fortran code becomes just
"i+j".

Sure, everyday language works differently, but then you don't write
algorithms in everyday language, for a good reason.

> By the way, what is leave-last-one-out slicing? Is it
> a[:-1]

That one, as is clear from my little picture above.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------



From hinsen at cnrs-orleans.fr  Wed Aug 29 06:06:25 2001
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Wed, 29 Aug 2001 12:06:25 +0200
Subject: [Numpy-discussion] one-offset arrays
In-Reply-To: <01082823381702.12637@travis> (message from Travis Oliphant on
	Tue, 28 Aug 2001 23:38:17 +0000)
References: <20010828220046.A4161@holler.physics.ubc.ca> <01082823381702.12637@travis>
Message-ID: <200108291006.f7TA6PM14091@chinon.cnrs-orleans.fr>

> This led me to some thinking about why UserArrays are not more often used.
> 
> I think one of the biggest reasons is that most of the functions can
> take UserArrays but returned the basic array type upon completion.
> So, you end up having to continually construct your UserArrays.

Exactly.

> Are there other reasons people have thought of?

Performance, in some cases. Or passing objects to C routines that expect
plain arrays. But the function issue is certainly the major one.

> So, here's a suggestion:
> 
> Why don't we modify PyArray_Return to return an object of the same class as 
> one of the arguments which was passed if the class defines an __array__ 
> method. 

I don't think it can be done at the level of PyArray_Return, it cannot
know what the classes of the arguments were. We *could* achieve the
same goal by making appropriate modifications to all the C functions
in Numeric.

> Which argument to pick and how this would be implemented without
> changing old code is an interesting question.

No idea about the first issue, but compatibility can be assured (well,
in most cases) by picking some attribute name (__array__) that is not
used in the current UserArray class. We'd then have some other class
derived from UserArray which sets that attribute.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------



From europax at home.com  Wed Aug 29 10:09:06 2001
From: europax at home.com (Rob)
Date: Wed, 29 Aug 2001 07:09:06 -0700
Subject: [Numpy-discussion] invert() with complex arrays
References: <3B8C5949.1F9C0531@home.com> <200108290938.f7T9cGe11697@chinon.cnrs-orleans.fr>
Message-ID: <3B8CF782.68C56084@home.com>

Konrad Hinsen wrote:
> 
> > Is there any way to get invert() to work with complex arrays?
> 
> If you mean LinearAlgebra.inverse(), it should work with complex arrays,
> although I haven't checked that for a long time.
> 
> Konrad.
> --
> -------------------------------------------------------------------------------
> Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
> Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
> Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
> 45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
> France                                   | Nederlands/Francais
> -------------------------------------------------------------------------------
> 
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> http://lists.sourceforge.net/lists/listinfo/numpy-discussion


I tested a small matrix and got an error, but I will try again, since
with my luck it was singular :)

Rob.
-- 
The Numeric Python EM Project

www.members.home.net/europax



From hinsen at cnrs-orleans.fr  Wed Aug 29 10:13:52 2001
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Wed, 29 Aug 2001 16:13:52 +0200
Subject: [Numpy-discussion] invert() with complex arrays
In-Reply-To: <3B8CF782.68C56084@home.com> (message from Rob on Wed, 29 Aug
	2001 07:09:06 -0700)
References: <3B8C5949.1F9C0531@home.com> <200108290938.f7T9cGe11697@chinon.cnrs-orleans.fr> <3B8CF782.68C56084@home.com>
Message-ID: <200108291413.f7TEDq105660@chinon.cnrs-orleans.fr>

> > > Is there any way to get invert() to work with complex arrays?
> > 
> > If you mean LinearAlgebra.inverse(), it should work with complex arrays,
> > although I haven't checked that for a long time.
>
> I tested a small matrix and got an error, but I will try again, since
> with my luck it was singular :)

No need to take chances:

   >>> from Numeric import *
   >>> from LinearAlgebra import inverse
   >>> a = 1.j*array([[1., 0.], [0., 1.]])
   >>> a
   array([[ 0.+1.j,  0.+0.j],
	  [ 0.+0.j,  0.+1.j]])
   >>> inverse(a)
   array([[ 0.-1.j,  0.+0.j],
	  [ 0.+0.j,  0.-1.j]])

So it works for at least one complex matrix. :-)

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------



From chrishbarker at home.net  Wed Aug 29 13:41:57 2001
From: chrishbarker at home.net (Chris Barker)
Date: Wed, 29 Aug 2001 10:41:57 -0700
Subject: [Numpy-discussion] one-offset arrays
References: <20010828220046.A4161@holler.physics.ubc.ca>
Message-ID: <3B8D2965.18095B10@home.net>

It's mostly been said already, but I can't help but add my $.02
sometimes...

Eric Nodwell wrote:

> >> Without one-offset indexing, it seems to me that Python is minimally
> >> useful for numerical computations. Many, perhaps the majority, of
> >> numerical algorithms are one-indexed. Matlab for example is one-based
> >> for this reason.  In fact it seems strange to me that a "high-level"
> >> language like Python should use zero-offset lists.

I was a heavy user of MATLAB for a long time before I discovered NumPy,
and I have to say that I like the 0-indexing scheme MUCH better!

> In my view, the most important reason to prefer 1-based indexing
> versus 0-based indexing is compatibility.  For numerical work, some of
> the languages which I use or have used are Matlab, Mathematica, Maple
> and Fortran. These are all 1-indexed.

Actually Fortran is indexed however you decide you want it:

DIMENSION array(0:9)

DIMENSION array(1:10) or DIMENSION array(10)

DIMENSION array(1900:1999)

Are all legal. This is a VERY handy feature, and I would say that I used
the 0-indexed version most often. The reason is related to C's pointer
arithmetic logic: Often the array would represent discrete points on a
continuous scale, so I could find the value of X, for instance, by
doing:

Xaxis(i) = MinX * i * DeltaX

with i-indexing, you have to subtract 1 all the time.

I suspect that the higher level nature of NumPy would make it a lot
harder to have arbitrary indexing of this fashion: if all you have to do
is access elements, it is easy, but if you have a whole collection of
array oriented operations, as NumPy does, you would probably have to
stick with one standard, and I think the 0-indexing standard is the
best.

> for experts in a
> particular field who are accustomed to certain ingrained notations, it
> is the code which breaks the conventional notation which is most
> error-prone.

This is why being able to set your own indexing notation is the best
option, but a difficult one to impliment.

> Python is otherwise such an elegant and
> natural language. Why the ugly exception of making the user conform to
> the underlying mechanism of an array being an address plus an offset?

I gave an example above, and others have too: Python's indexing scheme
is elegant and natural for MANY usages. As with many things Python
(indentation, anyone!), I found it awkward to make the transition at
first, but then found that it, in fact, made things easier in general.
For me, this is the very essence of truly usable design: it is designed
to make people most productive in the long run, not most comfortable
when they first start using it. 
 
> All this is really neither here nor there, since this debate, at least
> as far as Python is concerned, was probably settled 10 years ago and

Well, yes, and having NumPy different from the rest of Python would NOT
be a good idea either.

> I'm sure nobody wants to hear anything more about it at this point.
> As you point out, I can define my own array type with inheritance. I
> will also need my own range command and several other functions which
> haven't occured to me yet. I was hoping that there would be a standard
> module to implement this.

If it were truly generally useful, there probably would be such a
package. I imagine most people have found it easier to make the
transition than to write a whole package that would allow you not to
make the transition. If you really have a lot of code that is 1-indexed
that you want to translate, it may be worth the effort for you, and I'm
sure there are other folks that would find it useful, but remember that
it will always be incompatable with the rest of Python, which may make
it harder to use than you imagine. 

-Chris

-- 
Christopher Barker,
Ph.D.                                                           
ChrisHBarker at home.net                 ---           ---           ---
http://members.home.net/barkerlohmann ---@@       -----@@       -----@@
                                   ------@@@     ------@@@     ------@@@
Oil Spill Modeling                ------   @    ------   @   ------   @
Water Resources Engineering       -------      ---------     --------    
Coastal and Fluvial Hydrodynamics --------------------------------------
------------------------------------------------------------------------



From chrishbarker at home.net  Wed Aug 29 17:10:19 2001
From: chrishbarker at home.net (Chris Barker)
Date: Wed, 29 Aug 2001 14:10:19 -0700
Subject: [Numpy-discussion] API tricks any my new "clip" function
References: <20010828162422.A560@holler.physics.ubc.ca> <01082819421703.09959@travis>
Message-ID: <3B8D5A3B.6F5A8E09@home.net>

Hi all,

I wrote last week looking for hints for writing a new clip function,
that would be faster than the existing one. I got some suggestions, and
have written the function. It works, but I'd love to get a little code
review from the more experienced folks here. I have a few specific
questions:

A) Are there any tricks to making a function work with multiple types? I
currently have it working with only double arrays, but it would be great
for it ot be universal (and it could then be added to the main NumPy
distribution, I suppose) I seem tohave to typecast the data to do the
comparison I want, so I can't figure out how to make it general. For
example, I have this line:

if (*(double *)(Array->data + i*elementsize) > *(double *)(Max->data +
i*elementsize))
{
          *(double *)(Array->data + i*elementsize) = *(double
*)(Max->data + i*elementsize) ;
}

How could I do that without the explicit typecasts? I'm pretty sure I
could make the rest of the routine general.

B) I was trying to figure out how to loop through all the elements of a
non-contiguous array. I got two hints:
"Rob W. W. Hooft" wrote:

> Never done this in C for numpy yet, but I normally program this
> along the following lines:
> 
> n=[2,3,2,3]
> x=[0,0,0,0]
> while 1:
>     print x
>     i=len(n)-1
>     while i>=0 and x[i]+1==n[i]:
>        x[i]=0
>        i=i-1
>     if i<0:
>        break
>     x[i]=x[i]+1
> 
> It is always going to be slower than the straight loop for contiguous
> arrays. 

This is what I did, and it is a LOT slower (about a factor of ten). See
the enclosed C code. There has got to be a way to optimize this!

>So if you want to do it properly, you'd have to check whether
> the array is contiguous, and if it is, use the fast way.

Since I had already written code for the contiguous case, it was easy to
special case it.

Pete Shinners wrote:
> it's easiest to use recursion than nested loops. then you can
> support any number of dimensions. that should point you in the
> right direction :]

When I read this, I thought: "well, duh!", but then I couldn't figure
out how to do it. Would there be any advantage over the above loops
anyway?

Is there either a more elgant or faster way to loop through all the
elements of these three arrays?

C) Where is the DL_EXPORT macro defined? I havn't been able to find it
with grep yet. Does it work on all NumPy supported platforms?

D) Is there an exising function or Macro for checking if two arrays are
the same shape? I wrote one, but I imagine it exists.

F) I needed to work with input that could be any Python number, or an
Array of the right size. What I did was call: PyArray_FromObject on the
min and max inputs, so that I would get a PyArray, then I could check if
it was a rank-o array, or the right size. I havn't been able to decide
if this is a nifty trcik, or a kludge. Any other suggestions?

E) Have I got reference counting right? I tried to Py_DECREF all my temp
arrays, but I'm not sure I did it right.

Sorry to enclose the code if you have no interest in it, but it's not
really that big.

-Chris

-- 
Christopher Barker,
Ph.D.                                                           
ChrisHBarker at home.net                 ---           ---           ---
http://members.home.net/barkerlohmann ---@@       -----@@       -----@@
                                   ------@@@     ------@@@     ------@@@
Oil Spill Modeling                ------   @    ------   @   ------   @
Water Resources Engineering       -------      ---------     --------    
Coastal and Fluvial Hydrodynamics --------------------------------------
------------------------------------------------------------------------
-------------- next part --------------
#include "Python.h"
#include "arrayobject.h"

// These are little macros I use to access array values for specific rank arrays
#define ARRAYVAL0(aType,a) ( *(aType *)(a->data))	
#define ARRAYVAL1(aType,a,i) ( *(aType *)(a->data + (i)*a->strides[0]))	
#define ARRAYVAL2(aType,a,i,j) ( *(aType *)(a->data + (i)*a->strides[0] + (j)*a->strides[1]))	
#define ARRAYVAL3(aType,a,i,j,k) ( *(aType *)(a->data + (i)*a->strides[0] + (j)*a->strides[1] + (k)*a->strides[2]))	


int AreSameShape(PyArrayObject *Array1, PyArrayObject *Array2){

  int i;
  
  if (Array1->nd != Array2->nd){
    return 0;
  }
  for (i = 0; i < Array1->nd; i++){
    if (Array1->dimensions[i] != Array2->dimensions[i]){
      return 0;
    }
  }
  return 1;
}


int ComputeOffset(PyArrayObject *Array, int *indexes){

  int i, offset = 0;

  for (i = 0; i < Array->nd; i++)
    {
      offset += indexes[i]*Array->strides[i];
    }
  return offset;
}

void ReplaceValue(PyArrayObject *Array, PyArrayObject *Min, PyArrayObject *Max, int *indexes){
  // This function does the actual replacement of the values.
  // it allows Min and Max to be one element, or the same shape as Array
  // the offsets are computed separately, so as long as they are the same shape
  // it should work even if some are contiguous, and some not, for example

  double min, max;

  int offset;
    
  offset = ComputeOffset(Array,indexes);

  if (PyArray_Size((PyObject *)Min) == 1){
    min = ARRAYVAL0(double,Min);
  }
  else {
    min = *(double *)(Min->data + ComputeOffset(Min,indexes));
  }
  if (PyArray_Size((PyObject *)Max) == 1){
    max = ARRAYVAL0(double,Max);
  }
  else {
    max = *(double *)(Max->data + ComputeOffset(Max,indexes));
  }
  if (*(double *)(Array->data + offset) > max){
    *(double *)(Array->data + offset) = max ;
  }
  else if (*(double *)(Array->data + offset) < min){
    *(double *)(Array->data + offset) = min;
  }
  return;
}

static char doc_fastclip[] =
"fastclip(a,min,max):  a is clipped so that there are no values \n"
"less than min, or greater than max.\n\n"
"It only works on PyArrays of type Float.\n"
"min and max can be either scalar or the same size as a.\n"
"If a, min, and max are all contiguous (or scalar) then it is a LOT faster\n";

static PyObject * NumericExtras_fastclip(PyObject *self, PyObject *args)
{
  
  int NumArray, elementsize, i;
  int *indexes;

  PyObject *InMin;
  PyObject *InMax;

  PyArrayObject *Array;
  PyArrayObject *Min;
  PyArrayObject *Max;

  if (!PyArg_ParseTuple(args, "O!OO", 
						&PyArray_Type, &Array,
					    &InMin,
						&InMax))
    {
      return NULL;
    }  
  
  // check types of input

  if (Array->descr->type_num != PyArray_DOUBLE){
	PyErr_SetString (PyExc_ValueError,
					 "a must be a NumPy array of type Float");
    return NULL;
  }
  
  // convert min and max to double arrays:
  // if they don't convert, the input values are not valid
  // also check if they are the right size
  Min = (PyArrayObject *) PyArray_FromObject(InMin, PyArray_DOUBLE, 0, 0);
  if (Min == NULL){
    PyErr_SetString (PyExc_ValueError,
					 "min must be an object that can be converted to an array of Floats");
    return NULL;
  }

  if (!((PyArray_Size((PyObject *)Min) == 1) ||  AreSameShape(Min, Array))){
    PyErr_SetString (PyExc_ValueError,
					 "min must be either a scalar or the same size as a");
    Py_DECREF(Min);
    return NULL;
  }

  Max = (PyArrayObject *) PyArray_FromObject(InMax, PyArray_DOUBLE, 0, 0);
  if (Max == NULL){
    PyErr_SetString (PyExc_ValueError,
					 "max  must be an object that can be converted to an array of Floats");
    Py_DECREF(Min);
    return NULL;
  }

  if (!((PyArray_Size((PyObject *)Max) == 1) ||  AreSameShape(Max, Array))){
    PyErr_SetString (PyExc_ValueError,
					 "max must be either a scalar or the same size as a");
    Py_DECREF(Min);
    Py_DECREF(Max);
    return NULL;
  }

  // After all that input checking, switch between the contiguous and non-contiguous cases.
  if (PyArray_ISCONTIGUOUS(Array) && PyArray_ISCONTIGUOUS(Min) && PyArray_ISCONTIGUOUS(Max)){ //we have the contiguous case


    // This seems pretty kludgy to have the four cases, but it was the first
    // thing that came to mind that I was sure would work, and would accomidate
    // either array or scalar arguments for min and max. 
    // it's also faster than checking if they are scalar inside the loop

    NumArray = PyArray_Size((PyObject *)Array);
    elementsize = sizeof(double);

    if (PyArray_Size((PyObject *)Min) == 1 && PyArray_Size((PyObject *)Max) == 1){ // both limits are scalar
      for (i = 0; i < NumArray; i++) { // loop over each element
        if (*(double *)(Array->data + i*elementsize) > ARRAYVAL0(double,Max)){
          *(double *)(Array->data + i*elementsize) = ARRAYVAL0(double,Max);
        }
        else if (*(double *)(Array->data + i*elementsize) < ARRAYVAL0(double,Min)){
          *(double *)(Array->data + i*elementsize) = ARRAYVAL0(double,Min);
        }
      }
    }
    else if (PyArray_Size((PyObject *)Min) == 1 && PyArray_Size((PyObject *)Max) == NumArray){ // Min is scalar
      for (i = 0; i < NumArray; i++) { // loop over each element
        if (*(double *)(Array->data + i*elementsize) > *(double *)(Max->data + i*elementsize)){
          *(double *)(Array->data + i*elementsize) = *(double *)(Max->data + i*elementsize) ;
        }
        else if (*(double *)(Array->data + i*elementsize) < ARRAYVAL0(double,Min)){
          *(double *)(Array->data + i*elementsize) = ARRAYVAL0(double,Min);
        }
      }
    }    
    else if (PyArray_Size((PyObject *)Max) == 1 && PyArray_Size((PyObject *)Min) == NumArray){ // Max is scalar
      for (i = 0; i < NumArray; i++) { // loop over each element
        if (*(double *)(Array->data + i*elementsize) > ARRAYVAL0(double,Max)){
          *(double *)(Array->data + i*elementsize) = ARRAYVAL0(double,Max);
        }
        else if (*(double *)(Array->data + i*elementsize) < *(double *)(Min->data + i*elementsize)){
          *(double *)(Array->data + i*elementsize) = *(double *)(Min->data + i*elementsize) ;
        }
      }
    }    
    else if (PyArray_Size((PyObject *)Min) == NumArray && PyArray_Size((PyObject *)Max) == NumArray){ // Neither is scalar
      for (i = 0; i < NumArray; i++) { // loop over each element
        if (*(double *)(Array->data + i*elementsize) > *(double *)(Max->data + i*elementsize)){
          *(double *)(Array->data + i*elementsize) = *(double *)(Max->data + i*elementsize) ;
        }
        else if (*(double *)(Array->data + i*elementsize) < *(double *)(Min->data + i*elementsize)){
          *(double *)(Array->data + i*elementsize) = *(double *)(Min->data + i*elementsize) ;
        }
      }    
    }
    else { // The size of Min and/or Max don't match Array: we should never get here, do to previous checks.
      PyErr_SetString (PyExc_ValueError,
                       "min and max must be either scalar or the same shape as a");
      Py_DECREF(Min);
      Py_DECREF(Max);
      return NULL;
    }

  }
  else { // this now the non-contiguous case: it is much slower, but presumably could be optimised : suggestions gratefully excepted!
      
    indexes = calloc(Array->nd,sizeof(int));

    while (1){

      ReplaceValue(Array, Min, Max, indexes);

      i = (Array->nd) - 1;
      while ((i >= 0) && (indexes[i]+1 == Array->dimensions[i])){
        indexes[i] = 0;
        i -= 1;
      }
      if (i<0) {
        break;
      }
      indexes[i] = indexes[i]+1;
    } 

    free(indexes);
  }    
  Py_DECREF(Min);
  Py_DECREF(Max);
  return Py_None;
}

	
static PyMethodDef NumericExtrasMethods[] = {
  {"fastclip", NumericExtras_fastclip, METH_VARARGS, doc_fastclip},
  {NULL, NULL} /* Sentinel */
};

DL_EXPORT (void) initNumericExtras(){

  (void) Py_InitModule("NumericExtras", NumericExtrasMethods);
  import_array()
}





















From jsaenz at wm.lc.ehu.es  Wed Aug 29 17:16:19 2001
From: jsaenz at wm.lc.ehu.es (Jon Saenz)
Date: Wed, 29 Aug 2001 23:16:19 +0200 (MET DST)
Subject: [Numpy-discussion] API tricks any my new "clip" function
In-Reply-To: <3B8D5A3B.6F5A8E09@home.net>
Message-ID: <Pine.OSF.3.95.1010829230810.8194A-100000@lcdx00.wm.lc.ehu.es>

On Wed, 29 Aug 2001, Chris Barker wrote:

> A) Are there any tricks to making a function work with multiple types? I
> currently have it working with only double arrays, but it would be great
> for it ot be universal (and it could then be added to the main NumPy
> distribution, I suppose) I seem tohave to typecast the data to do the
> comparison I want, so I can't figure out how to make it general. For
> example, I have this line:
> 
> if (*(double *)(Array->data + i*elementsize) > *(double *)(Max->data +
> i*elementsize))
> {
>           *(double *)(Array->data + i*elementsize) = *(double
> *)(Max->data + i*elementsize) ;
> }
> 
> How could I do that without the explicit typecasts? I'm pretty sure I
> could make the rest of the routine general.
Some time ago, I used a union like:
typedef union {
short *sdata;
int *idata;
long *ldata;
float *fdata;
double *ddata; 
/* And so on */
} MyNumPyUnion;

Then, you can assign (and CAST) the data field of the NumPy array
according to the typecode of the array, like in:

MyNumPyUnion theunion;

theunion.fdata=(float*) thenumpyarray->data; /* If it is a Float32 */

Finally, you use  sdata, idata, ldata, fdsata and so on to iterate
dereferencing accordingly, depending on the typecode of the array.

It is a nightmare to write this code, but I can not imagine of any other
approach. May be other developers can help you with a better approach.

Hope this helps.


Jon Saenz.				| Tfno: +34 946012470
Depto. Fisica Aplicada II               | Fax:  +34 944648500
Facultad de Ciencias.   \\ Universidad del Pais Vasco \\
Apdo. 644   \\ 48080 - Bilbao  \\ SPAIN






From frankpit at erols.com  Wed Aug 29 23:43:27 2001
From: frankpit at erols.com (Bernard Frankpitt)
Date: Wed, 29 Aug 2001 23:43:27 -0400
Subject: [Numpy-discussion] cephes-1.3 install problem
References: <200108290328.f7T3SYP00391@cgpp.com>
Message-ID: <3B8DB65F.3010803@erols.com>

I came across this problem a few months ago, I hacked the indicated 
lines in what seemed,
at the time, to be the obvious fashion and the thing worked fine.  I'm 
sorry that I can't
be more helpful, but I haven't kept the file. The hack might just have 
been to comment out the offending statements.  

Bernie




From nodwell at physics.ubc.ca  Thu Aug 30 04:07:48 2001
From: nodwell at physics.ubc.ca (Eric Nodwell)
Date: Thu, 30 Aug 2001 01:07:48 -0700
Subject: [Numpy-discussion] one-offset arrays
Message-ID: <20010830010748.A28244@holler.physics.ubc.ca>

Thanks to everyone who responded to my comments about one-offset
arrays in Python.  I understand much better now why zero-offset arrays
are a good choice for Python.  For the reasons already discussed
(mostly ease of translating one-offset algorithms), one-offset arrays
would nevertheless be useful in certain situations, so I went ahead
and wrote a class for a one-offset array.

It ended up being a somewhat bigger job than I expected, and since it
is rather too much material for a mailing-list submission I created a
web site for it. Anyone who is interested in this topic please have a
look:

http://www.physics.ubc.ca/~mbelab/python/arrayone/index.html

I'd love some feedback along the following lines:

1. There are 4 outstanding problems/bugs which I could identify but
was unable to fix. Most of these seem to be due to limitations of the
UserArray class, but they could also be due to limitations of the
programmer. None of these problems make the class unusable. One of
them is the issue which Travis identified about functions taking
UserArrays but returning standard arrays. It would be nice if the
resulting discussion led to something be done... Any suggestions for
fixing the other 3 issues would be most welcome.

2. I would like this module to be as accessible as possible to others,
particularly those who are just getting started with Python and may
therefore be especially encumbered with one-offset baggage. How can I
do this?  Submit it to a Python resource such as the Vaults of
Parnassus? Get people to link to my web page? Does someone with a more
prominant website want to volunteer to host it? Is there any
possibility of having it included eventually with Numerical Python or
Scientific Python?  Probably some people will consider it too trivial
to include, but I found it less trivial than I expected once all the
cases were covered - and there are still the 4 outstanding
issues. (Yes the code sucks!  Suggestions for improvement are most
welcome!) Why make people reinvent the wheel? People coming from
MatLab for example might be inclined to give up and go back to MatLab.

Eric


P.S. In case anyone is interested in the outstanding problems but for
some reason doesn't want to or can't visit the web site, here are the
summaries:

Problem 1
---------

Description
for loops don't work with variables of type arrayone

Example

    X=arrayone((1,2,3))
    for item in X:
        print item
 This just generates an error.

Reason
In Python, a for loop works by starting at x(0) and incrementing until
an out-of-bounds error occurs. arrayone types have no 0 element.

Work-around:
Cast to type array in for loops. For example 
    X=arrayone((1,2,3))
    for item in array(X):
        print item


Possible solutions:
Is it possible to "overload" "for" so that it behaves differently for
arrayone type?


Problem 2
---------

Description
Slicing an arrayone from 0 doesn't generate an error but it should.

Example:

    X=arrayone((1,2,3))
    X[0:3]
 This returns ArrayOne.arrayone([1, 2, 3]) instead of an error.

Reason
X[:3] results in a call to __getslice__ with low=0. This cannot be
distinguished from X[0:3]. Therefore in order to deal correctly with
the X[:] case, we have to assume that low=0 means an unspecified low.

Work-around
If you don't trust your code, you have to implement specific checking
for this condition before slicing.

Possible solution
If it was possible to get access to the unparsed input (literally
'0:3' for example), this could be fixed.


Problem 3
---------

Description
Negative subscripts return a slice offset by -1 from the expected
result.

Example

    X=arrayone((1,2,3,4))
    X[-3:-2]
 This returns ArrayOne.arrayone([1, 2]) instead of
 ArrayOne.arrayone([2, 3]).

Reason
X[-3:-2] in the above example results in a call to __getslice__ with
low=1, high=2, which cannot be distinguished from X[1:2].

Work-around
Don't use negative index slicing with arrayone types.

Possible solution:
If had access to unparsed input, could fix this.

Problem 4
---------

Description
Functions which take arrayone arguments return type array.

Work-around
Have to cast back into arrayone, for example: 
    Y=arrayone(somefunc(X))

Possible solution
See http://www.geocrawler.com/lists/3/SourceForge/1329/0/6505926



From jsw at cdc.noaa.gov  Thu Aug 30 08:10:41 2001
From: jsw at cdc.noaa.gov (Jeff Whitaker)
Date: Thu, 30 Aug 2001 06:10:41 -0600 (MDT)
Subject: [Numpy-discussion] LinearAlgebra patch
Message-ID: <Pine.OSX.4.33.0108300603200.19262-100000@localhost.cdc.noaa.gov>

Has anyone tried my patch to LinearAlgebra.py and lapack_litemodule.c?
I've found major speedups (by a factor of 3-5) in all operations involving
an svd or a eigenanalysis of a symmetric matrix. Paul's initial comment
was that since it was not self-contained (it required linking external
blas and lapack libs) he could not consider putting it in.  I have
rectified that now by including new versions of f2c_lite.c, blas_lite.c
and d(z)lapack_lite.c that support the newer, faster lapack routines.

I would appreciate any reports on how it works (or doesn't) on other
platforms - I've tried it in on MacOS X and Solaris.

Grab it from the 'Patches' section of the sourceforge project page.


-Jeff

 --
Jeffrey S. Whitaker
Phone  : (303)497-6313 Meteorologist               FAX    : (303)497-6449
NOAA/OAR/CDC  R/CDC1        Email  : jsw at cdc.noaa.gov
325 Broadway                Web    : www.cdc.noaa.gov/~jsw
Boulder, CO, USA 80303-3328 Office : Skaggs Research Cntr 1D-124




From tim.hochberg at ieee.org  Thu Aug 30 10:44:07 2001
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Thu, 30 Aug 2001 07:44:07 -0700
Subject: [Numpy-discussion] one-offset arrays
References: <20010830010748.A28244@holler.physics.ubc.ca>
Message-ID: <042501c13162$393894d0$87740918@cx781526b>

Hi Eric,

Here are some comments on the problems you identify.

> Problem 1
> ---------
>
> Description
> for loops don't work with variables of type arrayone

Soon (Python 2.2?) you will be able to use the new iterator protocol to do
this correctly. So target the iterator protocol and this will begin to work
as people begin to upgrade.

> Problem 2
> ---------
>
> Description
> Slicing an arrayone from 0 doesn't generate an error but it should.

I was going to suggest removing __getslice__, and only using __getitem__.
However, the slices recieved by __getitem__ *sometimes* gets their first
value filled in with zero (in the same cases that __getslice__ is called)
instead of None. I regard this as a bug/misfeature and I just reported it on
sourceforge.


> Problem 3
> ---------
>
> Description
> Negative subscripts return a slice offset by -1 from the expected
> result.

This one _should_ be fixable just by removing __getslice__ and working with
__getitem__ only. (getitem is called instead of getslice). However it fails
for the same reason that this fix fails for problem 2. Mumble.

Note that both fixes work fine if you always include the second colon. One
approach would be to do all [] operations in getitem and have getslice raise
an error. This would force your users to always use a[-3:-2:] instead of
a[-2:-2]. The former can be made to work since it calls getitem with the raw
values [slice(-3,-2,None)] as opposed to munging them like the latter does.


-tim




From tim.hochberg at ieee.org  Thu Aug 30 11:07:48 2001
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Thu, 30 Aug 2001 08:07:48 -0700
Subject: [Numpy-discussion] one-offset arrays
References: <20010830010748.A28244@holler.physics.ubc.ca>
Message-ID: <043c01c13165$881c0480$87740918@cx781526b>

This is just a followup to my earlier message.

The getitem issues I identify there are fixed in Python 2.2, if you use new
style class (inherit from object). So, problems 1-3 can be fixed in Python
2.2. It seems that problem 4 could be fixed fairly easily in the python
layer by writing wrapper functions in Numeric.py. For example:

def cos(a):
   r = umath.cos(a)
   if isinstance(a, UserArray):
      r = a.__class__(r)
   return r

This would be potentially a bit tedious, but straightforward.

-tim




From chrishbarker at home.net  Thu Aug 30 13:12:59 2001
From: chrishbarker at home.net (Chris Barker)
Date: Thu, 30 Aug 2001 10:12:59 -0700
Subject: [Numpy-discussion] LinearAlgebra patch
References: <Pine.OSX.4.33.0108300603200.19262-100000@localhost.cdc.noaa.gov>
Message-ID: <3B8E741B.2A1073A6@home.net>

Jeff Whitaker wrote:
> Paul's initial comment
> was that since it was not self-contained (it required linking external
> blas and lapack libs) he could not consider putting it in.  I have
> rectified that now by including new versions of f2c_lite.c, blas_lite.c
> and d(z)lapack_lite.c that support the newer, faster lapack routines.

I don't know what it would take, but I would LOVE it if NumPy used a
highly optimised BLAS wherever possible. I have no idea whether there is
a good BLAS available for every major platform that has a licence that
allows it to be included with NumPy, but if so, I think it should be
included. I know this is more effor than just including a generic one,
but having optimum performace "out of the box" would be a great thing.

just my $.02, which is worth about that, as I am in littel position to
help...


-Chris



-- 
Christopher Barker,
Ph.D.                                                           
ChrisHBarker at home.net                 ---           ---           ---
http://members.home.net/barkerlohmann ---@@       -----@@       -----@@
                                   ------@@@     ------@@@     ------@@@
Oil Spill Modeling                ------   @    ------   @   ------   @
Water Resources Engineering       -------      ---------     --------    
Coastal and Fluvial Hydrodynamics --------------------------------------
------------------------------------------------------------------------



From jsw at cdc.noaa.gov  Thu Aug 30 13:08:07 2001
From: jsw at cdc.noaa.gov (Jeff Whitaker)
Date: Thu, 30 Aug 2001 11:08:07 -0600 (MDT)
Subject: [Numpy-discussion] LinearAlgebra patch
In-Reply-To: <3B8E741B.2A1073A6@home.net>
Message-ID: <Pine.OSX.4.33.0108301103000.2852-100000@crdmac10.cdc.noaa.gov>

Chris:  The ATLAS project (http://www.netlib.org/ATLAS) provides a
mechanism for creating optimized BLAS/LAPACK libs on almost any
(unixish) platform.  The tuning process is very complicated.  It would not
be feasible to include this in the Numpy distribution - however the
mechanism does exist to link these libs when compiling numpy.  Perhaps a
more realistic goal would be to modify the setup.py script to
automatically detect ATLAS and use those libs if they are installed.

-Jeff

 On Thu, 30 Aug 2001, Chris Barker wrote:

> Jeff Whitaker wrote:
> > Paul's initial comment
> > was that since it was not self-contained (it required linking external
> > blas and lapack libs) he could not consider putting it in.  I have
> > rectified that now by including new versions of f2c_lite.c, blas_lite.c
> > and d(z)lapack_lite.c that support the newer, faster lapack routines.
>
> I don't know what it would take, but I would LOVE it if NumPy used a
> highly optimised BLAS wherever possible. I have no idea whether there is
> a good BLAS available for every major platform that has a licence that
> allows it to be included with NumPy, but if so, I think it should be
> included. I know this is more effor than just including a generic one,
> but having optimum performace "out of the box" would be a great thing.
>
> just my $.02, which is worth about that, as I am in littel position to
> help...
>
>
> -Chris
>
>
>
>

-- 
Jeffrey S. Whitaker         Phone  : (303)497-6313
Meteorologist               FAX    : (303)497-6449
NOAA/OAR/CDC  R/CDC1        Email  : jsw at cdc.noaa.gov
325 Broadway                Web    : www.cdc.noaa.gov/~jsw
Boulder, CO, USA 80303-3328 Office : Skaggs Research Cntr 1D-124




From europax at home.com  Thu Aug 30 20:23:52 2001
From: europax at home.com (Rob)
Date: Thu, 30 Aug 2001 17:23:52 -0700
Subject: [Numpy-discussion] LinearAlgebra patch/ Athlon optimization
References: <Pine.OSX.4.33.0108300603200.19262-100000@localhost.cdc.noaa.gov> <3B8E741B.2A1073A6@home.net>
Message-ID: <3B8ED918.A7A3DDD6@home.com>

I'd love to have Python and Numpy optimized for the Athlon.  I will have
to wait for gcc3.0 to get absorbed into FreeBSD.  (Mandrake linux has a
beta that uses it)  It turns out that mpg encoding runs at 10x the rate
on my Athlon DDR machine compared to my laptop due to the highly Athlon
optimized "GoGo" encoding program.  Rob.  



Chris Barker wrote:
> 
> Jeff Whitaker wrote:
> > Paul's initial comment
> > was that since it was not self-contained (it required linking external
> > blas and lapack libs) he could not consider putting it in.  I have
> > rectified that now by including new versions of f2c_lite.c, blas_lite.c
> > and d(z)lapack_lite.c that support the newer, faster lapack routines.
> 
> I don't know what it would take, but I would LOVE it if NumPy used a
> highly optimised BLAS wherever possible. I have no idea whether there is
> a good BLAS available for every major platform that has a licence that
> allows it to be included with NumPy, but if so, I think it should be
> included. I know this is more effor than just including a generic one,
> but having optimum performace "out of the box" would be a great thing.
> 
> just my $.02, which is worth about that, as I am in littel position to
> help...
> 
> -Chris
> 
> --
> Christopher Barker,
> Ph.D.
> ChrisHBarker at home.net                 ---           ---           ---
> http://members.home.net/barkerlohmann ---@@       -----@@       -----@@
>                                    ------@@@     ------@@@     ------@@@
> Oil Spill Modeling                ------   @    ------   @   ------   @
> Water Resources Engineering       -------      ---------     --------
> Coastal and Fluvial Hydrodynamics --------------------------------------
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> http://lists.sourceforge.net/lists/listinfo/numpy-discussion

-- 
The Numeric Python EM Project

www.members.home.net/europax



From europax at home.com  Thu Aug 30 20:28:06 2001
From: europax at home.com (Rob)
Date: Thu, 30 Aug 2001 17:28:06 -0700
Subject: [Numpy-discussion] invert() function works just fine
Message-ID: <3B8EDA16.A0F3C134@home.com>

My problems with the invert() function has been resolved.  I found that
somewhere in my program it was trying to invert [0].  So I just added a
conditional statement ahead of it.  Evidently the C program did the same
sort of thing.  So now I've shaved another 20% of execution time off of
my program!

Rob.
-- 
The Numeric Python EM Project

www.members.home.net/europax



From fredsten at kth.se  Fri Aug 31 08:21:43 2001
From: fredsten at kth.se (Fredrik Stenberg)
Date: Fri, 31 Aug 2001 14:21:43 +0200
Subject: [Numpy-discussion] Cholesky factorization
Message-ID: <184672001853112214393@kth.se>

Hi..

Has anyone written a Cholesky factorization for general 
matrices?

Any package out there?

The only one i know of is the python 
wrappers around lapack (pylapack).

I could write it myself, but i prefer to be lazy ;))

/fredriks




From Aureli.Soria_Frisch at ipk.fhg.de  Fri Aug 31 10:54:03 2001
From: Aureli.Soria_Frisch at ipk.fhg.de (Aureli Soria Frisch)
Date: Fri, 31 Aug 2001 16:54:03 +0200
Subject: [Numpy-discussion] Help: Problem embedding Python with Numeric
In-Reply-To: <184672001853112214393@kth.se>
Message-ID: <l03010d01b7b552d9dd80@[153.97.92.109]>

Hello,

I am writing a C program, which uses a class I have written in Python. The
class has a method 'operate_image' that receives as parameters a Numeric
array, an integer and a float.

The class has been repeatedly used with no problems.

When I try to call the method I use:

pres_fi=PyObject_CallMethod(pFI,"operate_image","(Oif)",parray,1,1.0);

But the result (pres-fi) is NULL. I have tested pFI to exist and to have
the mentioned method, with no troubles.

So it seems to me that PyObject_CallMethod is casting down the parray
object, which is a PyArrayObject. That is the reason of malfunctioning.

Questions:
1) Is my interpratation right?
2) Which would be the correct way to do such an operation?

It is the first time I write an Python embedded/extension, so I am quite lost.

Thanks in advance,
Aureli


#################################
Aureli Soria Frisch
Fraunhofer IPK
Dept. Pattern Recognition

post: Pascalstr. 8-9, 10587 Berlin, Germany
e-mail:aureli at ipk.fhg.de
fon: +49 30 39006-150
fax: +49 30 3917517
#################################





From KRodgers at ryanaero.com  Fri Aug 31 12:23:36 2001
From: KRodgers at ryanaero.com (Rodgers, Kevin)
Date: Fri, 31 Aug 2001 09:23:36 -0700
Subject: [Numpy-discussion] one-offset arrays
Message-ID: <0D8C1A50C283D311ABB800508B612E53054F85F2@xcgca065.ryanaero.com>

Indices in Fortran can be defined at the time an array is declared, so
Fortran is not necessarily "one-indexed".

Kevin Rodgers
Northrop Grumman Ryan Aeronautical

-----Original Message-----
From: Eric Nodwell [mailto:nodwell at physics.ubc.ca]
Sent: Tuesday, August 28, 2001 10:01 PM
To: Numpy-discussion at lists.sourceforge.net
Subject: Re: [Numpy-discussion] one-offset arrays



>> Does anyone have a simple method of using one-offset arrays?
>> Specifically, can I define an array "a" so that a[1] refers to the
>> first element?
>>

> Inherit from the UserArray and redefine slicing to your hearts content.

>> Without one-offset indexing, it seems to me that Python is minimally
>> useful for numerical computations. Many, perhaps the majority, of
>> numerical algorithms are one-indexed. Matlab for example is one-based
>> for this reason.  In fact it seems strange to me that a "high-level"
>> language like Python should use zero-offset lists.
>>

> Wow, that is a pretty condescending-sounding statement, though I'm sure
you 
> didn't mean it that way.  Python is indeed being used for quite serious 
> numerical computations.  I have been using Python for quite some time for 
> Numerical work and find it's zero-based indexing combined with the 
> leave-last-one-out slicing notation to be much more convenient.  

Oops, what I really need is a wet-ware (i.e. brain) macro which
enforces the correct order of the pair (think,write,send)! The above
unconsidered comments arose from the sequence
(frustration,write,send,think,regret).  ;)

Obviously there is a lot of numerical work being done with python and
people are very happy with it. But for me I still think it would be
"minimally useful" without 1-indexed arrays. Here's why:

In my view, the most important reason to prefer 1-based indexing
versus 0-based indexing is compatibility.  For numerical work, some of
the languages which I use or have used are Matlab, Mathematica, Maple
and Fortran. These are all 1-indexed. (C is by nature 0-indexed
because it is so close to machine architecture, but with a little bit
of not-entirely-clean pointer manipulation, you can easily make
1-indexed arrays and matrices.) Obviously Python can't be compatible
with these languages in a strict sense, but like most people who do
some programming work, I've built up a library of my own commonly used
routines specific to my work; in general it's a trivial matter to
translate numerical routines from one language to the another if
translation is just a matter of substituting of one set of syntactical
symbols and function names for anther.  However it can be damn tricky
to convert 1-indexed code to 0-indexed code or visa versa without
introducing any errors- believe me!  (Yes it's possible to call nearly
any language from nearly any other language these days so in theory
you don't have to recode, but there are lots of reasons why often
recoding is the preferable route.)

The other reason for choosing 1-based indexing is to keep the code as
near to the standard notation as possible. This is one of the
advantages of using a high level language - it approximates the way
you think about things instead of the way the computer organizes
them. Of course, this can go either way depending on the quantity in
question: as a simple example a spatial vector (x,y,z) is
conventionally labelled 1,2,3 (1-indexed), but a relativistic
four-vector with time included (t,x,y,z) is conventionally labelled
0,1,2,3 (0-indexed). So ideally one would be able to choose the
indexing-type case-by-case. I'm sure that computer programmers will
argue vehemently that code which mixes both 0-indexed and 1-indexed
arrays is hard to understand and maintain, but for experts in a
particular field who are accustomed to certain ingrained notations, it
is the code which breaks the conventional notation which is most
error-prone. In my case, I'm dealing at the moment with crystal
structures with which are associated certain conventional sets of
vectors and tensors - all 1-indexed by convention. I find it a
delicate enough task to always get the correct vector or tensor
without having to remember that d[2] is actually d3. Defining d1,d2,d3
is not convenient because often the index itself needs to be
calculated.

I guess if I understood the reason for 0-indexed lists and tuples in
Python I would be happier. In normal, everyday usage, sets,
collections and lists are 1-indexed (the first item, the second item,
the third item, and so on). Python is otherwise such an elegant and
natural language. Why the ugly exception of making the user conform to
the underlying mechanism of an array being an address plus an offset?

All this is really neither here nor there, since this debate, at least
as far as Python is concerned, was probably settled 10 years ago and
I'm sure nobody wants to hear anything more about it at this point.
As you point out, I can define my own array type with inheritance. I
will also need my own range command and several other functions which
haven't occured to me yet. I was hoping that there would be a standard
module to implement this.


By the way, what is leave-last-one-out slicing? Is it
a[:-1]
or is it
a[0,...]
or is it something else?


Eric

_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
http://lists.sourceforge.net/lists/listinfo/numpy-discussion



From huaiyu_zhu at yahoo.com  Fri Aug 31 04:11:56 2001
From: huaiyu_zhu at yahoo.com (Huaiyu Zhu)
Date: Fri, 31 Aug 2001 01:11:56 -0700 (PDT)
Subject: [Numpy-discussion] Subarray with with arbitrary index?
In-Reply-To: <ADEOIFHFONCLEEPKCACCKEBNCMAA.paul@pfdubois.com>
Message-ID: <Pine.LNX.4.10.10108310048010.1394-100000@mars.localdomain>

Following up to an earlier discussion, I've written up some wrapper
functions for subarrays.  The 'takes' function only works for 2d arrays,
but the 'puts' function works for arbitrary array.  Hopefully something
like this can be included in Numeric module.

Huaiyu Zhu


"""
Wrapper functions for dealing with arbitrary subarrays.

Example: addeqs(x, [[0,2], [1,3,4], y), would add (2,3)array y to the
	(2,3)subarray of x comprised of rows 0, 2 and columns 1, 3, 4.

A more natural notation of addeqs(x, ij, y) would be x[ij] += y, but seems
difficult to do with current numpy. (How to let slice handle lists?)

"""

from Numeric import *

def puts(x, irows, icols, v):
	"""
	puts(x, i, j, v): Put v in subgrid of 2d array x given by i, j.
	"""
	nrow, ncol = x.shape
	if irows is None:		irows = arange(nrow)
	if icols is None:		icols = arange(ncol)
	if len(shape(icols)) == 1:
		icols = icols[:ncol]
	if len(shape(irows)) == 0 or len(shape(icols)) == 0:
		ii = irows*ncol + icols
		v1 = v
	else:
		ii = (irows*ncol)[:, NewAxis] + icols[NewAxis, :]
		v1 = reshape(v, shape(ii))
	put(x, ii, v1)

def takes(x, I):
	"""
	takes(x, I): Takes a subgrid from array x.
	I is a list of list of subindices.
	"""
	for i in xrange(len(I)):
		ii = I[i]
		if ii is not None:
			x = take(x, ii, i)
	return x

def addeqs(x, ij, y):
	"""
	Simulates x[ij] += y, where ij can be arbitray subarray.
	"""
	i, j = ij
	puts(x, i, j, takes(x, ij) + y)

if __name__ == "__main__":
	a5 = arange(5)
	a2 = arange(2)
	a3 = arange(3)
	d = array([a3, a3+3])
	print d; print

	b = array([a5, a5+5, a5+10, a5+15]);	print b; print
	c = b.copy();	puts(c, None, 3, a5+1000);	print c; print
	c = b.copy();	puts(c, a2*2, 3, a5+1000);	print c; print
	c = b.copy();	puts(c, 2, a2*2, a5+1000);	print c; print
	c = b.copy();	puts(c, a2*2+1, a3*2, d+1000);	print c; print
	
	c = b.copy();	d1 = takes(c, (a2*2+1, a3*2))
	c1 = c
	print d1; print
	puts(c, a2*2+1, a3*2, d+1000);	print c; print
	puts(c, a2*2+1, a3*2, d1);	print c; print

	addeqs(c, (a2*2+1, a3*2), d1*0+100);	print c; print
	print c1; print
	d1 += 20; print c; print  # Alas, this does not change c

	d2 = takes(c, (a2*2+1, None))
	print d2; print
	print shape(c), shape(a2), shape(d2)
	addeqs(c, (a2*2+1, None), -d2)
	print c; print

"""
The expected results are
[[0 1 2]
 [3 4 5]]

[[ 0  1  2  3  4]
 [ 5  6  7  8  9]
 [10 11 12 13 14]
 [15 16 17 18 19]]

[[   0    1    2 1000    4]
 [   5    6    7 1001    9]
 [  10   11   12 1002   14]
 [  15   16   17 1003   19]]

[[   0    1    2 1000    4]
 [   5    6    7    8    9]
 [  10   11   12 1001   14]
 [  15   16   17   18   19]]

[[   0    1    2    3    4]
 [   5    6    7    8    9]
 [1000   11 1001   13   14]
 [  15   16   17   18   19]]

[[   0    1    2    3    4]
 [1000    6 1001    8 1002]
 [  10   11   12   13   14]
 [1003   16 1004   18 1005]]

[[ 5  7  9]
 [15 17 19]]

[[   0    1    2    3    4]
 [1000    6 1001    8 1002]
 [  10   11   12   13   14]
 [1003   16 1004   18 1005]]

[[ 0  1  2  3  4]
 [ 5  6  7  8  9]
 [10 11 12 13 14]
 [15 16 17 18 19]]

[[  0   1   2   3   4]
 [105   6 107   8 109]
 [ 10  11  12  13  14]
 [115  16 117  18 119]]

[[  0   1   2   3   4]
 [105   6 107   8 109]
 [ 10  11  12  13  14]
 [115  16 117  18 119]]

[[  0   1   2   3   4]
 [105   6 107   8 109]
 [ 10  11  12  13  14]
 [115  16 117  18 119]]

[[105   6 107   8 109]
 [115  16 117  18 119]]

(4, 5) (2,) (2, 5)
[[ 0  1  2  3  4]
 [ 0  0  0  0  0]
 [10 11 12 13 14]
 [ 0  0  0  0  0]]
"""




From jsw at cdc.noaa.gov  Fri Aug 31 16:51:13 2001
From: jsw at cdc.noaa.gov (Jeff Whitaker)
Date: Fri, 31 Aug 2001 14:51:13 -0600 (MDT)
Subject: [Numpy-discussion] Cholesky factorization
In-Reply-To: <633420018531173032787@kth.se>
Message-ID: <Pine.OSX.4.33.0108311448560.7556-100000@crdmac10.cdc.noaa.gov>

I've uploaded a new version of my LinearAlgebra patch that adds a Cholesky
decomposition routine (cholesky_decomposition), as per Frederik's request.

-Jeff

On Fri, 31 Aug 2001, Fredrik Stenberg wrote:

> You read my mind......
>
> ;))
>
> /fredriks
>
>
> ------- Ursprungligt meddelande -------
> Fr?n: jsw at cdc.noaa.gov
> Datum: Fri, 31 Aug 2001 08:32:01 -0700
>
> This message was sent from Geocrawler.com by "jeff whitaker"
> <jsw at cdc.noaa.gov>
>
>
> Fredrik:  I could easily add this to the
> LinearAlgebra patches I've just submitted.  Are
> you thinking of an interface to the lapack routine
> dpotrf (cholesky factorization of a real symmteric
> pos definite matrix), or something else?
>
> -Jeff
>
> ---------------------------------------
> Hi..
>
> Has anyone written a Cholesky factorization for
> general
> matrices?
>
> Any package out there?
>
> The only one i know of is the python
> wrappers around lapack (pylapack).
>
> I could write it myself, but i prefer to be lazy ;))
>
> /fredriks
>
>
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> http://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
>
>
> Geocrawler.com - The Knowledge Archive
>
>
>
>

-- 
Jeffrey S. Whitaker         Phone  : (303)497-6313
Meteorologist               FAX    : (303)497-6449
NOAA/OAR/CDC  R/CDC1        Email  : jsw at cdc.noaa.gov
325 Broadway                Web    : www.cdc.noaa.gov/~jsw
Boulder, CO, USA 80303-3328 Office : Skaggs Research Cntr 1D-124




From milgram at cgpp.com  Fri Aug 31 08:32:11 2001
From: milgram at cgpp.com (J. Milgram)
Date: Fri, 31 Aug 2001 08:32:11 -0400
Subject: [Numpy-discussion] cephes-1.3 install problem 
In-Reply-To: Your message of "Wed, 29 Aug 2001 23:43:27 EDT."
             <3B8DB65F.3010803@erols.com> 
Message-ID: <200108311233.f7VCWCm16235@cgpp.com>


> I came across this problem a few months ago, I hacked the indicated
> lines in what seemed, at the time, to be the obvious fashion and the
> thing worked fine.  I'm sorry that I can't be more helpful, but I
> haven't kept the file. The hack might just have been to comment out
> the offending statements.

Yes, I tried just that (also suggested by Travis Oliphant), and it
compiled just fine.

Thanks

Judah


Judah Milgram                             milgram at cgpp.com
College Park Press                        http://www.cgpp.com
P.O. Box 143, College Park MD, USA 20741  +001 (301) 422-4626 (422-3047 fax)



From europax at home.com  Fri Aug 31 19:43:03 2001
From: europax at home.com (Rob)
Date: Fri Aug 31 19:43:03 2001
Subject: [Numpy-discussion] matrix invert() problems
Message-ID: <3B904AF6.D0857135@home.com>

I found a slight discrepancy in my Python EM code output when I use the
invert() function.  It may be that the C routine that I am comparing it
to actually has the bug.  For now I don't know.  It would be cool to
have a large two dimensional array that has a known inverse.  I think
NIST keeps an array collection, but I don't know if they have the
inverses.  Rob.


-- 
The Numeric Python EM Project

www.members.home.net/europax



From europax at home.com  Fri Aug 31 22:48:02 2001
From: europax at home.com (Rob)
Date: Fri Aug 31 22:48:02 2001
Subject: [Numpy-discussion] matrix invert() problems
References: <3B904AF6.D0857135@home.com>
Message-ID: <3B907642.A0A9D3EE@home.com>

I'm pretty certain now that this is a numeric precision problem.  The
discrepancy only shows up for very small numbers.  For now I'll ignore
it.  Rob.

Rob wrote:
> 
> I found a slight discrepancy in my Python EM code output when I use the
> invert() function.  It may be that the C routine that I am comparing it
> to actually has the bug.  For now I don't know.  It would be cool to
> have a large two dimensional array that has a known inverse.  I think
> NIST keeps an array collection, but I don't know if they have the
> inverses.  Rob.
> 
> --
> The Numeric Python EM Project
> 
> www.members.home.net/europax
> 
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion

-- 
The Numeric Python EM Project

www.members.home.net/europax



From n8gray at caltech.edu  Thu Aug  2 20:18:13 2001
From: n8gray at caltech.edu (Nathaniel Gray)
Date: Thu, 2 Aug 2001 17:18:13 -0700
Subject: [Numpy-discussion] A savespace bug, perhaps?
Message-ID: <01080217181302.01319@DHCP-134-182>

Hi everybody,

I'm using Python 2.1.1 and Numpy 20.1.0  This seems to be a problem:

Python 2.1.1 (#2, Jul 31 2001, 14:10:42)                                     
[GCC 2.96 20000731 (Linux-Mandrake 8.0 2.96-0.48mdk)] on linux2
Type "copyright", "credits" or "license" for more information.
>>> from Numeric import *
>>> from MLab import zeros
>>> joe = zeros(3,'l',1)
>>> fred = array([1,2,3],'l')
>>> joe += fred
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: return array has incorrect type

This fails whether or not fred has the spacesaver flag set.

This, however, works fine:
>>> joe = zeros(3, 'l')
>>> fred = array([1,2,3], 'l')
>>> joe += fred
>>> joe
array([1, 2, 3])

Am I doing something wrong or is this a bug?

Please CC to me since I'm not subscribed to the list.
Thanks a lot!
-Nathan

-- 
Nathaniel Gray

California Institute of Technology
Computation and Neural Systems
--



From oliphant.travis at ieee.org  Thu Aug  2 15:13:48 2001
From: oliphant.travis at ieee.org (Travis Oliphant)
Date: Thu, 2 Aug 2001 19:13:48 +0000
Subject: [Numpy-discussion] A savespace bug, perhaps?
In-Reply-To: <01080217181302.01319@DHCP-134-182>
References: <01080217181302.01319@DHCP-134-182>
Message-ID: <01080219134800.17512@travis>

On Fri, 03 Aug 2001, Nathaniel Gray wrote:
> Hi everybody,
>
> I'm using Python 2.1.1 and Numpy 20.1.0  This seems to be a problem:
>
> Python 2.1.1 (#2, Jul 31 2001, 14:10:42)
> [GCC 2.96 20000731 (Linux-Mandrake 8.0 2.96-0.48mdk)] on linux2
> Type "copyright", "credits" or "license" for more information.
>
> >>> from Numeric import *
> >>> from MLab import zeros
> >>> joe = zeros(3,'l',1)
> >>> fred = array([1,2,3],'l')
> >>> joe += fred
>
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
> TypeError: return array has incorrect type
>
> This fails whether or not fred has the spacesaver flag set.
>
> This, however, works fine:
> >>> joe = zeros(3, 'l')
> >>> fred = array([1,2,3], 'l')
> >>> joe += fred
> >>> joe
>
> array([1, 2, 3])
>
> Am I doing something wrong or is this a bug?
>
> Please CC to me since I'm not subscribed to the list.
> Thanks a lot!
> -Nathan

Thanks for the bug report, Nathan. 

It was indeed a bug, which I hope has now been squashed in the latest CVS 
version of Numeric.  

Thanks for your help.

Travis




From europax at home.com  Fri Aug  3 18:11:12 2001
From: europax at home.com (Rob)
Date: Fri, 03 Aug 2001 15:11:12 -0700
Subject: [Numpy-discussion] new to list
Message-ID: <3B6B2180.FBDCDBF9@home.com>

Hello,

I'm new to the list so I thought I'd introduce myself.  I use Numpy at
work for some Python programs that interface with Agilent and/or Anritsu
microwave equipment, using the GPIB bus.

However, my main interest lately is porting some EM codes into Numeric
Python.  I've had varied success.  See my web site
www.members.home.net/europax.  Right now I'm trying to Numpy-ize this
FEM-MOM code.  I've profiled it using the Python profile class.  Using
MOM for a dipole takes 5 minutes of cpu :) :)  Interestingly, some of
the worst routines are simple ones that really have little to do with
matrix math.
Anyway, I'm having fun.  Its wonderful to be able to focus on algorithms
rather than wading through arcane programming syntax !!

Rob.



From gblee at mail.paichai.ac.kr  Mon Aug  6 22:45:03 2001
From: gblee at mail.paichai.ac.kr (=?ks_c_5601-1987?B?wMyx1LrA?=)
Date: Tue, 7 Aug 2001 11:45:03 +0900
Subject: [Numpy-discussion] NumTut
Message-ID: <000a01c11eea$f5e09040$7386facb@paichai.ac.kr>

Dear who may be concerned,

I want to install the package NumTut. But I can't find the directory of source distribution. Please let me know how to get the directory odf souce distribution and the subdirectory Demo.

Thanks

Gyoy-Bong Lee from Korea
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20010807/766674bd/attachment-0001.html>

From dubois1 at llnl.gov  Tue Aug  7 13:20:19 2001
From: dubois1 at llnl.gov (Paul F. Dubois)
Date: Tue, 7 Aug 2001 10:20:19 -0700
Subject: [Numpy-discussion] swig and Numeric
Message-ID: <01080710213100.17772@almanac>

Does someone have an example of using SWIG to connect Numeric arrays to C code
that expects double* arguments?



From hungjunglu at yahoo.com  Tue Aug  7 15:50:04 2001
From: hungjunglu at yahoo.com (Hung Jung Lu)
Date: Tue, 7 Aug 2001 12:50:04 -0700 (PDT)
Subject: [Numpy-discussion] fast constructor for arrays from byte data (unpickling?)
Message-ID: <20010807195004.19068.qmail@web12606.mail.yahoo.com>

Hi,

Arrays have a method called tostring() which generates
the binary data. Is there an inverse function to that?
That is, generating an array from the binary data
string?

For large matrices, pickling/unpickling is a bit too
much overhead (data stored as ASCII character strings
instead of binary data strings.) I know, I am talking
about a factor 4 here. But there is a big difference
between 1 minute loading time and 4 minute loading
time.

I would imagine this is a very common problem/request
for people working with large matrices. And I am sure
hacking the C code to provide another fast constructor
for arrays from binary strings won't be too hard. The
questions is: has anyone already tried it? Is it
already there?

(For the kludge masters: one kludge is of course to
store the binary data on disk, then use cStringIO to
build the pickled file and then unpickle from the
cStringIO. Speed is probably OK since the pickled file
lives on RAM. But that's a kludge. :) )

regards,

Hung Jung


__________________________________________________
Do You Yahoo!?
Make international calls for as low as $.04/minute with Yahoo! Messenger
http://phonecard.yahoo.com/



From kern at caltech.edu  Tue Aug  7 15:59:37 2001
From: kern at caltech.edu (Robert Kern)
Date: Tue, 7 Aug 2001 12:59:37 -0700
Subject: [Numpy-discussion] fast constructor for arrays from byte data (unpickling?)
In-Reply-To: <20010807195004.19068.qmail@web12606.mail.yahoo.com>
References: <20010807195004.19068.qmail@web12606.mail.yahoo.com>
Message-ID: <20010807125937.E22660@myrddin.caltech.edu>

On Tue, Aug 07, 2001 at 12:50:04PM -0700, Hung Jung Lu wrote:
> Hi,
> 
> Arrays have a method called tostring() which generates
> the binary data. Is there an inverse function to that?
> That is, generating an array from the binary data
> string?

Numeric.fromstring

HTH.

-- 
Robert Kern
kern at caltech.edu

"In the fields of hell where the grass grows high
 Are the graves of dreams allowed to die."
  -- Richard Harter



From chrishbarker at home.net  Tue Aug  7 16:26:57 2001
From: chrishbarker at home.net (Chris Barker)
Date: Tue, 07 Aug 2001 13:26:57 -0700
Subject: [Numpy-discussion] fast constructor for arrays from byte data 
 (unpickling?)
References: <20010807195004.19068.qmail@web12606.mail.yahoo.com>
Message-ID: <3B704F10.9457A928@home.net>

Hung Jung Lu wrote:
> Arrays have a method called tostring() which generates
> the binary data. Is there an inverse function to that?
> That is, generating an array from the binary data
> string?

As you suspected, there is such a thing, and it is called (surprise):
fromstring(). It is a function, rather than a method, as it is a
constructor for an array. To get data from a file, you have to get it
into a string first, so I use:

M = fromstring(file.read(),Float)

This makes a rank -1 array. You might have to reshape it.

Note that you end up creating a Python string of the data first, and
then a NumPy array from that. This doesn't really cost that much, but it
can be an issue with huge data sets. I wish there was a fromfile()
function. I may get around to writing it one day.

-Chris







-- 
Christopher Barker,
Ph.D.                                                           
ChrisHBarker at home.net                 ---           ---           ---
http://members.home.net/barkerlohmann ---@@       -----@@       -----@@
                                   ------@@@     ------@@@     ------@@@
Oil Spill Modeling                ------   @    ------   @   ------   @
Water Resources Engineering       -------      ---------     --------    
Coastal and Fluvial Hydrodynamics --------------------------------------
------------------------------------------------------------------------



From hungjunglu at yahoo.com  Tue Aug  7 16:08:54 2001
From: hungjunglu at yahoo.com (Hung Jung Lu)
Date: Tue, 7 Aug 2001 13:08:54 -0700 (PDT)
Subject: [Numpy-discussion] fast constructor for arrays from byte data (unpickling?)
In-Reply-To: <20010807125937.E22660@myrddin.caltech.edu>
Message-ID: <20010807200854.20400.qmail@web12601.mail.yahoo.com>

> > That is, generating an array from the binary data
> > string?
> 
> Numeric.fromstring

Ahhh! Now that seems so obvious. :) (Banging my head.)

thanks!

Hung Jung


__________________________________________________
Do You Yahoo!?
Make international calls for as low as $.04/minute with Yahoo! Messenger
http://phonecard.yahoo.com/



From paul at pfdubois.com  Tue Aug  7 15:56:02 2001
From: paul at pfdubois.com (Paul F. Dubois)
Date: Tue, 7 Aug 2001 12:56:02 -0700
Subject: [Numpy-discussion] fast constructor for arrays from byte data (unpickling?)
In-Reply-To: <20010807195004.19068.qmail@web12606.mail.yahoo.com>
Message-ID: <ADEOIFHFONCLEEPKCACCGELPCLAA.paul@pfdubois.com>

See fromstring() 
>>> import Numeric
>>> x=Numeric.arange(10)
>>> s=x.tostring()
>>> y = Numeric.fromstring(s)
>>> y
array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
>>> 

-----Original Message-----
From: numpy-discussion-admin at lists.sourceforge.net
[mailto:numpy-discussion-admin at lists.sourceforge.net]On Behalf Of Hung
Jung Lu
Sent: Tuesday, August 07, 2001 12:50 PM
To: numpy-discussion at lists.sourceforge.net
Subject: [Numpy-discussion] fast constructor for arrays from byte data
(unpickling?)


Hi,

Arrays have a method called tostring() which generates
the binary data. Is there an inverse function to that?
That is, generating an array from the binary data
string?

For large matrices, pickling/unpickling is a bit too
much overhead (data stored as ASCII character strings
instead of binary data strings.) I know, I am talking
about a factor 4 here. But there is a big difference
between 1 minute loading time and 4 minute loading
time.

I would imagine this is a very common problem/request
for people working with large matrices. And I am sure
hacking the C code to provide another fast constructor
for arrays from binary strings won't be too hard. The
questions is: has anyone already tried it? Is it
already there?

(For the kludge masters: one kludge is of course to
store the binary data on disk, then use cStringIO to
build the pickled file and then unpickle from the
cStringIO. Speed is probably OK since the pickled file
lives on RAM. But that's a kludge. :) )

regards,

Hung Jung


__________________________________________________
Do You Yahoo!?
Make international calls for as low as $.04/minute with Yahoo! Messenger
http://phonecard.yahoo.com/

_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
http://lists.sourceforge.net/lists/listinfo/numpy-discussion




From kern at caltech.edu  Tue Aug  7 16:17:23 2001
From: kern at caltech.edu (Robert Kern)
Date: Tue, 7 Aug 2001 13:17:23 -0700
Subject: [Numpy-discussion] fast constructor for arrays from byte data (unpickling?)
In-Reply-To: <3B704F10.9457A928@home.net>
References: <20010807195004.19068.qmail@web12606.mail.yahoo.com> <3B704F10.9457A928@home.net>
Message-ID: <20010807131723.F22660@myrddin.caltech.edu>

On Tue, Aug 07, 2001 at 01:26:57PM -0700, Chris Barker wrote:

[snip]

> Note that you end up creating a Python string of the data first, and
> then a NumPy array from that. This doesn't really cost that much, but it
> can be an issue with huge data sets. I wish there was a fromfile()
> function. I may get around to writing it one day.

With Python 2.0 or greater, one can use mmap'ed files as arguments to
fromstring.

> -Chris

-- 
Robert Kern
kern at caltech.edu

"In the fields of hell where the grass grows high
 Are the graves of dreams allowed to die."
  -- Richard Harter



From chrishbarker at home.net  Tue Aug  7 18:02:01 2001
From: chrishbarker at home.net (Chris Barker)
Date: Tue, 07 Aug 2001 15:02:01 -0700
Subject: [Numpy-discussion] fast constructor for arrays from byte data 
 (unpickling?)
References: <20010807195004.19068.qmail@web12606.mail.yahoo.com> <3B704F10.9457A928@home.net> <20010807131723.F22660@myrddin.caltech.edu>
Message-ID: <3B706559.597944ED@home.net>

Robert Kern wrote:
> On Tue, Aug 07, 2001 at 01:26:57PM -0700, Chris Barker wrote:
> > I wish there was a fromfile() function.
> 
> With Python 2.0 or greater, one can use mmap'ed files as arguments to
> fromstring.

Can you give me an example of how to use it? I can not get it to work at
all, the docs are pretty sketchy. I can't even get a mmap's file to be
created properly on linux. I havn't tried Windows yet, but I'll need it
to work there too!

Also, mmap does not seem to be supported on the Mac, which is where I am
having memory problems (due to the Mac's lousy memeory management). I'll
ask about it on the MacPython list.

-Chris



-- 
Christopher Barker,
Ph.D.                                                           
ChrisHBarker at home.net                 ---           ---           ---
http://members.home.net/barkerlohmann ---@@       -----@@       -----@@
                                   ------@@@     ------@@@     ------@@@
Oil Spill Modeling                ------   @    ------   @   ------   @
Water Resources Engineering       -------      ---------     --------    
Coastal and Fluvial Hydrodynamics --------------------------------------
------------------------------------------------------------------------



From kern at caltech.edu  Tue Aug  7 17:54:36 2001
From: kern at caltech.edu (Robert Kern)
Date: Tue, 7 Aug 2001 14:54:36 -0700
Subject: [Numpy-discussion] fast constructor for arrays from byte data (unpickling?)
In-Reply-To: <3B706559.597944ED@home.net>
References: <20010807195004.19068.qmail@web12606.mail.yahoo.com> <3B704F10.9457A928@home.net> <20010807131723.F22660@myrddin.caltech.edu> <3B706559.597944ED@home.net>
Message-ID: <20010807145436.G22660@myrddin.caltech.edu>

On Tue, Aug 07, 2001 at 03:02:01PM -0700, Chris Barker wrote:
> Robert Kern wrote:
> > On Tue, Aug 07, 2001 at 01:26:57PM -0700, Chris Barker wrote:
> > > I wish there was a fromfile() function.
> > 
> > With Python 2.0 or greater, one can use mmap'ed files as arguments to
> > fromstring.
> 
> Can you give me an example of how to use it? I can not get it to work at
> all, the docs are pretty sketchy. I can't even get a mmap's file to be
> created properly on linux. I havn't tried Windows yet, but I'll need it
> to work there too!

Yeah, it took me some fiddling, too, before I got it to work. The Windows call
to mmap has slightly different semantics in that second argument, but I think 
that if you want to snarf the whole file, then what I do below should work as
well. The Windows API lets you use 0 to default to the whole file, but that's
not portable.

Python 2.0.1 (#0, Jul  3 2001, 12:36:30) 
[GCC 2.95.4 20010629 (Debian prerelease)] on linux2
Type "copyright", "credits" or "license" for more information.
>>> import Numeric
>>> import mmap
>>> f = open('earth.dat', 'r+')
>>> f.seek(0, 2)  # seek to EOF
>>> size = f.tell()
>>> f.seek(0)  # rewind; possibly not necessary
>>> m = mmap.mmap(f.fileno(), size)
>>> a = Numeric.fromstring(m)
>>> size
548240
>>> a.shape
(137060,)
>>> size / 4
137060
>>> 

> Also, mmap does not seem to be supported on the Mac, which is where I am
> having memory problems (due to the Mac's lousy memeory management). I'll
> ask about it on the MacPython list.

You're probably SOL, there, but I'll wish you good luck anyways.

if-wishes-were-mmap-implementations-ly yours,

-- 
Robert Kern
kern at caltech.edu

"In the fields of hell where the grass grows high
 Are the graves of dreams allowed to die."
  -- Richard Harter



From chrishbarker at home.net  Tue Aug  7 19:33:30 2001
From: chrishbarker at home.net (Chris Barker)
Date: Tue, 07 Aug 2001 16:33:30 -0700
Subject: [Numpy-discussion] fast constructor for arrays from byte data 
 (unpickling?)
References: <20010807195004.19068.qmail@web12606.mail.yahoo.com> <3B704F10.9457A928@home.net> <20010807131723.F22660@myrddin.caltech.edu> <3B706559.597944ED@home.net> <20010807145436.G22660@myrddin.caltech.edu>
Message-ID: <3B707ACA.B5FE8E55@home.net>

Robert Kern wrote:

> Yeah, it took me some fiddling, too, before I got it to work. The Windows call
> to mmap has slightly different semantics in that second argument, but I think
> that if you want to snarf the whole file, then what I do below should work as
> well. The Windows API lets you use 0 to default to the whole file, but that's
> not portable.

[code snipped]

no, the "0 argument does not seem to work on Unix. It seems odd to not
have a way to just snarf the whole file automatically, but so be it

Thanks, that works, but now I am wondering: what I want is a fast and
memory efficient way to get the contents of a fjile into a NujmPy array,
this sure doesn't look any better than:

a = fromstring(file.read()))

thanks anyway,

-Chris


-- 
Christopher Barker,
Ph.D.                                                           
ChrisHBarker at home.net                 ---           ---           ---
http://members.home.net/barkerlohmann ---@@       -----@@       -----@@
                                   ------@@@     ------@@@     ------@@@
Oil Spill Modeling                ------   @    ------   @   ------   @
Water Resources Engineering       -------      ---------     --------    
Coastal and Fluvial Hydrodynamics --------------------------------------
------------------------------------------------------------------------



From kern at caltech.edu  Tue Aug  7 19:28:18 2001
From: kern at caltech.edu (Robert Kern)
Date: Tue, 7 Aug 2001 16:28:18 -0700
Subject: [Numpy-discussion] fast constructor for arrays from byte data (unpickling?)
In-Reply-To: <3B707ACA.B5FE8E55@home.net>
References: <20010807195004.19068.qmail@web12606.mail.yahoo.com> <3B704F10.9457A928@home.net> <20010807131723.F22660@myrddin.caltech.edu> <3B706559.597944ED@home.net> <20010807145436.G22660@myrddin.caltech.edu> <3B707ACA.B5FE8E55@home.net>
Message-ID: <20010807162818.H22660@myrddin.caltech.edu>

On Tue, Aug 07, 2001 at 04:33:30PM -0700, Chris Barker wrote:

[snip]

> Thanks, that works, but now I am wondering: what I want is a fast and
> memory efficient way to get the contents of a fjile into a NujmPy array,
> this sure doesn't look any better than:
> 
> a = fromstring(file.read()))

Depends on how large the file is. file.read() creates a temporary string the
size of the file. That string isn't freed until fromstring() finishes and
returns the array object. For a brief time, both the string and the array have
duplicates of the same data taking up space in memory.

I don't know the details of mmap, so it's certainly possible that the only way
that fromstring knows how to access the data is to pull all of it into memory 
first, thus recreating the problem. Alas.

> thanks anyway,
> 
> -Chris

-- 
Robert Kern
kern at caltech.edu

"In the fields of hell where the grass grows high
 Are the graves of dreams allowed to die."
  -- Richard Harter



From serge at rohan.sdsu.edu  Wed Aug  8 00:38:30 2001
From: serge at rohan.sdsu.edu (Serge Rey)
Date: Wed, 8 Aug 2001 14:38:30 +1000
Subject: [Numpy-discussion] permutation and array subsets
Message-ID: <20010808143830.E314@typhoon.sdsu.edu>

i'm trying to use the results of permutation (from RandomArray.py) to
reorder the rows of an array:

>>> r=arange(20)
>>> id=permutation(20)
>>> y=r[id]
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  IndexError: invalid index
  >>> 

i can use a for loop to permutate the ordering of r, but that has a
speed hit associated with it.  can anyone offer a suggestion for a way
to do this more efficiently? 

thanks,
serge
-- 
Sergio J. Rey	http://typhoon.sdsu.edu/rey.html



From ransom at cfa.harvard.edu  Wed Aug  8 00:46:41 2001
From: ransom at cfa.harvard.edu (Scott Ransom)
Date: Wed, 08 Aug 2001 00:46:41 -0400
Subject: [Numpy-discussion] permutation and array subsets
References: <20010808143830.E314@typhoon.sdsu.edu>
Message-ID: <3B70C431.D1F833C6@cfa.harvard.edu>

Serge Rey wrote:
>
> i'm trying to use the results of permutation (from RandomArray.py) to
> reorder the rows of an array:
> 
> >>> r=arange(20)
> >>> id=permutation(20)
> >>> y=r[id]

Try:

y = take(r, permutation(len(r)))

Scott

-- 
Scott M. Ransom                   Address:  Harvard-Smithsonian CfA
Phone:  (617) 496-7908                      60 Garden St.  MS 10 
email:  ransom at cfa.harvard.edu              Cambridge, MA  02138
GPG Fingerprint: 06A9 9553 78BE 16DB 407B  FFCA 9BFA B6FF FFD3 2989



From ransom at cfa.harvard.edu  Wed Aug  8 00:54:49 2001
From: ransom at cfa.harvard.edu (Scott Ransom)
Date: Wed, 08 Aug 2001 00:54:49 -0400
Subject: [Numpy-discussion] swig and Numeric
References: <01080710213100.17772@almanac>
Message-ID: <3B70C619.4FE32847@cfa.harvard.edu>

"Paul F. Dubois" wrote:
> 
> Does someone have an example of using SWIG to connect Numeric arrays to C code
> that expects double* arguments?

Hi Paul,

I have a fairly comprehensive set of typedefs (including 1-D and 2-D
array input and output) for interfacing Numeric arrays here:

ftp://cfa-ftp.harvard.edu/pub/ransom/numpy.i

As an example, say you have a function with a prototype:

int foo(double *array, int len_array);

You would simply make a ".i" file with the following:

======================
%module foo
%include numpy.i 

%apply double* IN_1D_DOUBLE { double *array };
int bar(double *array, int len_array);

=======================

Run swig on it, compile it, and import it.

Hope this helps,

Scott

-- 
Scott M. Ransom                   Address:  Harvard-Smithsonian CfA
Phone:  (617) 496-7908                      60 Garden St.  MS 10 
email:  ransom at cfa.harvard.edu              Cambridge, MA  02138
GPG Fingerprint: 06A9 9553 78BE 16DB 407B  FFCA 9BFA B6FF FFD3 2989



From chrishbarker at home.net  Wed Aug  8 16:37:02 2001
From: chrishbarker at home.net (Chris Barker)
Date: Wed, 08 Aug 2001 13:37:02 -0700
Subject: [Numpy-discussion] fast constructor for arrays from byte data 
 (unpickling?)
References: <20010807195004.19068.qmail@web12606.mail.yahoo.com> <3B704F10.9457A928@home.net> <20010807131723.F22660@myrddin.caltech.edu> <3B706559.597944ED@home.net> <20010807145436.G22660@myrddin.caltech.edu> <3B707ACA.B5FE8E55@home.net> <20010807162818.H22660@myrddin.caltech.edu>
Message-ID: <3B71A2EE.EC3BBE46@home.net>

Robert Kern wrote:

> > Thanks, that works, but now I am wondering: what I want is a fast and
> > memory efficient way to get the contents of a file into a NujmPy array,
> > this sure doesn't look any better than:
> >
> > a = fromstring(file.read()))
> 
> Depends on how large the file is. file.read() creates a temporary string the
> size of the file. That string isn't freed until fromstring() finishes and
> returns the array object. For a brief time, both the string and the array have
> duplicates of the same data taking up space in memory.

Exactly. Also is the memory used for hte string guarranteed to be freed
right away? I have no idea how Python internals work.

Anyway, that's why I want a "fromfile()" method, like the one inthe
library array module.
 
> I don't know the details of mmap, so it's certainly possible that the only way
> that fromstring knows how to access the data is to pull all of it into memory
> first, thus recreating the problem. Alas.

I don't understand mmap at all. From the name, it sounds like the entire
contents of the file is mapped into memory, so the memory would get used
as soon as you set it up. If anyone knows, I'd like to hear...


-Chris





-- 
Christopher Barker,
Ph.D.                                                           
ChrisHBarker at home.net                 ---           ---           ---
http://members.home.net/barkerlohmann ---@@       -----@@       -----@@
                                   ------@@@     ------@@@     ------@@@
Oil Spill Modeling                ------   @    ------   @   ------   @
Water Resources Engineering       -------      ---------     --------    
Coastal and Fluvial Hydrodynamics --------------------------------------
------------------------------------------------------------------------



From phil at geog.ubc.ca  Wed Aug  8 16:56:56 2001
From: phil at geog.ubc.ca (Phil Austin)
Date: Wed, 8 Aug 2001 13:56:56 -0700 (PDT)
Subject: [Numpy-discussion] mmap (was: fast constructor for arrays from byte data)
Message-ID: <15217.42904.327270.469067@curlew.geog.ubc.ca>

Chris Barker writes:

> I don't understand mmap at all. From the name, it sounds like the entire
> contents of the file is mapped into memory, so the memory would get used
> as soon as you set it up. If anyone knows, I'd like to hear...

This might be more than you need to know, but there's a 
copy of the article:

"Why aren't you using mmap() yet?"
by Kevin Sheehan at:

http://thrush.eos.ubc.ca/users/phil/mmap/mmap.pdf (note reversed page
order)

and a Solaris/SGI email debate in mhonarc format:

http://thrush.eos.ubc.ca/users/phil/mmap/threads.html

bottom line: you really need a vm implementation that supports
madvise() to get the maximum benefit from mmap.  (Linux 2.4
might have madvise, I haven't tried it out yet -- 2.2 didn't).

The way I used to use mmap was through a numpy array class that was
initiated from a pointer to a mmap'd file.  This made reading/writing
to the file as simple as indexing into the numpy array.  I haven't
been back to reimplement that class on linux since I left solaris,
though.  You can also get some of the benefits of mmap via netcdf,
which will use mmap if it exists.

Regards, Phil



From kern at caltech.edu  Wed Aug  8 17:23:10 2001
From: kern at caltech.edu (Robert Kern)
Date: Wed, 8 Aug 2001 14:23:10 -0700
Subject: [Numpy-discussion] mmap (was: fast constructor for arrays from byte data)
In-Reply-To: <15217.42904.327270.469067@curlew.geog.ubc.ca>
References: <15217.42904.327270.469067@curlew.geog.ubc.ca>
Message-ID: <20010808142310.A518@myrddin.caltech.edu>

On Wed, Aug 08, 2001 at 01:56:56PM -0700, Phil Austin wrote:

[snip pointers to info]

> bottom line: you really need a vm implementation that supports
> madvise() to get the maximum benefit from mmap.  (Linux 2.4
> might have madvise, I haven't tried it out yet -- 2.2 didn't).

FWIW, there appears to be a madvise(2) system call in Linux 2.4.7 (in 
mm/filemap.c for those of you with kernel source trees lying around), but I
can't find it in the headers anywhere. Sigh. I don't know if madvise is supposed
to be internal or not, though.

-- 
Robert Kern
kern at caltech.edu

"In the fields of hell where the grass grows high
 Are the graves of dreams allowed to die."
  -- Richard Harter



From kern at caltech.edu  Wed Aug  8 17:24:14 2001
From: kern at caltech.edu (Robert Kern)
Date: Wed, 8 Aug 2001 14:24:14 -0700
Subject: [Numpy-discussion] fast constructor for arrays from byte data (unpickling?)
In-Reply-To: <3B71A2EE.EC3BBE46@home.net>
References: <20010807195004.19068.qmail@web12606.mail.yahoo.com> <3B704F10.9457A928@home.net> <20010807131723.F22660@myrddin.caltech.edu> <3B706559.597944ED@home.net> <20010807145436.G22660@myrddin.caltech.edu> <3B707ACA.B5FE8E55@home.net> <20010807162818.H22660@myrddin.caltech.edu> <3B71A2EE.EC3BBE46@home.net>
Message-ID: <20010808142414.B518@myrddin.caltech.edu>

On Wed, Aug 08, 2001 at 01:37:02PM -0700, Chris Barker wrote:
> Robert Kern wrote:
> 
> > > Thanks, that works, but now I am wondering: what I want is a fast and
> > > memory efficient way to get the contents of a file into a NujmPy array,
> > > this sure doesn't look any better than:
> > >
> > > a = fromstring(file.read()))
> > 
> > Depends on how large the file is. file.read() creates a temporary string the
> > size of the file. That string isn't freed until fromstring() finishes and
> > returns the array object. For a brief time, both the string and the array have
> > duplicates of the same data taking up space in memory.
> 
> Exactly. Also is the memory used for hte string guarranteed to be freed
> right away? I have no idea how Python internals work.

Once fromstring returns, the string's refcount is 0 and should be freed just
about right away. I'm not sure, but I don't think the new gc will affect that.

> Anyway, that's why I want a "fromfile()" method, like the one inthe
> library array module.

Yes, I agree, it would be nice to have.

> > I don't know the details of mmap, so it's certainly possible that the only way
> > that fromstring knows how to access the data is to pull all of it into memory
> > first, thus recreating the problem. Alas.
> 
> I don't understand mmap at all. From the name, it sounds like the entire
> contents of the file is mapped into memory, so the memory would get used
> as soon as you set it up. If anyone knows, I'd like to hear...

Performing a very cursory, very non-scientific study, I created a 110M file,
mmap'ed it, then made a string from some of it. I'm using 64MB of RAM, 128MB
swap partition on a Linux 2.4.7 machine. According to top(1), the memory use 
didn't jump up until I made the string. Also, given that the call to mmap
returned almost instantaneously, I'd say that mmap doesn't pull the whole file
into memory when the object is created (one could just read the file for that).

I don't know what test to perform to see whether the fromstring() constructor
uses double memory, but my guess would be that memcpy won't pull in the whole
file before copying. OTOH, the accessed portions of the mmap'ed file may be kept
in memory.

Does anyone know the details on mmap? I'm shooting in the dark, here.
<reads Phil Austin's e-mail>  Ooh, nice.

> -Chris

-- 
Robert Kern
kern at caltech.edu

"In the fields of hell where the grass grows high
 Are the graves of dreams allowed to die."
  -- Richard Harter



From tariq_rashid at lineone.net  Wed Aug  8 18:17:48 2001
From: tariq_rashid at lineone.net (Tariq Rashid)
Date: Wed, 8 Aug 2001 23:17:48 +0100
Subject: [Numpy-discussion] swig and C/C++
In-Reply-To: <E15UCBk-0004VJ-00@usw-sf-list1.sourceforge.net>
Message-ID: <NFBBLBJKHCHBHGGGHJDOAEMOCAAA.tariq_rashid@lineone.net>

in fact ... even simpler ... can anyone point me to an example of passing
Numeric arrays to and from C ... the swig examples are fine for integers /
floats ... but as i'm a new to this stuff I need a bit if guidance with
Numeric arrays ...

any ideas / pointers would be very gratefully receieved... i'm working on
wavelet stuff which i want to give back to the Numeric community....

thanks

tariq		(tariq_rashid at lineone.net)

----------------------------

Today's Topics:

   1. NumTut (=?ks_c_5601-1987?B?wMyx1LrA?=)
   2. swig and Numeric (Paul F. Dubois)


From dubois1 at llnl.gov  Tue Aug  7 13:20:19 2001
From: dubois1 at llnl.gov (Paul F. Dubois)
Date: Tue, 7 Aug 2001 10:20:19 -0700
Subject: [Numpy-discussion] swig and Numeric
Message-ID: <mailman.27.1490332390.18468.numpy-discussion@python.org>

Does someone have an example of using SWIG to connect Numeric arrays to C=
 code
that expects double* arguments?





From just at letterror.com  Fri Aug 10 03:29:42 2001
From: just at letterror.com (Just van Rossum)
Date: Fri, 10 Aug 2001 09:29:42 +0200
Subject: [Numpy-discussion] NumPy installation instructions for Windows
Message-ID: <20010810092947-r01010700-1f9414d2-0910-010c@10.0.0.21>

I develop a package that depends on NumPy, and I would like to add a line or two
to my documentation about how to install NumPy under Windows -- just the basic
set of modules, not the extensions.

I understand users should grab the .exe archive. The FAQ only mentions "The .exe
files are  Windows installers of the sort that you are used to". Does this mean
everything is explained and done by the installer itself, or is there some
additional action the user should perform? Thanks!

(I have no Windows box here so I can't see for myself...)

Just



From paul at pfdubois.com  Fri Aug 10 12:04:11 2001
From: paul at pfdubois.com (Paul F. Dubois)
Date: Fri, 10 Aug 2001 09:04:11 -0700
Subject: [Numpy-discussion] NumPy installation instructions for Windows
In-Reply-To: <20010810092947-r01010700-1f9414d2-0910-010c@10.0.0.21>
Message-ID: <ADEOIFHFONCLEEPKCACCEEOBCLAA.paul@pfdubois.com>

The windows installer does everything required to import Numeric.

-----Original Message-----
From: numpy-discussion-admin at lists.sourceforge.net
[mailto:numpy-discussion-admin at lists.sourceforge.net]On Behalf Of Just
van Rossum
Sent: Friday, August 10, 2001 12:30 AM
To: numpy-discussion at lists.sourceforge.net
Subject: [Numpy-discussion] NumPy installation instructions for Windows


I develop a package that depends on NumPy, and I would like to add a line or
two
to my documentation about how to install NumPy under Windows -- just the
basic
set of modules, not the extensions.

I understand users should grab the .exe archive. The FAQ only mentions "The
.exe
files are  Windows installers of the sort that you are used to". Does this
mean
everything is explained and done by the installer itself, or is there some
additional action the user should perform? Thanks!

(I have no Windows box here so I can't see for myself...)

Just

_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
http://lists.sourceforge.net/lists/listinfo/numpy-discussion




From edcjones at erols.com  Fri Aug 10 17:16:04 2001
From: edcjones at erols.com (Edward C. Jones)
Date: Fri, 10 Aug 2001 17:16:04 -0400
Subject: [Numpy-discussion] SWIG / Numeric core dump
Message-ID: <3B744F14.9080104@erols.com>

The following Python / SWIG / Numeric program dumps core.
I use RedHat Linux 7.1 on a PC. Python 2.1, Numeric 20.1.0,
SWIG 1.1 build 883, gcc 2.96. What is the problem?

The output looks like:

> doit.py
called import_array
Calling PyArray_FromDims
Segmentation fault (core dumped)
>

The program files are:
----------------------------------------------
doit.py:

#! /usr/bin/python

import Numeric, spam

spam.funct()
print 'spam.funct completed'

----------------------------------------------
spam.h:

#include "Python.h"
#include "/usr/include/python2.1/Numeric/arrayobject.h"

void funct(void);

----------------------------------------------
spam.i

%module spam

%{
#include "spam.h"
%}  

%init %{
    import_array();
    printf("called import_array\n");
%}

void funct(void);

----------------------------------------------
spammodule.c

#include "spam.h"

void funct(void)
{
    PyArrayObject *pao;
    int dims[2];

    dims[0] = 100;
    dims[1] = 200;
    printf("Calling PyArray_FromDims\n");
    pao = (PyArrayObject*) PyArray_FromDims(2, dims, PyArray_UBYTE);
    printf("Completed PyArray_FromDims\n");
    Py_DECREF(pao);
}

----------------------------------------------
swigit:

swig -python spam.i
gcc -c -Wall spammodule.c spam_wrap.c -I/usr/include/python2.1
ld -shared spammodule.o spam_wrap.o -o spam.so





From vanandel at atd.ucar.edu  Fri Aug 10 17:47:12 2001
From: vanandel at atd.ucar.edu (Joe Van Andel)
Date: Fri, 10 Aug 2001 15:47:12 -0600
Subject: [Numpy-discussion] SWIG / Numeric core dump
References: <3B744F14.9080104@erols.com>
Message-ID: <3B745660.476E49EE@atd.ucar.edu>

The best way of finding core dumps with a Python extension is to compile
the extension source with '-g', and then follow these steps.
(You may also have to compile the Numeric Extensions with '-g', to trace
into them).

% gdb /usr/bin/python2.1
(gdb) br _PyImport_LoadDynamicModule 
(gdb) cont   # repeat until your extension is loaded


(gdb) finish # to load your extension
(gdb) br wrap_myfunction
(gdb) disable 1   # don't want to break for more modules being loaded
(gdb) continue


Note: you can't set a breakpoint in your code until your module has been
loaded.


-- 
Joe VanAndel  	          
National Center for Atmospheric Research
http://www.atd.ucar.edu/~vanandel/
Internet: vanandel at ucar.edu



From edcjones at erols.com  Sat Aug 11 21:56:22 2001
From: edcjones at erols.com (Edward C. Jones)
Date: Sat, 11 Aug 2001 21:56:22 -0400
Subject: [Numpy-discussion] SWIG + Numeric 20.1.0 = core dump: the fix
Message-ID: <3B75E246.4030505@erols.com>

If
    Numeric and SWIG are used together in creating a Python extension
  and
    there are C/C++ files as part of the extension
then
  Numeric 20.1.0 will probably break your code.

This happens because changes were made to arrayobject.h between
Numeric 20.0.0 and 20.1.0.

The changes that need to be made in your code are explained
by Konrad Hinsen:
(http://www.geocrawler.com/archives/3/1329/2001/5/0/5718231/)

     FROM: Konrad Hinsen
     DATE: 05/09/2001 09:12:25
     SUBJECT:  [Numpy-discussion] Modified header files

     Recently we had a discussion about how to use NumPy arrays from
     extension modules with multiple source files, on various
     platforms.  The modified header files that are attached to this
     message provide a (hopefully!) general solution. In fact, I
     propose to make them part of the official distribution, unless
     there are objections.

     If used like before, these header files give exactly the same
     result as the ones in NumPy 20.0.0. However, they permit to define
     the name of the C API pointer array and make it globally visible.
     Under the condition that the chosen name is unique, this should
     not create problems under any platform, no matter if static or
     dynamic linking is used.

     To use NumPy features in multiple source file extension modules,
     you have to write

        #define PY_ARRAY_UNIQUE_SYMBOL PyArrayXXX
        #include "Numeric/arrayobject.h"

     in the main source file (the one that contains the init function)
     and
     
        #define PY_ARRAY_UNIQUE_SYMBOL PyArrayXXX
        #define NO_IMPORT_ARRAY
        #include "Numeric/arrayobject.h"

     in all the others. The symbol you choose instead of PyArrayXXX
     should contain both the name of the imported module (array) and
     the name of the importing module (whatever your module is called)
     in order to be unique with a reasonably high probability.

     The same applies to the Ufunc module, just replace "array" by
     "ufunc" in the example. I have also applied the "static"
     correction to the Ufunc header file, there is no reason not to do
     it.

     Konrad.
     -- 
     -----------------------------------------------------------------------
     Konrad Hinsen                            | E-Mail:
     hinsen at NO-SPAM.cnrs-orleans.fr
     Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
     Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
     45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
     France                                   | Nederlands/Francais
     -----------------------------------------------------------------------

I do this by modifying one header file that is included in all
the ".c" and ".i" files. In the following example the header file
is "spam.h".

------------------------------------------------------------
doit.py:

#! /usr/bin/python

import Numeric, spam

spam.funct()
print 'spam.funct completed'
------------------------------------------------------------
spam.h:

#include "Python.h"
#define PY_ARRAY_UNIQUE_SYMBOL Py_Array_API_spam
#ifndef SWIG_FILE_WITH_INIT
  #define NO_IMPORT_ARRAY
#endif

#include "/usr/include/python2.1/Numeric/arrayobject.h"
void funct(void);
------------------------------------------------------------
spam.i:

%module spam

%{
#define SWIG_FILE_WITH_INIT
#include "spam.h"
%}  

%init %{
import_array();
printf("called import_array\n");
%}

void funct(void);
------------------------------------------------------------
spammodule.c

#include "spam.h"

void funct(void)
{
    PyArrayObject *pao;
    int dims[2];

    dims[0] = 100;
    dims[1] = 200;
    printf("Calling PyArray_FromDims\n");
    pao = (PyArrayObject*) PyArray_FromDims(2, dims, PyArray_UBYTE);
    printf("Completed PyArray_FromDims\n");
    Py_DECREF(pao);
}
------------------------------------------------------------
compile script:

swig -python spam.i
gcc -c -Wall spammodule.c spam_wrap.c -I/usr/include/python2.1
ld -shared spammodule.o spam_wrap.o -o spam.so
------------------------------------------------------------

I use RedHat 7.1 Linux on a PC, Python 2.1, Numeric 20.1.0,
SWIG 1.1 build 883, and gcc 2.96. Anyone is welcome to use
the above example in documentation.





From huaiyu_zhu at yahoo.com  Fri Aug 17 02:30:20 2001
From: huaiyu_zhu at yahoo.com (Huaiyu Zhu)
Date: Thu, 16 Aug 2001 23:30:20 -0700 (PDT)
Subject: [Numpy-discussion] Subarray with with arbitrary index?
Message-ID: <Pine.LNX.4.10.10108162312270.1077-100000@mars.localdomain>

Hi,

Is it possible to assign to a subarray with arbitrary index?

Suppose I have three arrays

a = arange(8)
b = array([2, 3, 5])
c = arange(3)+100

I want a function f, such that calling  f(a, b, c) would change a to

[0 1 100 101 4 102 6 7]

This would be useful in certain applications that would otherwise
require sparse matrices.   



Huaiyu Zhu




From jjl at pobox.com  Fri Aug 17 09:36:26 2001
From: jjl at pobox.com (John J. Lee)
Date: Fri, 17 Aug 2001 14:36:26 +0100 (BST)
Subject: [Numpy-discussion] Subarray with with arbitrary index?
In-Reply-To: <Pine.LNX.4.10.10108162312270.1077-100000@mars.localdomain>
Message-ID: <Pine.SOL.4.30.0108171427230.15545-100000@mimosa.csv.warwick.ac.uk>

On Thu, 16 Aug 2001, Huaiyu Zhu wrote:

> Hi,
>
> Is it possible to assign to a subarray with arbitrary index?
>
> Suppose I have three arrays
>
> a = arange(8)
> b = array([2, 3, 5])
> c = arange(3)+100
>
> I want a function f, such that calling  f(a, b, c) would change a to
>
> [0 1 100 101 4 102 6 7]

f = Numeric.put
f(a, b, c)

put used to be in Python, but it's been in C since some release 17.x.x, I
think.

I have a sinking feeling that I must have missed something (no interpreter
here to check it works)...

BTW, a week ago I noticed that I had reinvented the wheel in rewriting, in
an uglier and less efficient form, Numeric.allclose (hope I got the name
right).  As far as I can see, it isn't listed in the manual.  Did I miss
it?  All it would need is the docstring copying over.


John




From paul at pfdubois.com  Fri Aug 17 11:23:28 2001
From: paul at pfdubois.com (Paul F. Dubois)
Date: Fri, 17 Aug 2001 08:23:28 -0700
Subject: [Numpy-discussion] Subarray with with arbitrary index?
In-Reply-To: <Pine.SOL.4.30.0108171427230.15545-100000@mimosa.csv.warwick.ac.uk>
Message-ID: <ADEOIFHFONCLEEPKCACCKEBNCMAA.paul@pfdubois.com>

John is right:
>>> a=Numeric.arange(8)
>>> b=Numeric.array([2,3,5])
>>> c=Numeric.arange(3)+100
>>> Numeric.put(a,b,c)
>>> print a
[  0   1 100 101   4 102   6   7]

Thanks for pointing out that I had left allclose out of the Numeric part of
the manual. I did it in the MA part and then forgot. I'm fixing it now. BTW:
There are changenotes at source forge that are sometimes ahead of the
manual.

-----Original Message-----
From: numpy-discussion-admin at lists.sourceforge.net
[mailto:numpy-discussion-admin at lists.sourceforge.net]On Behalf Of John
J. Lee
Sent: Friday, August 17, 2001 6:36 AM
To: Huaiyu Zhu
Cc: numpy-discussion at lists.sourceforge.net
Subject: Re: [Numpy-discussion] Subarray with with arbitrary index?


On Thu, 16 Aug 2001, Huaiyu Zhu wrote:

> Hi,
>
> Is it possible to assign to a subarray with arbitrary index?
>
> Suppose I have three arrays
>
> a = arange(8)
> b = array([2, 3, 5])
> c = arange(3)+100
>
> I want a function f, such that calling  f(a, b, c) would change a to
>
> [0 1 100 101 4 102 6 7]

f = Numeric.put
f(a, b, c)

put used to be in Python, but it's been in C since some release 17.x.x, I
think.

I have a sinking feeling that I must have missed something (no interpreter
here to check it works)...

BTW, a week ago I noticed that I had reinvented the wheel in rewriting, in
an uglier and less efficient form, Numeric.allclose (hope I got the name
right).  As far as I can see, it isn't listed in the manual.  Did I miss
it?  All it would need is the docstring copying over.


John


_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
http://lists.sourceforge.net/lists/listinfo/numpy-discussion




From paul at pfdubois.com  Sat Aug 18 12:29:23 2001
From: paul at pfdubois.com (Paul F. Dubois)
Date: Sat, 18 Aug 2001 09:29:23 -0700
Subject: [Numpy-discussion] Subarray with with arbitrary index?
In-Reply-To: <Pine.LNX.4.10.10108172206320.972-100000@mars.localdomain>
Message-ID: <000001c12802$f12a2040$0301a8c0@plstn1.sfba.home.com>

I'm afraid that you have to do put using one-d indices. But you do *not*
have to try to ravel the source. I.e., the first arg is just the name of
the array.
>>> from Numeric import *
>>> x=array([[1,2,3],[4,5,6]])
>>> x
array([[1, 2, 3],
       [4, 5, 6]])
>>> put(x,[0,4],[100,200])
>>> x
array([[100,   2,   3],
       [  4, 200,   6]])
>>>
-----Original Message-----
From: Huaiyu Zhu [mailto:huaiyu_zhu at yahoo.com] 
Sent: Friday, August 17, 2001 11:36 PM
To: Paul F. Dubois
Cc: John J. Lee
Subject: RE: [Numpy-discussion] Subarray with with arbitrary index?

Thanks, John and Paul.  That is what I was looking for.

It did not occur to me to look for verbs put and take, rather than words
line sub- index, slice and so on.  Maybe puting some of these words in
the manual could help people doing a search?

Now that this made the most costly part of my program about seven times
faster, other problems become more prominent.  One of such question is:
How do we do it on more than one axis?

Suppose a is a 2d array. Then put(a[1,:], b, c) works, but put(a[:,1],
b,
c) complains about the first argument not a continuous array.  Doing
transpose does not help.  So do I guess it right that this is
implemented
only in the representation of a linear array?

If so, there would be no hope of using put(a, ([2, 4], [1,2]), v) or
even
more exotic ones like using += on an arbitray subgrid?

Regards,

Huaiyu

On Fri, 17 Aug 2001, Paul F. Dubois wrote:

> John is right:
> >>> a=Numeric.arange(8)
> >>> b=Numeric.array([2,3,5])
> >>> c=Numeric.arange(3)+100
> >>> Numeric.put(a,b,c)
> >>> print a
> [  0   1 100 101   4 102   6   7]
> 
> Thanks for pointing out that I had left allclose out of the Numeric
part of
> the manual. I did it in the MA part and then forgot. I'm fixing it
now. BTW:
> There are changenotes at source forge that are sometimes ahead of the
> manual.
> 
> -----Original Message-----
> From: numpy-discussion-admin at lists.sourceforge.net
> [mailto:numpy-discussion-admin at lists.sourceforge.net]On Behalf Of John
> J. Lee
> Sent: Friday, August 17, 2001 6:36 AM
> To: Huaiyu Zhu
> Cc: numpy-discussion at lists.sourceforge.net
> Subject: Re: [Numpy-discussion] Subarray with with arbitrary index?
> 
> 
> On Thu, 16 Aug 2001, Huaiyu Zhu wrote:
> 
> > Hi,
> >
> > Is it possible to assign to a subarray with arbitrary index?
> >
> > Suppose I have three arrays
> >
> > a = arange(8)
> > b = array([2, 3, 5])
> > c = arange(3)+100
> >
> > I want a function f, such that calling  f(a, b, c) would change a to
> >
> > [0 1 100 101 4 102 6 7]
> 
> f = Numeric.put
> f(a, b, c)
> 
> put used to be in Python, but it's been in C since some release
17.x.x, I
> think.
> 
> I have a sinking feeling that I must have missed something (no
interpreter
> here to check it works)...
> 
> BTW, a week ago I noticed that I had reinvented the wheel in
rewriting, in
> an uglier and less efficient form, Numeric.allclose (hope I got the
name
> right).  As far as I can see, it isn't listed in the manual.  Did I
miss
> it?  All it would need is the docstring copying over.
> 
> 
> John
> 
> 
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> http://lists.sourceforge.net/lists/listinfo/numpy-discussion
> 





From paul at pfdubois.com  Sat Aug 18 12:33:08 2001
From: paul at pfdubois.com (Paul F. Dubois)
Date: Sat, 18 Aug 2001 09:33:08 -0700
Subject: [Numpy-discussion] MA patch submitter, please upload file
Message-ID: <000101c12803$76f70e40$0301a8c0@plstn1.sfba.home.com>

Would the person who submitted the patch concerning MA binary ops,
please upload the file to the patch, or email it to me? You have
identified a real problem whose answer is not obvious so I'd like to see
your idea of how to fix it.
 
If you log in to SourceForge before submitting bugs and patches then I
can know who you are to thank you or make followup inquiries like this
one.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20010818/76388cce/attachment-0001.html>

From ehagemann at home.com  Sat Aug 18 20:21:27 2001
From: ehagemann at home.com (Eric Hagemann)
Date: Sat, 18 Aug 2001 20:21:27 -0400
Subject: [Numpy-discussion] For the next round on the manual
Message-ID: <006801c12844$e384e870$6401a8c0@cc911046a>

Type-o in the manual

    Check the definition for the 'convolve' function.  Manual tells of mode=0 as default when code and experiment point to mode=2 being default

    No biggie -- but it might be nice to correct if anybody is editing the manual any time soon

Cheers
Eric

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20010818/92c6568d/attachment-0001.html>

From karshi.hasanov at utoronto.ca  Sun Aug 19 14:17:36 2001
From: karshi.hasanov at utoronto.ca (Karshi Hasanov)
Date: Sun, 19 Aug 2001 14:17:36 -0400
Subject: [Numpy-discussion] Matlab_NumPy_Array?
Message-ID: <01081914173600.00854@cr322577-a>

Hi all,

 I  don't understand how to conver arrays from Matlab to Numpy. Here what I 
did: 1) Converted Matlab A(i,j,k) to 'A.dat' binary file
      2) file_open=open('~/A.dat', 'rb')
      3) read_file=file_open.read()
      4) A_string=fromstring(read_file, 'f')
      5) A=reshape(A_string, (i,j,k))
where i,j,k are size of A[i,j,k]

I am not getting the right. 
What is the best/right way of convering arrays?
Thanks



From gball at cfa.harvard.edu  Wed Aug 22 19:04:09 2001
From: gball at cfa.harvard.edu (Greg Ball)
Date: Wed, 22 Aug 2001 19:04:09 -0400 (EDT)
Subject: [Numpy-discussion] Matlab_NumPy_Array?
In-Reply-To: <01081914173600.00854@cr322577-a>
Message-ID: <Pine.OSF.4.10.10108221901390.8371-100000@surya.harvard.edu>

Try replacing step 4 with

A_string=fromstring(read_file, 'd')

'f' is for 32 bit floats, 'd' is for 64 bit floats.
Matlab uses 64 bit (double) floats.

HTH

--Greg Ball



On Sun, 19 Aug 2001, Karshi Hasanov wrote:

> 
> Hi all,
> 
>  I  don't understand how to conver arrays from Matlab to Numpy. Here what I 
> did: 1) Converted Matlab A(i,j,k) to 'A.dat' binary file
>       2) file_open=open('~/A.dat', 'rb')
>       3) read_file=file_open.read()
>       4) A_string=fromstring(read_file, 'f')
>       5) A=reshape(A_string, (i,j,k))
> where i,j,k are size of A[i,j,k]
> 
> I am not getting the right. 
> What is the best/right way of convering arrays?
> Thanks
> 
> 
> 
> 




From jsw at cdc.noaa.gov  Thu Aug 23 11:22:17 2001
From: jsw at cdc.noaa.gov (Jeff Whitaker)
Date: Thu, 23 Aug 2001 09:22:17 -0600 (MDT)
Subject: [Numpy-discussion] patch to speed up LinearAlgebra module
Message-ID: <Pine.OSX.4.33.0108230915100.23509-100000@crdmac10.cdc.noaa.gov>

I've uploaded a patch to LinearAlgebra.py and lapack_litemodule.c that
speeds up some computations (svd, linear least-squares, and hermitian
eigenanalysis) by 3-5 times when linking with external (preferably
optimized) blas and lapack libs.  The patch replaces calls to dgesvd,
dgelss and dsyev with the newer, faster lapack routines dgesdd, dgelsd and
dsyevd.  Here are my timing results before and after (G4 350mhz powermac,
optimized ATLAS blas lib and lapack 3.0 from netlib.org):

Before:

svd of 1000X1000 matrix takes 609.19
generalized inverse of 1000X1000 matrix takes 744.36
linear least-squares solution of 1000X1000 matrix takes
451.68
eigenvectors of 1000X1000 symmetric matrix takes 210.08

After:

svd of 1000X1000 matrix takes 142.55
generalized inverse of 1000X1000 matrix takes 237.08
linear least-squares solution of 1000X1000 matrix takes
81.91
eigenvectors of 1000X1000 symmetric matrix takes 56.23


Note that since these newer lapack routines are not in lapack_lite, you
must link external blas and lapack3 libs.

-Jeff

-- 
Jeffrey S. Whitaker         Phone  : (303)497-6313
Meteorologist               FAX    : (303)497-6449
NOAA/OAR/CDC  R/CDC1        Email  : jsw at cdc.noaa.gov
325 Broadway                Web    : www.cdc.noaa.gov/~jsw
Boulder, CO, USA 80303-3328 Office : Skaggs Research Cntr 1D-124




From chrishbarker at home.net  Thu Aug 23 12:42:14 2001
From: chrishbarker at home.net (Chris Barker)
Date: Thu, 23 Aug 2001 09:42:14 -0700
Subject: [Numpy-discussion] C API questions and tricks.
References: <000001c12802$f12a2040$0301a8c0@plstn1.sfba.home.com>
Message-ID: <3B853266.A5B36D8F@home.net>

Hi all,

I recently discovered the "clip()" function, and thought it was just
what I'd been wanting, because I need to do that a lot, and it provided
a nice notation, and I was hoping speed improvement over a couple of
where() statements.

I did some experiments, and discovered that it was, in fact, slower than
the where statements, and that the fastest way to do it is to use two
putmask() calls. (note: this changes the array in place, rather than
creating a new one)

The reason it is not faster is because it is written in Python, calling
choose(), similarly to how where() does. I decided that I could use a
fast version, so I set out to write one in C, resulting in the following
questions:

A) How do I loop through all the elements of a discontiguous array of
arbitraty dimensions? If I knew the rank ahead of time, I could just
nest some for loops and use the strides[] values. Not knowing before
hand, it seems that I should be able to do some nesting of loops using
nd and dimensions[], but I can't seem to work it out. Someone must have
come up with a nifty way to do this. Is there an existing macro or
function to do it?

B) How can I write a function that can act on any of the NumPy data
types? I currently have it written to only work with contiguous Float
arrays, which is what i need at the moment, but I'd much rather have one
for the general case.

C) I'd also like any feedback on other elements of my code (enclosed
with this email). A few comments on what the function (I've called it
fastclip) is supposed to do:

"""
fastclip(A,min,max)

changes the array, A, in place, so that all the elements less than min
are replaced by min, and all the elements greater that max are replaced
by max.

min and max can be either scalars, or anything that can be converted to
an array with the same number of elements as A (using
PyArray_ContiguousFromObject() ). If min and/or max is an array, than
the coresponding elements are used. This allows, among other things, a
way to clip to just a min or max value by calling it as:
fastclip(A,A,max) or fastclip(A,min,A).

"""

I wrote a little test script to benchmark the function, and it is much
faster that the alternatives that I have thought of:

#!/usr/bin/env python
# testing speed of where vs clip vs fastclip

from Numeric import *
from RandomArray import uniform
from NumericExtras import fastclip
import time

n = 5000

a = uniform(0,100,(n,))
b = uniform(0,100,(n,))
c = uniform(0,100,(n,))

min = 20.0
max = 80.0

print "n = %i"%(n,)

start = time.clock()
for i in range(100):
    a = clip(a,min,max)
print "clip took %f seconds"%(time.clock()-start)

start = time.clock()
for i in range(100):
    putmask(a,a < min,min)
    putmask(a,a > max,max)
print "putmask took %f seconds"%(time.clock()-start)

start = time.clock()
for i in range(100):
    fastclip(a,min,max)
print "fastclip took %f seconds"%(time.clock()-start)

Here are some results:

n = 50
clip took 0.020000 seconds
putmask took 0.050000 seconds
fastclip took 0.010000 seconds

n = 5000
clip took 0.300000 seconds
putmask took 0.230000 seconds
fastclip took 0.030000 seconds

As expected the large the array is, the better the improvement.

I'd love to here any feedbackyou can give me: I've new to writing Python
extensions, using the  Numeric API, and C itself, for that matter.

-thanks, Chris



















-- 
Christopher Barker,
Ph.D.                                                           
ChrisHBarker at home.net                 ---           ---           ---
http://members.home.net/barkerlohmann ---@@       -----@@       -----@@
                                   ------@@@     ------@@@     ------@@@
Oil Spill Modeling                ------   @    ------   @   ------   @
Water Resources Engineering       -------      ---------     --------    
Coastal and Fluvial Hydrodynamics --------------------------------------
------------------------------------------------------------------------
-------------- next part --------------

#include "Python.h"
#include "arrayobject.h"

// This define is to accomidate the different ways shared libs are built (see the bottom of the file)

#define _LINUX // this shouild be one of:  _LINUX, _WIN32, or _MACOS

// These are little macros I use to access array values for specific rank arrays
#define ARRAYVAL0(aType,a) ( *(aType *)(a->data))	
#define ARRAYVAL1(aType,a,i) ( *(aType *)(a->data + (i)*a->strides[0]))	
#define ARRAYVAL2(aType,a,i,j) ( *(aType *)(a->data + (i)*a->strides[0] + (j)*a->strides[1]))	
#define ARRAYVAL3(aType,a,i,j,k) ( *(aType *)(a->data + (i)*a->strides[0] + (j)*a->strides[1] + (k)*a->strides[2]))	



// A function that computes the total number of elements in and array 
// Does this exist already?
int TotalNumberOfElements(PyArrayObject *Array)
{
  int total = 1;
  short i;
  
  for (i = 0; i < Array->nd; i++)
    {
      total *= Array->dimensions[i];
    }
  return total;
}

// This is the fastclip code:
// called from Python as: fastclip(Array,min,max)
// min, max can be scalar or the same size as Array.
// At the moment, Array has to be contiguous, and it only works with arrays of type Float.
static PyObject * NumericExtras_fastclip(PyObject *self, PyObject *args)
{
  
  int NumArray, elementsize, i;

  PyObject *InMin;
  PyObject *InMax;

  PyArrayObject *Array;
  PyArrayObject *Min;
  PyArrayObject *Max;

  //printf("I'm starting\n");

  if (!PyArg_ParseTuple(args, "O!OO", 
						&PyArray_Type, &Array,
					    &InMin,
						&InMax))
  {
    return NULL;
  }  
  
  // check types of input

  // check type of Array
  if (!PyArray_ISCONTIGUOUS(Array)){
	PyErr_SetString (PyExc_ValueError,
					 "a must be contiguous");
	return NULL;
  }
                            
  if (Array->descr->type_num != PyArray_DOUBLE){
	PyErr_SetString (PyExc_ValueError,
					 "a must be a NumPy array of type Float");
	return NULL;
  }

  // convert min and max to arrays:
  // if they don't convert, the input values are not valid
  Min = (PyArrayObject *) PyArray_ContiguousFromObject(InMin, PyArray_DOUBLE, 0, 0);
  if (Min == NULL){
    PyErr_SetString (PyExc_ValueError,
					 "min must be either a scalar or the same size as a");
    return NULL;
  }
  Max = (PyArrayObject *) PyArray_ContiguousFromObject(InMax, PyArray_DOUBLE, 0, 0);
  if (Max == NULL){
    PyErr_SetString (PyExc_ValueError,
					 "max must be either a scalar or the same size as a");
    Py_DECREF(Min);
    return NULL;
  }


  // This seems pretty kludgy to have the four cases, but it was the first
  // thing that came to mind that I was sure would work, and would accomidate
  // either array or scalar arguments for min and max. 
  NumArray = TotalNumberOfElements(Array);
  elementsize = sizeof(double);

  if (TotalNumberOfElements(Min) == 1 && TotalNumberOfElements(Max) == 1){ // both limits are scalar
    for (i = 0; i < NumArray; i++) { // loop over each element
      if (*(double *)(Array->data + i*elementsize) > ARRAYVAL0(double,Max)){
         *(double *)(Array->data + i*elementsize) = ARRAYVAL0(double,Max);
          }
      else if (*(double *)(Array->data + i*elementsize) < ARRAYVAL0(double,Min)){
         *(double *)(Array->data + i*elementsize) = ARRAYVAL0(double,Min);
      }
    }
  }
  else if (TotalNumberOfElements(Min) == 1 && TotalNumberOfElements(Max) == NumArray){ // Min is scalar
    for (i = 0; i < NumArray; i++) { // loop over each element
      if (*(double *)(Array->data + i*elementsize) > *(double *)(Max->data + i*elementsize)){
         *(double *)(Array->data + i*elementsize) = *(double *)(Max->data + i*elementsize) ;
          }
      else if (*(double *)(Array->data + i*elementsize) < ARRAYVAL0(double,Min)){
         *(double *)(Array->data + i*elementsize) = ARRAYVAL0(double,Min);
      }
    }
  }    
  else if (TotalNumberOfElements(Max) == 1 && TotalNumberOfElements(Min) == NumArray){ // Max is scalar
    for (i = 0; i < NumArray; i++) { // loop over each element
      if (*(double *)(Array->data + i*elementsize) > ARRAYVAL0(double,Max)){
         *(double *)(Array->data + i*elementsize) = ARRAYVAL0(double,Max);
          }
      else if (*(double *)(Array->data + i*elementsize) < *(double *)(Min->data + i*elementsize)){
         *(double *)(Array->data + i*elementsize) = *(double *)(Min->data + i*elementsize) ;
      }
    }
  }    
  else if (TotalNumberOfElements(Min) == NumArray && TotalNumberOfElements(Max) == NumArray){ // Neither is scalar
    for (i = 0; i < NumArray; i++) { // loop over each element
      if (*(double *)(Array->data + i*elementsize) > *(double *)(Max->data + i*elementsize)){
         *(double *)(Array->data + i*elementsize) = *(double *)(Max->data + i*elementsize) ;
          }
      else if (*(double *)(Array->data + i*elementsize) < *(double *)(Min->data + i*elementsize)){
         *(double *)(Array->data + i*elementsize) = *(double *)(Min->data + i*elementsize) ;
      }
    }    
  }
  else { // The size of Min and/or Max don't match Array
    	PyErr_SetString (PyExc_ValueError,
					 "min and max must be either scalar or the same number of elements as a");
        Py_DECREF(Min);
        Py_DECREF(Max);
        return NULL;
  }

  return Py_None;
}
  

	
static PyMethodDef NumericExtrasMethods[] = {
  {"fastclip", NumericExtras_fastclip, METH_VARARGS},
  {NULL, NULL} /* Sentinel */
};


#if defined _WIN32
  void _declspec(dllexport)
#elif defined _LINUX
  void 
#elif defined _MACOS
  void
#else
  #error There is no valid platform defined (should be in platfrom.h and be one of: _WIN32,_LINUX,_MACOS)
#endif

initNumericExtras()

{
  (void) Py_InitModule("NumericExtras", NumericExtrasMethods);
  import_array()
}





















From paul at pfdubois.com  Thu Aug 23 13:41:40 2001
From: paul at pfdubois.com (Paul F. Dubois)
Date: Thu, 23 Aug 2001 10:41:40 -0700
Subject: [Numpy-discussion] For the next round on the manual
In-Reply-To: <006801c12844$e384e870$6401a8c0@cc911046a>
Message-ID: <000101c12bfa$de43fae0$0301a8c0@plstn1.sfba.home.com>

Fixed in my manual source, thank you. 
 
-----Original Message-----
From: numpy-discussion-admin at lists.sourceforge.net
[mailto:numpy-discussion-admin at lists.sourceforge.net] On Behalf Of Eric
Hagemann
Sent: Saturday, August 18, 2001 5:21 PM
To: Numpy-discussion at lists.sourceforge.net
Subject: [Numpy-discussion] For the next round on the manual
 
Type-o in the manual
 
    Check the definition for the 'convolve' function.  Manual tells of
mode=0 as default when code and experiment point to mode=2 being default
 
    No biggie -- but it might be nice to correct if anybody is editing
the manual any time soon
 
Cheers
Eric
 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20010823/51079634/attachment-0001.html>

From chrishbarker at home.net  Thu Aug 23 15:19:33 2001
From: chrishbarker at home.net (Chris Barker)
Date: Thu, 23 Aug 2001 12:19:33 -0700
Subject: [Numpy-discussion] For the next round on the manual
References: <000101c12bfa$de43fae0$0301a8c0@plstn1.sfba.home.com>
Message-ID: <3B855745.2FE519D6@home.net>

While you're at it, I found what appears to be another small typo:

In the API section, description of the ArrayObject structure, is the
line:

a.shape = (dimensions[0]...dimensions[nd])


shouldn't this be dimensions[nd-1] ?


-Chris


-- 
Christopher Barker,
Ph.D.                                                           
ChrisHBarker at home.net                 ---           ---           ---
http://members.home.net/barkerlohmann ---@@       -----@@       -----@@
                                   ------@@@     ------@@@     ------@@@
Oil Spill Modeling                ------   @    ------   @   ------   @
Water Resources Engineering       -------      ---------     --------    
Coastal and Fluvial Hydrodynamics --------------------------------------
------------------------------------------------------------------------



From rob at hooft.net  Fri Aug 24 02:49:40 2001
From: rob at hooft.net (Rob W. W. Hooft)
Date: Fri, 24 Aug 2001 08:49:40 +0200
Subject: [Numpy-discussion] C API questions and tricks.
In-Reply-To: <3B853266.A5B36D8F@home.net>
References: <000001c12802$f12a2040$0301a8c0@plstn1.sfba.home.com>
	<3B853266.A5B36D8F@home.net>
Message-ID: <15237.63748.700849.145167@temoleh.chem.uu.nl>

>>>>> "CB" == Chris Barker <chrishbarker at home.net> writes:

I wrote my own specialized clip function too, only for 'f' and 'i'
types, and only for contiguous arrays....

 CB> A) How do I loop through all the elements of a discontiguous
 CB> array of arbitraty dimensions? If I knew the rank ahead of time,
 CB> I could just nest some for loops and use the strides[]
 CB> values. Not knowing before hand, it seems that I should be able
 CB> to do some nesting of loops using nd and dimensions[], but I
 CB> can't seem to work it out. Someone must have come up with a nifty
 CB> way to do this. Is there an existing macro or function to do it?

Never done this in C for numpy yet, but I normally program this
along the following lines:

n=[2,3,2,3]
x=[0,0,0,0]
while 1:
    print x
    i=len(n)-1
    while i>=0 and x[i]+1==n[i]:
       x[i]=0
       i=i-1
    if i<0:
       break
    x[i]=x[i]+1

It is always going to be slower than the straight loop for contiguous
arrays. So if you want to do it properly, you'd have to check whether
the array is contiguous, and if it is, use the fast way.

Regards,

Rob Hooft

-- 
=====   rob at hooft.net                 http://www.hooft.net/people/rob/  =====
=====   R&D, Bruker Nonius BV, Delft  http://www.nonius.nl/             =====
===== PGPid 0xFA19277D ================================= Use Linux! =========



From paul at pfdubois.com  Fri Aug 24 14:45:43 2001
From: paul at pfdubois.com (Paul F. Dubois)
Date: Fri, 24 Aug 2001 11:45:43 -0700
Subject: [Numpy-discussion] manual update
Message-ID: <000201c12ccc$fc388e80$0301a8c0@plstn1.sfba.home.com>

I have updated the Numeric manual on the web site. I added some to the
MLab section to reflect the axis arguments that were added some time ago
but never reflected in the manual. The manual also documents a change to
the behavior of the attribute "masked" in package MA. The document
corresponds to CVS now rather than the last release. I hope to release
Numeric-20.2 sometime soon but want more exercise of the MA and MLab
changes first.
 
I changed MLab.mean to return a floating point result when the argument
is integer. I'm sorry if this breaks anything but the prior behavior has
no mathematical meaning and then, worse, was being used by std to get
the wrong standard deviation too. I changed std to work for axis > 0, as
it was previously broken.
 
I don't own Matlab and so maintaining a package that emulates something
I don't have makes me nervous. I wish one of the other developers with
more knowledge in this area would pitch in.
 
Thanks to Chris Barker and others who sent in typos and suggestions.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20010824/8b3578c9/attachment-0001.html>

From europax at home.com  Fri Aug 24 16:14:52 2001
From: europax at home.com (Rob)
Date: Fri, 24 Aug 2001 13:14:52 -0700
Subject: [Numpy-discussion] manual update
References: <000201c12ccc$fc388e80$0301a8c0@plstn1.sfba.home.com>
Message-ID: <3B86B5BC.4AF0CB65@home.com>

Re Matlab,

I've never used it before, but I was very suprised to see that its
arrays work much like Numpy.  I have some Matlab FDTD code here and the
arrays are sliced instead of indexed.  I might be able to rope a few
more people at work into Python based on that :)    Rob.



From edcjones at erols.com  Sat Aug 25 11:16:33 2001
From: edcjones at erols.com (Edward C. Jones)
Date: Sat, 25 Aug 2001 11:16:33 -0400
Subject: [Numpy-discussion] HTML manual display problems in Mozilla
Message-ID: <3B87C151.4040108@erols.com>

Mozilla .0.9.3 is not correctly displaying the HTML version of the 
manual. The problem appears to be in footnotes. Netscape 4.76 had 
similar problems with the previous version of the manual. I use RedHat 
7.1 Linux on a PC.

Thanks,
Ed Jones





From europax at home.com  Sun Aug 26 11:38:56 2001
From: europax at home.com (Rob)
Date: Sun, 26 Aug 2001 08:38:56 -0700
Subject: [Numpy-discussion] Can this function by Numpy-ized?
Message-ID: <3B89180F.CD83EA10@home.com>

I finally got my FEM EM code working.  I profiled it and this function
uses up a big hunk of time.  It performs gaussian integration over a
triangle.  I am trying to figure out how to slice the arrays so as to
push it down into the C level.  Does anyone have any ideas?  Thanks, 
Rob.

ps. it looks to be intractible to me.  Maybe I need to look at writing a
C extension.  I've never done that before.


##***************************************************************************
##Prototype:  void  ComputeGaussQuadPoint(int QuadPoint, int
*a.TrngleNode,
##                                            double *SrcPointCol) 
##Description: To compute the coordinates of 7-point Gauss nodes of 
##             a triangular patch. 
##Input value: 
##    int QuadPoint  --- node index, it can be from 0 to 6. 
##    int *a.TrngleNode  ---  the three nodes of a tringular patch. 
##    double *SrcPointCol --- where to store the results 
##Return value: none 
##Global value used: a.NodeCord, a.Qpnt 
##Global value modified:    none 
##Subroutines called:    none 
##Note: Not very sure.           
****************************************************************************
def ComputeGaussQuadPoint(QuadPoint,TrngleNode,a):
    
    SrcPointCol=zeros((3),Float)
    
    SrcPointCol[0] =   a.Qpnt[QuadPoint,0]*a.NodeCord[TrngleNode[0],0]\
                     + a.Qpnt[QuadPoint,1]*a.NodeCord[TrngleNode[1],0]\
                     + a.Qpnt[QuadPoint,2]*a.NodeCord[TrngleNode[2],0]

    SrcPointCol[1] =   a.Qpnt[QuadPoint,0]*a.NodeCord[TrngleNode[0],1]\
                     + a.Qpnt[QuadPoint,1]*a.NodeCord[TrngleNode[1],1]\
                     + a.Qpnt[QuadPoint,2]*a.NodeCord[TrngleNode[2],1]

    SrcPointCol[2] =   a.Qpnt[QuadPoint,0]*a.NodeCord[TrngleNode[0],2]\
                     + a.Qpnt[QuadPoint,1]*a.NodeCord[TrngleNode[1],2]\
                     + a.Qpnt[QuadPoint,2]*a.NodeCord[TrngleNode[2],2]
                                     

    return SrcPointCol
  





-- 
The Numeric Python EM Project

www.members.home.net/europax



From jjl at pobox.com  Sun Aug 26 13:27:08 2001
From: jjl at pobox.com (John J. Lee)
Date: Sun, 26 Aug 2001 18:27:08 +0100 (BST)
Subject: [Numpy-discussion] Can this function by Numpy-ized?
In-Reply-To: <3B89180F.CD83EA10@home.com>
Message-ID: <Pine.SOL.4.30.0108261755160.10385-100000@mimosa.csv.warwick.ac.uk>

On Sun, 26 Aug 2001, Rob wrote:

> I finally got my FEM EM code working.  I profiled it and this function
> uses up a big hunk of time.  It performs gaussian integration over a
> triangle.  I am trying to figure out how to slice the arrays so as to
> push it down into the C level.  Does anyone have any ideas?  Thanks,
> Rob.
>
> ps. it looks to be intractible to me.  Maybe I need to look at writing a
> C extension.  I've never done that before.
[...]

I'm not a good enough ufunc / array function hacker to come up with
anything, but I doubt it, and I wonder how much time it would save anyway,
given the size of the arrays involved.  From the look of your comments
before the function, it looks like a) you're a C programmer, not quite
comfortable with Python yet, or b) you wrote it in C first, then moved it
into Python.  If the latter, you might want to try wrapping that C
function with SWIG, though to be honest I'm not sure the overhead of
learning to use SWIG is any less than for writing a C extension manually
(but less error-prone I expect, and less work if you have a lot of C to
wrap).  I think the Scipy project has some typemaps for passing Numeric
arrays; if not, I've seen some collections of SWIG typemaps for Numeric
somewhere...

BTW, the best place to put those comments is in a docstring.  Here is a
slightly more Pythonically-formatted version (I must be bored today):

def ComputeGaussQuadPoint(QuadPoint, a):
    """Return coordinates of 7-point Gauss nodes of a triangular patch.

    QuadPoint: node index, from 0 to 6
    a: triangular patch?
    """
    SrcPointCol=zeros((3),Float)
    tn = a.TrngleNode  # the three nodes of the triangular patch
    SrcPointCol[0] = (a.Qpnt[QuadPoint,0]*a.NodeCord[tn[0],0] +
                      a.Qpnt[QuadPoint,1]*a.NodeCord[tn[1],0] +
                      a.Qpnt[QuadPoint,2]*a.NodeCord[tn[2],0])

    SrcPointCol[1] = (a.Qpnt[QuadPoint,0]*a.NodeCord[tn[0],1] +
                      a.Qpnt[QuadPoint,1]*a.NodeCord[tn[1],1] +
                      a.Qpnt[QuadPoint,2]*a.NodeCord[tn[2],1])

    SrcPointCol[2] = (a.Qpnt[QuadPoint,0]*a.NodeCord[tn[0],2] +
                      a.Qpnt[QuadPoint,1]*a.NodeCord[tn[1],2] +
                      a.Qpnt[QuadPoint,2]*a.NodeCord[tn[2],2])

    return SrcPointCol


John




From paul at pfdubois.com  Sun Aug 26 13:32:28 2001
From: paul at pfdubois.com (Paul F. Dubois)
Date: Sun, 26 Aug 2001 10:32:28 -0700
Subject: [Numpy-discussion] Can this function by Numpy-ized?
In-Reply-To: <3B89180F.CD83EA10@home.com>
Message-ID: <000001c12e55$144a5c40$0301a8c0@plstn1.sfba.home.com>

def cgqp(QuadPoint, TrngleNode, a):
    s = a.Qpnt[Quadpoint,:]
    c = Numeric.take(a.NodeCord, TrngleNode)
    return Numeric.add.reduce(s * c, axis=1)

This may or may not be right. The data structures I would have to set up
to test it are too much for Sunday morning.

-----Original Message-----
From: numpy-discussion-admin at lists.sourceforge.net
[mailto:numpy-discussion-admin at lists.sourceforge.net] On Behalf Of Rob
Sent: Sunday, August 26, 2001 8:39 AM
To: numpy-discussion at lists.sourceforge.net.
Subject: [Numpy-discussion] Can this function by Numpy-ized?

I finally got my FEM EM code working.  I profiled it and this function
uses up a big hunk of time.  It performs gaussian integration over a
triangle.  I am trying to figure out how to slice the arrays so as to
push it down into the C level.  Does anyone have any ideas?  Thanks, 
Rob.

ps. it looks to be intractible to me.  Maybe I need to look at writing a
C extension.  I've never done that before.


##**********************************************************************
*****
##Prototype:  void  ComputeGaussQuadPoint(int QuadPoint, int
*a.TrngleNode,
##                                            double *SrcPointCol) 
##Description: To compute the coordinates of 7-point Gauss nodes of 
##             a triangular patch. 
##Input value: 
##    int QuadPoint  --- node index, it can be from 0 to 6. 
##    int *a.TrngleNode  ---  the three nodes of a tringular patch. 
##    double *SrcPointCol --- where to store the results 
##Return value: none 
##Global value used: a.NodeCord, a.Qpnt 
##Global value modified:    none 
##Subroutines called:    none 
##Note: Not very sure.           
************************************************************************
****
def ComputeGaussQuadPoint(QuadPoint,TrngleNode,a):
    
    SrcPointCol=zeros((3),Float)
    
    SrcPointCol[0] =   a.Qpnt[QuadPoint,0]*a.NodeCord[TrngleNode[0],0]\
                     + a.Qpnt[QuadPoint,1]*a.NodeCord[TrngleNode[1],0]\
                     + a.Qpnt[QuadPoint,2]*a.NodeCord[TrngleNode[2],0]

    SrcPointCol[1] =   a.Qpnt[QuadPoint,0]*a.NodeCord[TrngleNode[0],1]\
                     + a.Qpnt[QuadPoint,1]*a.NodeCord[TrngleNode[1],1]\
                     + a.Qpnt[QuadPoint,2]*a.NodeCord[TrngleNode[2],1]

    SrcPointCol[2] =   a.Qpnt[QuadPoint,0]*a.NodeCord[TrngleNode[0],2]\
                     + a.Qpnt[QuadPoint,1]*a.NodeCord[TrngleNode[1],2]\
                     + a.Qpnt[QuadPoint,2]*a.NodeCord[TrngleNode[2],2]
                                     

    return SrcPointCol
  





-- 
The Numeric Python EM Project

www.members.home.net/europax

_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
http://lists.sourceforge.net/lists/listinfo/numpy-discussion




From europax at home.com  Sun Aug 26 16:17:06 2001
From: europax at home.com (Rob)
Date: Sun, 26 Aug 2001 13:17:06 -0700
Subject: [Numpy-discussion] Can this function by Numpy-ized?
References: <000001c12e55$144a5c40$0301a8c0@plstn1.sfba.home.com>
Message-ID: <3B895942.F527D958@home.com>

Thanks Paul,  I will try it out.  That at least gives me some
direction.  I've been pouring over the Numpy manual, but there are so
many different functions!  I would like to try to write an extension
that includes some of these FEM routines, but thats for later.  

BTW, at work I have to use Windows NT, but I've grown to love the
calldll/DynWin extension.  You can even interact with the windows kernel
with that one.   Rob.


"Paul F. Dubois" wrote:
> 
> def cgqp(QuadPoint, TrngleNode, a):
>     s = a.Qpnt[Quadpoint,:]
>     c = Numeric.take(a.NodeCord, TrngleNode)
>     return Numeric.add.reduce(s * c, axis=1)
> 
> This may or may not be right. The data structures I would have to set up
> to test it are too much for Sunday morning.
> 
> -----Original Message-----
> From: numpy-discussion-admin at lists.sourceforge.net
> [mailto:numpy-discussion-admin at lists.sourceforge.net] On Behalf Of Rob
> Sent: Sunday, August 26, 2001 8:39 AM
> To: numpy-discussion at lists.sourceforge.net.
> Subject: [Numpy-discussion] Can this function by Numpy-ized?
> 
> I finally got my FEM EM code working.  I profiled it and this function
> uses up a big hunk of time.  It performs gaussian integration over a
> triangle.  I am trying to figure out how to slice the arrays so as to
> push it down into the C level.  Does anyone have any ideas?  Thanks,
> Rob.
> 
> ps. it looks to be intractible to me.  Maybe I need to look at writing a
> C extension.  I've never done that before.
> 
> ##**********************************************************************
> *****
> ##Prototype:  void  ComputeGaussQuadPoint(int QuadPoint, int
> *a.TrngleNode,
> ##                                            double *SrcPointCol)
> ##Description: To compute the coordinates of 7-point Gauss nodes of
> ##             a triangular patch.
> ##Input value:
> ##    int QuadPoint  --- node index, it can be from 0 to 6.
> ##    int *a.TrngleNode  ---  the three nodes of a tringular patch.
> ##    double *SrcPointCol --- where to store the results
> ##Return value: none
> ##Global value used: a.NodeCord, a.Qpnt
> ##Global value modified:    none
> ##Subroutines called:    none
> ##Note: Not very sure.
> ************************************************************************
> ****
> def ComputeGaussQuadPoint(QuadPoint,TrngleNode,a):
> 
>     SrcPointCol=zeros((3),Float)
> 
>     SrcPointCol[0] =   a.Qpnt[QuadPoint,0]*a.NodeCord[TrngleNode[0],0]\
>                      + a.Qpnt[QuadPoint,1]*a.NodeCord[TrngleNode[1],0]\
>                      + a.Qpnt[QuadPoint,2]*a.NodeCord[TrngleNode[2],0]
> 
>     SrcPointCol[1] =   a.Qpnt[QuadPoint,0]*a.NodeCord[TrngleNode[0],1]\
>                      + a.Qpnt[QuadPoint,1]*a.NodeCord[TrngleNode[1],1]\
>                      + a.Qpnt[QuadPoint,2]*a.NodeCord[TrngleNode[2],1]
> 
>     SrcPointCol[2] =   a.Qpnt[QuadPoint,0]*a.NodeCord[TrngleNode[0],2]\
>                      + a.Qpnt[QuadPoint,1]*a.NodeCord[TrngleNode[1],2]\
>                      + a.Qpnt[QuadPoint,2]*a.NodeCord[TrngleNode[2],2]
> 
> 
>     return SrcPointCol
> 
> 
> --
> The Numeric Python EM Project
> 
> www.members.home.net/europax
> 
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> http://lists.sourceforge.net/lists/listinfo/numpy-discussion
> 
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> http://lists.sourceforge.net/lists/listinfo/numpy-discussion

-- 
The Numeric Python EM Project

www.members.home.net/europax



From europax at home.com  Sun Aug 26 18:18:46 2001
From: europax at home.com (Rob)
Date: Sun, 26 Aug 2001 15:18:46 -0700
Subject: [Numpy-discussion] Can this function by Numpy-ized?
References: <000001c12e55$144a5c40$0301a8c0@plstn1.sfba.home.com>
Message-ID: <3B8975C6.5EE83A96@home.com>

The good news is that your routine reduces execution time by 30%.  The
bad news is that the wrong numbers are coming out of it.  I'm trying
some otner permutations of "take" and "add.reduce" in the function to
see if I can get it.  One method that worked was splitting up "c" into
c1,c2,c3, such that:

c1=Numeric.take(a.NodeCord[:,0],TrngleNode)
etc

and then using  Numeric.add.reduce(s*c1,1)
etc

This gives the right results, but is slower than plain Python.  I'll
keep at it.  Thanks again.


"Paul F. Dubois" wrote:
> 
> def cgqp(QuadPoint, TrngleNode, a):
>     s = a.Qpnt[Quadpoint,:]
>     c = Numeric.take(a.NodeCord, TrngleNode)
>     return Numeric.add.reduce(s * c, axis=1)
> 
> This may or may not be right. The data structures I would have to set up
> to test it are too much for Sunday morning.
> 
> -----Original Message-----
> From: numpy-discussion-admin at lists.sourceforge.net
> [mailto:numpy-discussion-admin at lists.sourceforge.net] On Behalf Of Rob
> Sent: Sunday, August 26, 2001 8:39 AM
> To: numpy-discussion at lists.sourceforge.net.
> Subject: [Numpy-discussion] Can this function by Numpy-ized?
> 
> I finally got my FEM EM code working.  I profiled it and this function
> uses up a big hunk of time.  It performs gaussian integration over a
> triangle.  I am trying to figure out how to slice the arrays so as to
> push it down into the C level.  Does anyone have any ideas?  Thanks,
> Rob.
> 
> ps. it looks to be intractible to me.  Maybe I need to look at writing a
> C extension.  I've never done that before.
> 
> ##**********************************************************************
> *****
> ##Prototype:  void  ComputeGaussQuadPoint(int QuadPoint, int
> *a.TrngleNode,
> ##                                            double *SrcPointCol)
> ##Description: To compute the coordinates of 7-point Gauss nodes of
> ##             a triangular patch.
> ##Input value:
> ##    int QuadPoint  --- node index, it can be from 0 to 6.
> ##    int *a.TrngleNode  ---  the three nodes of a tringular patch.
> ##    double *SrcPointCol --- where to store the results
> ##Return value: none
> ##Global value used: a.NodeCord, a.Qpnt
> ##Global value modified:    none
> ##Subroutines called:    none
> ##Note: Not very sure.
> ************************************************************************
> ****
> def ComputeGaussQuadPoint(QuadPoint,TrngleNode,a):
> 
>     SrcPointCol=zeros((3),Float)
> 
>     SrcPointCol[0] =   a.Qpnt[QuadPoint,0]*a.NodeCord[TrngleNode[0],0]\
>                      + a.Qpnt[QuadPoint,1]*a.NodeCord[TrngleNode[1],0]\
>                      + a.Qpnt[QuadPoint,2]*a.NodeCord[TrngleNode[2],0]
> 
>     SrcPointCol[1] =   a.Qpnt[QuadPoint,0]*a.NodeCord[TrngleNode[0],1]\
>                      + a.Qpnt[QuadPoint,1]*a.NodeCord[TrngleNode[1],1]\
>                      + a.Qpnt[QuadPoint,2]*a.NodeCord[TrngleNode[2],1]
> 
>     SrcPointCol[2] =   a.Qpnt[QuadPoint,0]*a.NodeCord[TrngleNode[0],2]\
>                      + a.Qpnt[QuadPoint,1]*a.NodeCord[TrngleNode[1],2]\
>                      + a.Qpnt[QuadPoint,2]*a.NodeCord[TrngleNode[2],2]
> 
> 
>     return SrcPointCol
> 
> 
> --
> The Numeric Python EM Project
> 
> www.members.home.net/europax
> 
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> http://lists.sourceforge.net/lists/listinfo/numpy-discussion

-- 
The Numeric Python EM Project

www.members.home.net/europax



From europax at home.com  Mon Aug 27 10:49:22 2001
From: europax at home.com (Rob)
Date: Mon, 27 Aug 2001 07:49:22 -0700
Subject: [Numpy-discussion] Gaussian Quadrature routine Numpy-ization :)
Message-ID: <3B8A5DF2.F6AC7487@home.com>

I finally got it to work, but the Numpy-ized version runs slower than
the plain Python one.  I think that I can transpose the NodeCord matrix
once in the program and feed that in, rather than the scratch matrix
that is generated here.  Evidently transpose is not very fast even for
smal matrices. Here is my test program:


from Numeric import *



Qpnt=array(([20,21,22],[23,24,25],[26,27,28]))
NodeCord=array(([1,2,3],[4,5,6],[7,8,9]))
TrngleNode=array((1,2,0))


#the original routine
def ComputeGaussQuadPoint(QuadPoint,TrngleNode,Qpnt,NodeCord):
    
    SrcPointCol=zeros((3))
    
    
    SrcPointCol[0] =   Qpnt[QuadPoint,0]*NodeCord[TrngleNode[0],0]\
                     + Qpnt[QuadPoint,1]*NodeCord[TrngleNode[1],0]\
                     + Qpnt[QuadPoint,2]*NodeCord[TrngleNode[2],0]

    SrcPointCol[1] =   Qpnt[QuadPoint,0]*NodeCord[TrngleNode[0],1]\
                     + Qpnt[QuadPoint,1]*NodeCord[TrngleNode[1],1]\
                     + Qpnt[QuadPoint,2]*NodeCord[TrngleNode[2],1]

    SrcPointCol[2] =   Qpnt[QuadPoint,0]*NodeCord[TrngleNode[0],2]\
                     + Qpnt[QuadPoint,1]*NodeCord[TrngleNode[1],2]\
                     + Qpnt[QuadPoint,2]*NodeCord[TrngleNode[2],2]

    return SrcPointCol


#the yet-to-be-faster routine

def Quad(QuadPoint, TrngleNode, Qpnt,NodeCord):
    s = Qpnt[QuadPoint,:]
    c= take(NodeCord, TrngleNode)
    SrcPointCol= add.reduce(s *
transpose(c),1)                               

    return SrcPointCol
  
QuadPoint=1

print "The Correct:"
print ComputeGaussQuadPoint(QuadPoint,TrngleNode,Qpnt,NodeCord)

print "The New"
print  Quad(QuadPoint,TrngleNode,Qpnt,NodeCord)





-- 
The Numeric Python EM Project

www.members.home.net/europax



From tim.hochberg at ieee.org  Mon Aug 27 12:10:01 2001
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Mon, 27 Aug 2001 09:10:01 -0700
Subject: [Numpy-discussion] Gaussian Quadrature routine Numpy-ization :)
References: <3B8A5DF2.F6AC7487@home.com>
Message-ID: <010b01c12f12$b9f6ca80$87740918@cx781526b>

Hi Rob,

It looks like you are trying to use the function below to integrate over a
single triangle. I'm almost certain that you will _not_ be able to get this
to run fast; there's just too much overhead per triangle from all the
function calls and what not. Instead you need to structure things so that
you do more work per call. One way is to pass a list of triangles and get
back a list of answers. This way you spread your area over many triangles.
Another possibility, assuming that you need to evaluate the integral for all
seven possible values of QPoint each time, is to compute the answer for all
values of QPoint at once so that you reduce the overhead per triangle by
seven. For example (minimally tested):

Qpnt=reshape(arange(7*3), (7,3))
# Other code from other messages elided....

def Quads(TrngleNode, Qpnt, NodeCord):
     c = take(NodeCord, TrngleNode)
     SrcPointCols= add.reduce(Qpnt[...,NewAxis] *  c[NewAxis,],1)
     return SrcPointCols

# Initialize stuff to minimize effects of timing....
from time import clock
q1 = [None]*7
q2 = [None]*7
rng = range(7)

# Time the three methods....

t0 = clock()
for i in rng:
    q1[i] = ComputeGaussQuadPoint(i,TrngleNode,Qpnt,NodeCord)
t1 = clock()
for i in rng:
    q2[i] = Quad(i,TrngleNode,Qpnt,NodeCord)
t2 = clock()
q3 = Quads(TrngleNode, Qpnt, NodeCord)
t3 = clock()

# And the results...

print "Times (us):", (t1-t0)*1e6, (t2-t1)*1e6, (t3-t2)*1e6
for i in range(7):
    print "The Answers:", q1[i], q2[i], q3[i]


If this code is still a bottleneck, you could do both (compute the values
for all nodes and all values of QPoint at once, but this may be enough to
move the bottleneck elsewhere; by my timing it's over 7 times as fast.

-tim

[snip]

> Evidently transpose is not very fast even for
> smal matrices.

Actually, transpose should be fast, and should look progressively faster for
larger matrices. Typically all that happens is that the strides are changed.

----- Original Message -----
From: "Rob" <europax at home.com>
To: <numpy-discussion at lists.sourceforge.net>
Sent: Monday, August 27, 2001 7:49 AM
Subject: [Numpy-discussion] Gaussian Quadrature routine Numpy-ization :)


> I finally got it to work, but the Numpy-ized version runs slower than
> the plain Python one.  I think that I can transpose the NodeCord matrix
> once in the program and feed that in, rather than the scratch matrix
> that is generated here.  Evidently transpose is not very fast even for
> smal matrices. Here is my test program:
>
>
> from Numeric import *
>
>
>
> Qpnt=array(([20,21,22],[23,24,25],[26,27,28]))
> NodeCord=array(([1,2,3],[4,5,6],[7,8,9]))
> TrngleNode=array((1,2,0))
>
>
> #the original routine
> def ComputeGaussQuadPoint(QuadPoint,TrngleNode,Qpnt,NodeCord):
>
>     SrcPointCol=zeros((3))
>
>
>     SrcPointCol[0] =   Qpnt[QuadPoint,0]*NodeCord[TrngleNode[0],0]\
>                      + Qpnt[QuadPoint,1]*NodeCord[TrngleNode[1],0]\
>                      + Qpnt[QuadPoint,2]*NodeCord[TrngleNode[2],0]
>
>     SrcPointCol[1] =   Qpnt[QuadPoint,0]*NodeCord[TrngleNode[0],1]\
>                      + Qpnt[QuadPoint,1]*NodeCord[TrngleNode[1],1]\
>                      + Qpnt[QuadPoint,2]*NodeCord[TrngleNode[2],1]
>
>     SrcPointCol[2] =   Qpnt[QuadPoint,0]*NodeCord[TrngleNode[0],2]\
>                      + Qpnt[QuadPoint,1]*NodeCord[TrngleNode[1],2]\
>                      + Qpnt[QuadPoint,2]*NodeCord[TrngleNode[2],2]
>
>     return SrcPointCol
>
>
> #the yet-to-be-faster routine
>
> def Quad(QuadPoint, TrngleNode, Qpnt,NodeCord):
>     s = Qpnt[QuadPoint,:]
>     c= take(NodeCord, TrngleNode)
>     SrcPointCol= add.reduce(s *
> transpose(c),1)
>
>     return SrcPointCol
>
> QuadPoint=1
>
> print "The Correct:"
> print ComputeGaussQuadPoint(QuadPoint,TrngleNode,Qpnt,NodeCord)
>
> print "The New"
> print  Quad(QuadPoint,TrngleNode,Qpnt,NodeCord)
>
>
>
>
>
> --
> The Numeric Python EM Project
>
> www.members.home.net/europax
>
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> http://lists.sourceforge.net/lists/listinfo/numpy-discussion




From chrishbarker at home.net  Mon Aug 27 16:17:41 2001
From: chrishbarker at home.net (Chris Barker)
Date: Mon, 27 Aug 2001 13:17:41 -0700
Subject: [Numpy-discussion] manual update
References: <000201c12ccc$fc388e80$0301a8c0@plstn1.sfba.home.com>
Message-ID: <3B8AAAE5.81E90762@home.net>

Paul F. Dubois wrote:

>I don't own Matlab and so maintaining a package that emulates something
>I don't have makes me nervous. I wish one of the other developers with
>more knowledge in this area would pitch in.


I'm not much of a NumPy developer, but I do own MATLAB, and was once a
heavy user of it (now I use Python a lot more, or course). I'm not sure
what I can contibute, but I'd be glad to test and comment.

-Chris




-- 
Christopher Barker,
Ph.D.                                                           
ChrisHBarker at home.net                 ---           ---           ---
http://members.home.net/barkerlohmann ---@@       -----@@       -----@@
                                   ------@@@     ------@@@     ------@@@
Oil Spill Modeling                ------   @    ------   @   ------   @
Water Resources Engineering       -------      ---------     --------    
Coastal and Fluvial Hydrodynamics --------------------------------------
------------------------------------------------------------------------



From europax at home.com  Mon Aug 27 20:40:30 2001
From: europax at home.com (Rob)
Date: Mon, 27 Aug 2001 17:40:30 -0700
Subject: [Numpy-discussion] Gaussian Quadrature routine Numpy-ization :)
References: <3B8A5DF2.F6AC7487@home.com> <010b01c12f12$b9f6ca80$87740918@cx781526b>
Message-ID: <3B8AE87E.E416C138@home.com>

Hi Tim,  Yes this is part of a FEM-MOM hybrid EM simulator.  It was
originally written in C, so I've had to deal with the C way of doing
things.  I'd like to consolidate many of these types of operations so
that the program becomes streamlined and easier to understand- "The
Python Way" TM.  :)

I will try your method to see how it works.  By contrast, I have a FDTD
simulator which is a true speed demon with Numpy.  But it was organized
in a way that I would easily use slicing to time march the 6 scalar
Maxwell's equations.   

In any case I'm having fun.  I do this at home.  At work I have Ansoft
Serenade and Agilent ADS at my disposal.

Rob.

Tim Hochberg wrote:
> 
> Hi Rob,
> 
> It looks like you are trying to use the function below to integrate over a
> single triangle. I'm almost certain that you will _not_ be able to get this
> to run fast; there's just too much overhead per triangle from all the
> function calls and what not. Instead you need to structure things so that
> you do more work per call. One way is to pass a list of triangles and get
> back a list of answers. This way you spread your area over many triangles.
> Another possibility, assuming that you need to evaluate the integral for all
> seven possible values of QPoint each time, is to compute the answer for all
> values of QPoint at once so that you reduce the overhead per triangle by
> seven. For example (minimally tested):
> 
> Qpnt=reshape(arange(7*3), (7,3))
> # Other code from other messages elided....
> 
> def Quads(TrngleNode, Qpnt, NodeCord):
>      c = take(NodeCord, TrngleNode)
>      SrcPointCols= add.reduce(Qpnt[...,NewAxis] *  c[NewAxis,],1)
>      return SrcPointCols
> 
> # Initialize stuff to minimize effects of timing....
> from time import clock
> q1 = [None]*7
> q2 = [None]*7
> rng = range(7)
> 
> # Time the three methods....
> 
> t0 = clock()
> for i in rng:
>     q1[i] = ComputeGaussQuadPoint(i,TrngleNode,Qpnt,NodeCord)
> t1 = clock()
> for i in rng:
>     q2[i] = Quad(i,TrngleNode,Qpnt,NodeCord)
> t2 = clock()
> q3 = Quads(TrngleNode, Qpnt, NodeCord)
> t3 = clock()
> 
> # And the results...
> 
> print "Times (us):", (t1-t0)*1e6, (t2-t1)*1e6, (t3-t2)*1e6
> for i in range(7):
>     print "The Answers:", q1[i], q2[i], q3[i]
> 
> If this code is still a bottleneck, you could do both (compute the values
> for all nodes and all values of QPoint at once, but this may be enough to
> move the bottleneck elsewhere; by my timing it's over 7 times as fast.
> 
> -tim
> 
> [snip]
> 
> > Evidently transpose is not very fast even for
> > smal matrices.
> 
> Actually, transpose should be fast, and should look progressively faster for
> larger matrices. Typically all that happens is that the strides are changed.
> 
> ----- Original Message -----
> From: "Rob" <europax at home.com>
> To: <numpy-discussion at lists.sourceforge.net>
> Sent: Monday, August 27, 2001 7:49 AM
> Subject: [Numpy-discussion] Gaussian Quadrature routine Numpy-ization :)
> 
> > I finally got it to work, but the Numpy-ized version runs slower than
> > the plain Python one.  I think that I can transpose the NodeCord matrix
> > once in the program and feed that in, rather than the scratch matrix
> > that is generated here.  Evidently transpose is not very fast even for
> > smal matrices. Here is my test program:
> >
> >
> > from Numeric import *
> >
> >
> >
> > Qpnt=array(([20,21,22],[23,24,25],[26,27,28]))
> > NodeCord=array(([1,2,3],[4,5,6],[7,8,9]))
> > TrngleNode=array((1,2,0))
> >
> >
> > #the original routine
> > def ComputeGaussQuadPoint(QuadPoint,TrngleNode,Qpnt,NodeCord):
> >
> >     SrcPointCol=zeros((3))
> >
> >
> >     SrcPointCol[0] =   Qpnt[QuadPoint,0]*NodeCord[TrngleNode[0],0]\
> >                      + Qpnt[QuadPoint,1]*NodeCord[TrngleNode[1],0]\
> >                      + Qpnt[QuadPoint,2]*NodeCord[TrngleNode[2],0]
> >
> >     SrcPointCol[1] =   Qpnt[QuadPoint,0]*NodeCord[TrngleNode[0],1]\
> >                      + Qpnt[QuadPoint,1]*NodeCord[TrngleNode[1],1]\
> >                      + Qpnt[QuadPoint,2]*NodeCord[TrngleNode[2],1]
> >
> >     SrcPointCol[2] =   Qpnt[QuadPoint,0]*NodeCord[TrngleNode[0],2]\
> >                      + Qpnt[QuadPoint,1]*NodeCord[TrngleNode[1],2]\
> >                      + Qpnt[QuadPoint,2]*NodeCord[TrngleNode[2],2]
> >
> >     return SrcPointCol
> >
> >
> > #the yet-to-be-faster routine
> >
> > def Quad(QuadPoint, TrngleNode, Qpnt,NodeCord):
> >     s = Qpnt[QuadPoint,:]
> >     c= take(NodeCord, TrngleNode)
> >     SrcPointCol= add.reduce(s *
> > transpose(c),1)
> >
> >     return SrcPointCol
> >
> > QuadPoint=1
> >
> > print "The Correct:"
> > print ComputeGaussQuadPoint(QuadPoint,TrngleNode,Qpnt,NodeCord)
> >
> > print "The New"
> > print  Quad(QuadPoint,TrngleNode,Qpnt,NodeCord)
> >
> >
> >
> >
> >
> > --
> > The Numeric Python EM Project
> >
> > www.members.home.net/europax
> >
> > _______________________________________________
> > Numpy-discussion mailing list
> > Numpy-discussion at lists.sourceforge.net
> > http://lists.sourceforge.net/lists/listinfo/numpy-discussion
> 
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> http://lists.sourceforge.net/lists/listinfo/numpy-discussion

-- 
The Numeric Python EM Project

www.members.home.net/europax



From europax at home.com  Tue Aug 28 00:34:56 2001
From: europax at home.com (Rob)
Date: Mon, 27 Aug 2001 21:34:56 -0700
Subject: [Numpy-discussion] Gaussian Quadrature routine Numpy-ization :)
References: <3B8A5DF2.F6AC7487@home.com> <010b01c12f12$b9f6ca80$87740918@cx781526b>
Message-ID: <3B8B1F70.B2807C42@home.com>

Tim, I used your routine, computing all of the triangles at once and
then feeding the resulting array back into the needy functions.  I
shaved 30% off the execution time!!  There is another routine which
computes correction terms which can use the "take()" method.  I'll work
on that tomorrow.  Thanks alot!

Rob.

ps. I'll have to study the Numpy manual to figure out how your routine
works.


Tim Hochberg wrote:
> 
> Hi Rob,
> 
> It looks like you are trying to use the function below to integrate over a
> single triangle. I'm almost certain that you will _not_ be able to get this
> to run fast; there's just too much overhead per triangle from all the
> function calls and what not. Instead you need to structure things so that
> you do more work per call. One way is to pass a list of triangles and get
> back a list of answers. This way you spread your area over many triangles.
> Another possibility, assuming that you need to evaluate the integral for all
> seven possible values of QPoint each time, is to compute the answer for all
> values of QPoint at once so that you reduce the overhead per triangle by
> seven. For example (minimally tested):
> 
> Qpnt=reshape(arange(7*3), (7,3))
> # Other code from other messages elided....
> 
> def Quads(TrngleNode, Qpnt, NodeCord):
>      c = take(NodeCord, TrngleNode)
>      SrcPointCols= add.reduce(Qpnt[...,NewAxis] *  c[NewAxis,],1)
>      return SrcPointCols
> 
> # Initialize stuff to minimize effects of timing....
> from time import clock
> q1 = [None]*7
> q2 = [None]*7
> rng = range(7)
> 
> # Time the three methods....
> 
> t0 = clock()
> for i in rng:
>     q1[i] = ComputeGaussQuadPoint(i,TrngleNode,Qpnt,NodeCord)
> t1 = clock()
> for i in rng:
>     q2[i] = Quad(i,TrngleNode,Qpnt,NodeCord)
> t2 = clock()
> q3 = Quads(TrngleNode, Qpnt, NodeCord)
> t3 = clock()
> 
> # And the results...
> 
> print "Times (us):", (t1-t0)*1e6, (t2-t1)*1e6, (t3-t2)*1e6
> for i in range(7):
>     print "The Answers:", q1[i], q2[i], q3[i]
> 
> If this code is still a bottleneck, you could do both (compute the values
> for all nodes and all values of QPoint at once, but this may be enough to
> move the bottleneck elsewhere; by my timing it's over 7 times as fast.
> 
> -tim
> 
> [snip]
> 
> > Evidently transpose is not very fast even for
> > smal matrices.
> 
> Actually, transpose should be fast, and should look progressively faster for
> larger matrices. Typically all that happens is that the strides are changed.
> 
> ----- Original Message -----
> From: "Rob" <europax at home.com>
> To: <numpy-discussion at lists.sourceforge.net>
> Sent: Monday, August 27, 2001 7:49 AM
> Subject: [Numpy-discussion] Gaussian Quadrature routine Numpy-ization :)
> 
> > I finally got it to work, but the Numpy-ized version runs slower than
> > the plain Python one.  I think that I can transpose the NodeCord matrix
> > once in the program and feed that in, rather than the scratch matrix
> > that is generated here.  Evidently transpose is not very fast even for
> > smal matrices. Here is my test program:
> >
> >
> > from Numeric import *
> >
> >
> >
> > Qpnt=array(([20,21,22],[23,24,25],[26,27,28]))
> > NodeCord=array(([1,2,3],[4,5,6],[7,8,9]))
> > TrngleNode=array((1,2,0))
> >
> >
> > #the original routine
> > def ComputeGaussQuadPoint(QuadPoint,TrngleNode,Qpnt,NodeCord):
> >
> >     SrcPointCol=zeros((3))
> >
> >
> >     SrcPointCol[0] =   Qpnt[QuadPoint,0]*NodeCord[TrngleNode[0],0]\
> >                      + Qpnt[QuadPoint,1]*NodeCord[TrngleNode[1],0]\
> >                      + Qpnt[QuadPoint,2]*NodeCord[TrngleNode[2],0]
> >
> >     SrcPointCol[1] =   Qpnt[QuadPoint,0]*NodeCord[TrngleNode[0],1]\
> >                      + Qpnt[QuadPoint,1]*NodeCord[TrngleNode[1],1]\
> >                      + Qpnt[QuadPoint,2]*NodeCord[TrngleNode[2],1]
> >
> >     SrcPointCol[2] =   Qpnt[QuadPoint,0]*NodeCord[TrngleNode[0],2]\
> >                      + Qpnt[QuadPoint,1]*NodeCord[TrngleNode[1],2]\
> >                      + Qpnt[QuadPoint,2]*NodeCord[TrngleNode[2],2]
> >
> >     return SrcPointCol
> >
> >
> > #the yet-to-be-faster routine
> >
> > def Quad(QuadPoint, TrngleNode, Qpnt,NodeCord):
> >     s = Qpnt[QuadPoint,:]
> >     c= take(NodeCord, TrngleNode)
> >     SrcPointCol= add.reduce(s *
> > transpose(c),1)
> >
> >     return SrcPointCol
> >
> > QuadPoint=1
> >
> > print "The Correct:"
> > print ComputeGaussQuadPoint(QuadPoint,TrngleNode,Qpnt,NodeCord)
> >
> > print "The New"
> > print  Quad(QuadPoint,TrngleNode,Qpnt,NodeCord)
> >
> >
> >
> >
> >
> > --
> > The Numeric Python EM Project
> >
> > www.members.home.net/europax
> >
> > _______________________________________________
> > Numpy-discussion mailing list
> > Numpy-discussion at lists.sourceforge.net
> > http://lists.sourceforge.net/lists/listinfo/numpy-discussion

-- 
The Numeric Python EM Project

www.members.home.net/europax



From a.culhane at ucc.ie  Tue Aug 28 12:22:05 2001
From: a.culhane at ucc.ie (Culhane, Aedin)
Date: Tue, 28 Aug 2001 17:22:05 +0100
Subject: [Numpy-discussion] Newbie Installation problems
Message-ID: <F64493091FAC4D4DAC46261FEC1967994796A1@xch4.ucc.ie>

Dear Numpy Discussion List Users,
I would really appreciate if someone could help me install numerical python
on Win98. I have downloaded Numerical Python twice now (older version and
20.1.0) but I am unable to get python to import numeric. I am running
python2.1.

I need it to be able to calculate basis statistics (mean, std dev, etc).
Will numerical python do this easily or should I write my own scripts?
Thanks a million for your help,
Aedin Culhane




From cgw at alum.mit.edu  Tue Aug 28 12:39:35 2001
From: cgw at alum.mit.edu (Charles G Waldman)
Date: Tue, 28 Aug 2001 11:39:35 -0500
Subject: [Numpy-discussion] Newbie Installation problems
In-Reply-To: <F64493091FAC4D4DAC46261FEC1967994796A1@xch4.ucc.ie>
References: <F64493091FAC4D4DAC46261FEC1967994796A1@xch4.ucc.ie>
Message-ID: <15243.51527.186135.887241@nyx.dyndns.org>

Culhane, Aedin writes:
 > Dear Numpy Discussion List Users,
 > I would really appreciate if someone could help me install numerical python
 > on Win98. I have downloaded Numerical Python twice now (older version and
 > 20.1.0) but I am unable to get python to import numeric. I am running
 > python2.1.

I'm sure a lot of folks on the list would be glad to help, but without
a little more detail it's hard to say much.  What errors are you
seeing?  How did you install Python?  What directory is it installed
into?  How did you try to install Numpy?

 > I need it to be able to calculate basis statistics (mean, std dev, etc).
 > Will numerical python do this easily or should I write my own scripts?
 > Thanks a million for your help,
 > Aedin Culhane

I think there are some statistics modules floating around on the Net,
but mean and standard deviation are one-liners in NumPy:


from Numeric import *

def mean(a): 
    return float(sum(a))/len(a)

def std_dev(a): 
    return sqrt(mean((a-mean(a))**2))





From chrishbarker at home.net  Tue Aug 28 13:40:10 2001
From: chrishbarker at home.net (Chris Barker)
Date: Tue, 28 Aug 2001 10:40:10 -0700
Subject: [Numpy-discussion] Newbie Installation problems
References: <F64493091FAC4D4DAC46261FEC1967994796A1@xch4.ucc.ie> <15243.51527.186135.887241@nyx.dyndns.org>
Message-ID: <3B8BD77A.1942DD31@home.net>

Charles G Waldman wrote:

> I think there are some statistics modules floating around on the Net,
> but mean and standard deviation are one-liners in NumPy:
> 
> from Numeric import *
> 
> def mean(a):
>     return float(sum(a))/len(a)
> 
> def std_dev(a):
>     return sqrt(mean((a-mean(a))**2))

or, for the unbiased estimate:

def std_dev(a):
    return sqrt(sum((a-mean(a))**2)/(len(a)-1))

You can also find these in Mlab.py

-Chris

-- 
Christopher Barker,
Ph.D.                                                           
ChrisHBarker at home.net                 ---           ---           ---
http://members.home.net/barkerlohmann ---@@       -----@@       -----@@
                                   ------@@@     ------@@@     ------@@@
Oil Spill Modeling                ------   @    ------   @   ------   @
Water Resources Engineering       -------      ---------     --------    
Coastal and Fluvial Hydrodynamics --------------------------------------
------------------------------------------------------------------------



From chrishbarker at home.net  Tue Aug 28 14:51:54 2001
From: chrishbarker at home.net (Chris Barker)
Date: Tue, 28 Aug 2001 11:51:54 -0700
Subject: [Numpy-discussion] MLab.py status: do you use it?
References: <000701c12f3b$c0605840$0301a8c0@plstn1.sfba.home.com>
Message-ID: <3B8BE84A.6CF21320@home.net>

Hi all,

Paul recently requested help with testing and debugging MLab.py, and I
offered to help out. He then sent me this note:

"Paul F. Dubois" wrote:
> MLab.py started out as a package that was supposed to give numpy a
> Matlab look and feel. So useful questions are: are there really useful
> things that should be added?

I can't answer these questions myself. Interestingly enough, despite the
fact that I came to NumPy having used MATLAB heavily for five years (and
one dissertation), I have made very little use of MLab.py. I'm wondering
if anyone is, indeed, using it.

As for trying to give numpy a MATLAB look and feel, I question the
usefulness of that. NumPy looks and feels a little different than
MATLAB, and, frankly, I like NumPy's feel better for the most part (now
that rich comparisons have been added, anyway). The one thing that
MATLAB has that NumPy doesn't, that I really miss, is list indexing.
Having to use put and take is so much less elegant! I'm sure this isn't
the least bit trivial to add, however. (is it even possible, given
Python's idea of slicing?)

Anyway, what I do think MATLAB does provide, and MLab.py should, is a
whole bunch of utility functions for various common manipulations.

> Do we really have what we have correct?
 
> Most of all, these is no test routine for it. If you could make one
> following the model of the test.py in directory Test, it would be great.
> The idea is to have something that does not require the runner of the
> test to know anything. It just runs silently unless something is wrong.

I certainly could write up a test routine. I will start work on that. In
the meantime, before I put too much effort into it, I'd really like some
feedback on what people want MLab.py to be, if they see using it at all.

As I said above, I'm not sure trying to give NumPy a MATLAB feel is a
useful goal, so what is left is having all those handy functions.
Perhaps we could shift away from MLab.py, and turn it into Utilities.py,
which could use MATLAB (and other user experience) as guidance as to
what utilities to include.

I'd really like folks' feedback on this before I put too much work into
something no one uses, or will use.

-Chris



-- 
Christopher Barker,
Ph.D.                                                           
ChrisHBarker at home.net                 ---           ---           ---
http://members.home.net/barkerlohmann ---@@       -----@@       -----@@
                                   ------@@@     ------@@@     ------@@@
Oil Spill Modeling                ------   @    ------   @   ------   @
Water Resources Engineering       -------      ---------     --------    
Coastal and Fluvial Hydrodynamics --------------------------------------
------------------------------------------------------------------------



From paul at pfdubois.com  Tue Aug 28 15:41:07 2001
From: paul at pfdubois.com (Paul F. Dubois)
Date: Tue, 28 Aug 2001 12:41:07 -0700
Subject: [Numpy-discussion] Newbie Installation problems
In-Reply-To: <15243.51527.186135.887241@nyx.dyndns.org>
Message-ID: <000001c12ff9$7b503220$0301a8c0@plstn1.sfba.home.com>

Install Python first. From the Numerical Python download site, get the
.exe file (not the zip) and run it. It is a conventional Setup-like
program. Get .exe files for the optional packages if desired and run
time too.

Functions mean and std are available in module MLab.py (note the capital
L).

Python 2.2a2 (#22, Aug 22 2001, 01:24:03) [MSC 32 bit (Intel)] on win32
Type "copyright", "credits" or "license" for more information.
IDLE 0.8 -- press F1 for help
>>> import MLab
>>> d = [1,2,4,9,12,24]
>>> MLab.mean(d)
8.6666666666666661
>>> MLab.std(d)
8.6178110136313997
>>>
-----Original Message-----
From: numpy-discussion-admin at lists.sourceforge.net
[mailto:numpy-discussion-admin at lists.sourceforge.net] On Behalf Of
Charles G Waldman
Sent: Tuesday, August 28, 2001 9:40 AM
To: Culhane, Aedin
Cc: numpy-discussion at lists.sourceforge.net
Subject: [Numpy-discussion] Newbie Installation problems

Culhane, Aedin writes:
 > Dear Numpy Discussion List Users,
 > I would really appreciate if someone could help me install numerical
python
 > on Win98. I have downloaded Numerical Python twice now (older version
and
 > 20.1.0) but I am unable to get python to import numeric. I am running
 > python2.1.

I'm sure a lot of folks on the list would be glad to help, but without
a little more detail it's hard to say much.  What errors are you
seeing?  How did you install Python?  What directory is it installed
into?  How did you try to install Numpy?

 > I need it to be able to calculate basis statistics (mean, std dev,
etc).
 > Will numerical python do this easily or should I write my own
scripts?
 > Thanks a million for your help,
 > Aedin Culhane

I think there are some statistics modules floating around on the Net,
but mean and standard deviation are one-liners in NumPy:


from Numeric import *

def mean(a): 
    return float(sum(a))/len(a)

def std_dev(a): 
    return sqrt(mean((a-mean(a))**2))



_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
http://lists.sourceforge.net/lists/listinfo/numpy-discussion




From nodwell at physics.ubc.ca  Tue Aug 28 19:24:22 2001
From: nodwell at physics.ubc.ca (Eric Nodwell)
Date: Tue, 28 Aug 2001 16:24:22 -0700
Subject: [Numpy-discussion] one-offset arrays
Message-ID: <20010828162422.A560@holler.physics.ubc.ca>

Does anyone have a simple method of using one-offset arrays?
Specifically, can I define an array "a" so that a[1] refers to the
first element?

Without one-offset indexing, it seems to me that Python is minimally
useful for numerical computations. Many, perhaps the majority, of
numerical algorithms are one-indexed. Matlab for example is one-based
for this reason.  In fact it seems strange to me that a "high-level"
language like Python should use zero-offset lists.






From serge at rohan.sdsu.edu  Tue Aug 28 20:18:34 2001
From: serge at rohan.sdsu.edu (Serge Rey)
Date: Wed, 29 Aug 2001 10:18:34 +1000
Subject: [Numpy-discussion] Re: MLab.py status: do you use it?
In-Reply-To: <E15boFC-0000dU-00@usw-sf-list1.sourceforge.net>; from numpy-discussion-request@lists.sourceforge.net on Tue, Aug 28, 2001 at 12:09:50PM -0700
References: <E15boFC-0000dU-00@usw-sf-list1.sourceforge.net>
Message-ID: <20010829101834.A657@typhoon.sdsu.edu>

On Tue, Aug 28, 2001 at 12:09:50PM -0700, numpy-discussion-request at lists.sourceforge.net wrote:

-snip- 
> I certainly could write up a test routine. I will start work on that. In
> the meantime, before I put too much effort into it, I'd really like some
> feedback on what people want MLab.py to be, if they see using it at all.
> 
> As I said above, I'm not sure trying to give NumPy a MATLAB feel is a
> useful goal, so what is left is having all those handy functions.
> Perhaps we could shift away from MLab.py, and turn it into Utilities.py,
> which could use MATLAB (and other user experience) as guidance as to
> what utilities to include.
> 
> I'd really like folks' feedback on this before I put too much work into
> something no one uses, or will use.

i use MLab.py fairly often but see it more as a collection of useful
functions. personally, i agree with moving it into Utilities.py, as that
is in line with how i use it.

otoh, MLab.py does provide a nice entry point for folks coming from
Matlab into numpy. so, perhaps any eventual move to Utilities.py could
be accompanied with some documentation to remind users of those
linkages.

i'd offer to help with the testing, but my copy of matlab is back in the
states (i'm in australia on sabbatical) so i'd have to go from memory
(dangerous). let me know if i can help you.

cheers,
serge

-- 
Sergio J. Rey	http://typhoon.sdsu.edu/rey.html
"NT is secure.... as long as you don't remove the shrink wrap." - G. Myers



From oliphant.travis at ieee.org  Tue Aug 28 15:42:17 2001
From: oliphant.travis at ieee.org (Travis Oliphant)
Date: Tue, 28 Aug 2001 19:42:17 +0000
Subject: [Numpy-discussion] one-offset arrays
In-Reply-To: <20010828162422.A560@holler.physics.ubc.ca>
References: <20010828162422.A560@holler.physics.ubc.ca>
Message-ID: <01082819421703.09959@travis>

On Tue, 28 Aug 2001, Eric Nodwell wrote:
> Does anyone have a simple method of using one-offset arrays?
> Specifically, can I define an array "a" so that a[1] refers to the
> first element?
>

Inherit from the UserArray and redefine slicing to your hearts content.

> Without one-offset indexing, it seems to me that Python is minimally
> useful for numerical computations. Many, perhaps the majority, of
> numerical algorithms are one-indexed. Matlab for example is one-based
> for this reason.  In fact it seems strange to me that a "high-level"
> language like Python should use zero-offset lists.
>

Wow, that is a pretty condescending-sounding statement, though I'm sure you 
didn't mean it that way.  Python is indeed being used for quite serious 
numerical computations.  I have been using Python for quite some time for 
Numerical work and find it's zero-based indexing combined with the 
leave-last-one-out slicing notation to be much more convenient.  

It is different from MATLAB but that hardly makes it less useful as I suspect 
you'd agree after trying it out for awhile.

What is your application domain that so requires 1-based indexing.

Travis E. Oliphant, Ph.D.
Assistant Professor
Brigham Young University
Provo, UT 84602
(801) 378-3108
oliphant.travis at ieee.org



From oliphant.travis at ieee.org  Tue Aug 28 16:09:30 2001
From: oliphant.travis at ieee.org (Travis Oliphant)
Date: Tue, 28 Aug 2001 20:09:30 +0000
Subject: [Numpy-discussion] MLab.py status: do you use it?
In-Reply-To: <3B8BE84A.6CF21320@home.net>
References: <000701c12f3b$c0605840$0301a8c0@plstn1.sfba.home.com> <3B8BE84A.6CF21320@home.net>
Message-ID: <01082820093006.09959@travis>

On Tue, 28 Aug 2001, Chris Barker wrote:
> Hi all,
>
> Paul recently requested help with testing and debugging MLab.py, and I
> offered to help out. He then sent me this note:
>
> "Paul F. Dubois" wrote:
> > MLab.py started out as a package that was supposed to give numpy a
> > Matlab look and feel. So useful questions are: are there really useful
> > things that should be added?

I use many of the functions in MLab regularly.  I have no problem renaming it 
to Utility.py or even incorporating it back into some other module.

scipy uses MLab currently has well.

>
> I can't answer these questions myself. Interestingly enough, despite the
> fact that I came to NumPy having used MATLAB heavily for five years (and
> one dissertation), I have made very little use of MLab.py. I'm wondering
> if anyone is, indeed, using it.

Yes, we are...

>
> As for trying to give numpy a MATLAB look and feel, I question the
> usefulness of that. NumPy looks and feels a little different than
> MATLAB, and, frankly, I like NumPy's feel better for the most part (now
> that rich comparisons have been added, anyway). The one thing that
> MATLAB has that NumPy doesn't, that I really miss, is list indexing.
> Having to use put and take is so much less elegant! I'm sure this isn't
> the least bit trivial to add, however. (is it even possible, given
> Python's idea of slicing?)

I don't think look and feel is the right word --- but increasing the number 
of easily accesible utility routines is what the goal of SciPy is all about.

List indexing is difficult to add to the C-code due to the complexity of that 
particular subroutine.  Both Paul and I have looked at the code to try and 
add this feature but neither one of us had the time to make it work right.  I 
think it would be possible if we could ever agree on what list-indexing 
should do exactly for multidimensional arrays.    

Python's idea of slicing is actually quite general.  It takes 
a[obj1, obj2, obj3]  and essentially calls  the (C-equivalent) of __getitem__ 
(the mapping interface) with the tuple (obj1, obj2, obj3).  Any objects which 
look like (3:10:2)  are translated to a slice-object.  So, the objects could 
easily be indices into the array or a mask array of ones and zeros. 

I seem to remember that there were some "difficult-to-decide" cases but I 
can't recall them at the moment.

>
> Anyway, what I do think MATLAB does provide, and MLab.py should, is a
> whole bunch of utility functions for various common manipulations.
>
> > Do we really have what we have correct?

I'm pretty confident in it (that's not a life-staking claim...)
Most of my utility-development work is taking place in SciPy now.  Bugs will 
get fixed in MLab.py since Scipy uses it.

> >
> > Most of all, these is no test routine for it. If you could make one
> > following the model of the test.py in directory Test, it would be great.
> > The idea is to have something that does not require the runner of the
> > test to know anything. It just runs silently unless something is wrong.
>
> I certainly could write up a test routine. I will start work on that. In
> the meantime, before I put too much effort into it, I'd really like some
> feedback on what people want MLab.py to be, if they see using it at all.
>

It's a good idea.

> As I said above, I'm not sure trying to give NumPy a MATLAB feel is a
> useful goal, so what is left is having all those handy functions.
> Perhaps we could shift away from MLab.py, and turn it into Utilities.py,
> which could use MATLAB (and other user experience) as guidance as to
> what utilities to include.

I've got some fun suggestions for utility functions which I've just been 
storing away.  Some of them are put into SciPy.





From europax at home.com  Tue Aug 28 22:54:01 2001
From: europax at home.com (Rob)
Date: Tue, 28 Aug 2001 19:54:01 -0700
Subject: [Numpy-discussion] invert() with complex arrays
Message-ID: <3B8C5949.1F9C0531@home.com>

Is there any way to get invert() to work with complex arrays?

Thanks,  Rob.

-- 
The Numeric Python EM Project

www.members.home.net/europax



From milgram at cgpp.com  Tue Aug 28 23:28:34 2001
From: milgram at cgpp.com (J. Milgram)
Date: Tue, 28 Aug 2001 23:28:34 -0400
Subject: [Numpy-discussion] cephes-1.3 install problem
In-Reply-To: Your message of "Tue, 28 Aug 2001 20:09:30 -0000."
             <01082820093006.09959@travis> 
Message-ID: <200108290328.f7T3SYP00391@cgpp.com>


Hi,

this is surely a FAQ so I apologize in advance, but:

I'm trying to install cephes-1.3 and getting this:

cc -O2 -march=pentium -I/usr/local/include/python2.0 -I/usr/local/include/python2.0/Numeric    -c amos_wrappers.c -o amos_wrappers.o
In file included from cephes/mconf.h:162,
                 from amos_wrappers.h:11,
                 from amos_wrappers.c:8: 
cephes/protos.h:67: parse error before `sizeof'
cephes/protos.h:68: parse error before `sizeof'
cephes/protos.h:69: parse error before `sizeof'
make: *** [amos_wrappers.o] Error 1

Am installing with:

Python 2.0 (#14, Aug  2 2001, 10:06:36) 
[GCC egcs-2.91.66 19990314/Linux (egcs-1.1.2 release)] on linux2

and have already installed:

Numeric-20.1.0

grateful for any help ... I did search a bit and it seems to be a
known problem but I couldn't figure out the answer. Surely something
simple.

thanks

Judah

Judah Milgram        milgram at cgpp.com
P.O. Box 8376, Langley Park, MD 20787
(301) 422-4626 (-3047 fax)





From nodwell at physics.ubc.ca  Wed Aug 29 01:00:50 2001
From: nodwell at physics.ubc.ca (Eric Nodwell)
Date: Tue, 28 Aug 2001 22:00:50 -0700
Subject: [Numpy-discussion] one-offset arrays
Message-ID: <20010828220046.A4161@holler.physics.ubc.ca>

>> Does anyone have a simple method of using one-offset arrays?
>> Specifically, can I define an array "a" so that a[1] refers to the
>> first element?
>>

> Inherit from the UserArray and redefine slicing to your hearts content.

>> Without one-offset indexing, it seems to me that Python is minimally
>> useful for numerical computations. Many, perhaps the majority, of
>> numerical algorithms are one-indexed. Matlab for example is one-based
>> for this reason.  In fact it seems strange to me that a "high-level"
>> language like Python should use zero-offset lists.
>>

> Wow, that is a pretty condescending-sounding statement, though I'm sure you 
> didn't mean it that way.  Python is indeed being used for quite serious 
> numerical computations.  I have been using Python for quite some time for 
> Numerical work and find it's zero-based indexing combined with the 
> leave-last-one-out slicing notation to be much more convenient.  

Oops, what I really need is a wet-ware (i.e. brain) macro which
enforces the correct order of the pair (think,write,send)! The above
unconsidered comments arose from the sequence
(frustration,write,send,think,regret).  ;)

Obviously there is a lot of numerical work being done with python and
people are very happy with it. But for me I still think it would be
"minimally useful" without 1-indexed arrays. Here's why:

In my view, the most important reason to prefer 1-based indexing
versus 0-based indexing is compatibility.  For numerical work, some of
the languages which I use or have used are Matlab, Mathematica, Maple
and Fortran. These are all 1-indexed. (C is by nature 0-indexed
because it is so close to machine architecture, but with a little bit
of not-entirely-clean pointer manipulation, you can easily make
1-indexed arrays and matrices.) Obviously Python can't be compatible
with these languages in a strict sense, but like most people who do
some programming work, I've built up a library of my own commonly used
routines specific to my work; in general it's a trivial matter to
translate numerical routines from one language to the another if
translation is just a matter of substituting of one set of syntactical
symbols and function names for anther.  However it can be damn tricky
to convert 1-indexed code to 0-indexed code or visa versa without
introducing any errors- believe me!  (Yes it's possible to call nearly
any language from nearly any other language these days so in theory
you don't have to recode, but there are lots of reasons why often
recoding is the preferable route.)

The other reason for choosing 1-based indexing is to keep the code as
near to the standard notation as possible. This is one of the
advantages of using a high level language - it approximates the way
you think about things instead of the way the computer organizes
them. Of course, this can go either way depending on the quantity in
question: as a simple example a spatial vector (x,y,z) is
conventionally labelled 1,2,3 (1-indexed), but a relativistic
four-vector with time included (t,x,y,z) is conventionally labelled
0,1,2,3 (0-indexed). So ideally one would be able to choose the
indexing-type case-by-case. I'm sure that computer programmers will
argue vehemently that code which mixes both 0-indexed and 1-indexed
arrays is hard to understand and maintain, but for experts in a
particular field who are accustomed to certain ingrained notations, it
is the code which breaks the conventional notation which is most
error-prone. In my case, I'm dealing at the moment with crystal
structures with which are associated certain conventional sets of
vectors and tensors - all 1-indexed by convention. I find it a
delicate enough task to always get the correct vector or tensor
without having to remember that d[2] is actually d3. Defining d1,d2,d3
is not convenient because often the index itself needs to be
calculated.

I guess if I understood the reason for 0-indexed lists and tuples in
Python I would be happier. In normal, everyday usage, sets,
collections and lists are 1-indexed (the first item, the second item,
the third item, and so on). Python is otherwise such an elegant and
natural language. Why the ugly exception of making the user conform to
the underlying mechanism of an array being an address plus an offset?

All this is really neither here nor there, since this debate, at least
as far as Python is concerned, was probably settled 10 years ago and
I'm sure nobody wants to hear anything more about it at this point.
As you point out, I can define my own array type with inheritance. I
will also need my own range command and several other functions which
haven't occured to me yet. I was hoping that there would be a standard
module to implement this.


By the way, what is leave-last-one-out slicing? Is it
a[:-1]
or is it
a[0,...]
or is it something else?


Eric



From oliphant.travis at ieee.org  Tue Aug 28 19:30:54 2001
From: oliphant.travis at ieee.org (Travis Oliphant)
Date: Tue, 28 Aug 2001 23:30:54 +0000
Subject: [Numpy-discussion] one-offset arrays
In-Reply-To: <20010828220046.A4161@holler.physics.ubc.ca>
References: <20010828220046.A4161@holler.physics.ubc.ca>
Message-ID: <01082823305401.12637@travis>

> In my view, the most important reason to prefer 1-based indexing
> versus 0-based indexing is compatibility.  For numerical work, some of
> the languages which I use or have used are Matlab, Mathematica, Maple
> and Fortran. These are all 1-indexed. (C is by nature 0-indexed
> because it is so close to machine architecture, but with a little bit
> of not-entirely-clean pointer manipulation, you can easily make
> 1-indexed arrays and matrices.) Obviously Python can't be compatible
> with these languages in a strict sense, but like most people who do
> some programming work, I've built up a library of my own commonly used
> routines specific to my work; in general it's a trivial matter to
> translate numerical routines from one language to the another if
> translation is just a matter of substituting of one set of syntactical
> symbols and function names for anther.  However it can be damn tricky
> to convert 1-indexed code to 0-indexed code or visa versa without
> introducing any errors- believe me!  (Yes it's possible to call nearly
> any language from nearly any other language these days so in theory
> you don't have to recode, but there are lots of reasons why often
> recoding is the preferable route.)

You aren't the first to raise this issue.  I wouldn't mind if the user had 
the option, but then I again I tend to prefer the flag-for-every-feature 
approach which others who have more computing experience than me have said 
leads to problems due to the presence of many different ways to do things and 
unforseen interaction.s

I could definitely see the coding advantage in dealing with implementing 
algorithms that uses notation that is already 1-based.  I have come across 
this myself -- in fact just yesterday when I was coding up the Pade 
approximation to the matrix exponential using the pseudo-code algorithm given 
by Golub and Van Loan in their "Matrix Computations" book.

It seems to me like it would be a lot of work to add this feature back into 
the code now (there would be a million places to look for places where the 
code inherently assumes 0-based indexing).  It would also, as you mention, be 
inconsistent with Python.

A general approach would be to inherit from the UserArray for your codes and 
reimplement the __getitem__  and __getslice__ commands.  Your objects should 
still be able to be passed to many of the routines which expect arrays 
(because under the covers one of the first things the array_from_object 
C-code does is check to see if the  object has an __array__ method and calls 
it).  Note that this will not copy data around so there is minimal overhead.

But, you would have to take care to wrap the returned object back into an 
array_object.  (Maybe something could be done here...Hmmm.)

>
> By the way, what is leave-last-one-out slicing? Is it
> a[:-1]
> or is it
> a[0,...]
> or is it something else?

I meant the fact that a[3:6] returns elements a[3], a[4], a[5]  but NOT a[6]. 
I'm sorry for using my own poorly-worded term.  I can't remember what other 
Pythonistas call it.



From oliphant.travis at ieee.org  Tue Aug 28 19:38:17 2001
From: oliphant.travis at ieee.org (Travis Oliphant)
Date: Tue, 28 Aug 2001 23:38:17 +0000
Subject: [Numpy-discussion] one-offset arrays
In-Reply-To: <20010828220046.A4161@holler.physics.ubc.ca>
References: <20010828220046.A4161@holler.physics.ubc.ca>
Message-ID: <01082823381702.12637@travis>

Eric brought up the point of one-offset arrays which can be relatively easily 
created with UserArrays.

This led me to some thinking about why UserArrays are not more often used.

I think one of the biggest reasons is that most of the functions can take 
UserArrays but returned the basic array type upon completion.  So, you end up 
having to continually construct your UserArrays.

Are there other reasons people have thought of?

So, here's a suggestion:

Why don't we modify PyArray_Return to return an object of the same class as 
one of the arguments which was passed if the class defines an __array__ 
method. 

Which argument to pick and how this would be implemented without changing old 
code is an interesting question.

Assuming it were possible to cause PyArray_Return to do such a thing, would 
this be a bad idea?

Sincrely,

Tried-to-use-Matrix-objects-but-always-resort-to-dot(x,x)

Travis Oliphant



From hinsen at cnrs-orleans.fr  Wed Aug 29 05:38:16 2001
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Wed, 29 Aug 2001 11:38:16 +0200
Subject: [Numpy-discussion] invert() with complex arrays
In-Reply-To: <3B8C5949.1F9C0531@home.com> (message from Rob on Tue, 28 Aug
	2001 19:54:01 -0700)
References: <3B8C5949.1F9C0531@home.com>
Message-ID: <200108290938.f7T9cGe11697@chinon.cnrs-orleans.fr>

> Is there any way to get invert() to work with complex arrays?

If you mean LinearAlgebra.inverse(), it should work with complex arrays,
although I haven't checked that for a long time.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------



From hinsen at cnrs-orleans.fr  Wed Aug 29 06:01:10 2001
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Wed, 29 Aug 2001 12:01:10 +0200
Subject: [Numpy-discussion] one-offset arrays
In-Reply-To: <20010828220046.A4161@holler.physics.ubc.ca> (message from Eric
	Nodwell on Tue, 28 Aug 2001 22:00:50 -0700)
References: <20010828220046.A4161@holler.physics.ubc.ca>
Message-ID: <200108291001.f7TA1AE14088@chinon.cnrs-orleans.fr>

> versus 0-based indexing is compatibility.  For numerical work, some of
> the languages which I use or have used are Matlab, Mathematica, Maple
> and Fortran. These are all 1-indexed. (C is by nature 0-indexed
> because it is so close to machine architecture, but with a little bit

I don't think this is a matter if numerics vs. machine orientation, it
is just an historical accident. Having used 0-based and 1-based
languages extensively, I find both perfectly suitable for numerical
work, and in my experience 0-based indexing leads to shorter index
expressions in many cases.

> symbols and function names for anther.  However it can be damn tricky
> to convert 1-indexed code to 0-indexed code or visa versa without
> introducing any errors- believe me!  (Yes it's possible to call nearly

Having done so recently, I agree :-(

> The other reason for choosing 1-based indexing is to keep the code as
> near to the standard notation as possible. This is one of the

I wouldn't call 1-based indexing "standard" - it all depends on your
background.

> question: as a simple example a spatial vector (x,y,z) is
> conventionally labelled 1,2,3 (1-indexed), but a relativistic
> four-vector with time included (t,x,y,z) is conventionally labelled
> 0,1,2,3 (0-indexed). So ideally one would be able to choose the

So, when you implement classes that represent vectors, you should
stick to whatever notation is common in the application domain.

I see bare-bones arrays as we have them in NumPy as a low-level
building block for high-level classes, i.e. not as a data type one
should use directly in non-trivial application code. It's not just
indexing, also operations. For your four-vectors, you would want a
method that calculates the norm in the Einstein metric, for example.
If you don't have a special four-vector class but use arrays directly,
any length calculation will look messy. That's what OO design is good
for. In fact, with special four-vector classes and appropriate
methods, there will hardly be any explicit indices in the application
code at all!

> indexing-type case-by-case. I'm sure that computer programmers will
> argue vehemently that code which mixes both 0-indexed and 1-indexed
> arrays is hard to understand and maintain, but for experts in a

Yes, if they are supposed to be the same data type. If different
classes use different indexing schemes, everything remains clear,
as long as the interactions between the classes are well defined.

> error-prone. In my case, I'm dealing at the moment with crystal
> structures with which are associated certain conventional sets of
> vectors and tensors - all 1-indexed by convention. I find it a

Have a look at the vector and tensor classes in Scientific Python
(http://dirac.cnrs-orleans.fr/programs/scientific.html). Although they
are 0-based as well (because I got so used to that!), you might find
that the predefined methods cover most of your needs and that explicit
indices are hardly needed. But you are also free to modify the
__getitem__ and __setitem__ methods to make the classes 1-based.

> I guess if I understood the reason for 0-indexed lists and tuples in
> Python I would be happier. In normal, everyday usage, sets,

Python has a very consistent indexing scheme, which you can best
understand in the following way: imagine that the indices are pointers
to positions *between* the elements:

elements           a   b   c   d

index            0   1   2   3   4
negative index  -4  -3  -2  -1

Then all the slicing and indexing rules make a lot of sense, and
fulfill practically useful relations. For example, it is true that

     x[a:b] + x[b:c]  equals  x[a:c]

for any list/array x, which would not be the case with 1-based
indices. Adding indices is also more straightforward with base 0, all
those "i+j-1" you see so frequently in Fortran code becomes just
"i+j".

Sure, everyday language works differently, but then you don't write
algorithms in everyday language, for a good reason.

> By the way, what is leave-last-one-out slicing? Is it
> a[:-1]

That one, as is clear from my little picture above.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------



From hinsen at cnrs-orleans.fr  Wed Aug 29 06:06:25 2001
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Wed, 29 Aug 2001 12:06:25 +0200
Subject: [Numpy-discussion] one-offset arrays
In-Reply-To: <01082823381702.12637@travis> (message from Travis Oliphant on
	Tue, 28 Aug 2001 23:38:17 +0000)
References: <20010828220046.A4161@holler.physics.ubc.ca> <01082823381702.12637@travis>
Message-ID: <200108291006.f7TA6PM14091@chinon.cnrs-orleans.fr>

> This led me to some thinking about why UserArrays are not more often used.
> 
> I think one of the biggest reasons is that most of the functions can
> take UserArrays but returned the basic array type upon completion.
> So, you end up having to continually construct your UserArrays.

Exactly.

> Are there other reasons people have thought of?

Performance, in some cases. Or passing objects to C routines that expect
plain arrays. But the function issue is certainly the major one.

> So, here's a suggestion:
> 
> Why don't we modify PyArray_Return to return an object of the same class as 
> one of the arguments which was passed if the class defines an __array__ 
> method. 

I don't think it can be done at the level of PyArray_Return, it cannot
know what the classes of the arguments were. We *could* achieve the
same goal by making appropriate modifications to all the C functions
in Numeric.

> Which argument to pick and how this would be implemented without
> changing old code is an interesting question.

No idea about the first issue, but compatibility can be assured (well,
in most cases) by picking some attribute name (__array__) that is not
used in the current UserArray class. We'd then have some other class
derived from UserArray which sets that attribute.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------



From europax at home.com  Wed Aug 29 10:09:06 2001
From: europax at home.com (Rob)
Date: Wed, 29 Aug 2001 07:09:06 -0700
Subject: [Numpy-discussion] invert() with complex arrays
References: <3B8C5949.1F9C0531@home.com> <200108290938.f7T9cGe11697@chinon.cnrs-orleans.fr>
Message-ID: <3B8CF782.68C56084@home.com>

Konrad Hinsen wrote:
> 
> > Is there any way to get invert() to work with complex arrays?
> 
> If you mean LinearAlgebra.inverse(), it should work with complex arrays,
> although I haven't checked that for a long time.
> 
> Konrad.
> --
> -------------------------------------------------------------------------------
> Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
> Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
> Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
> 45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
> France                                   | Nederlands/Francais
> -------------------------------------------------------------------------------
> 
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> http://lists.sourceforge.net/lists/listinfo/numpy-discussion


I tested a small matrix and got an error, but I will try again, since
with my luck it was singular :)

Rob.
-- 
The Numeric Python EM Project

www.members.home.net/europax



From hinsen at cnrs-orleans.fr  Wed Aug 29 10:13:52 2001
From: hinsen at cnrs-orleans.fr (Konrad Hinsen)
Date: Wed, 29 Aug 2001 16:13:52 +0200
Subject: [Numpy-discussion] invert() with complex arrays
In-Reply-To: <3B8CF782.68C56084@home.com> (message from Rob on Wed, 29 Aug
	2001 07:09:06 -0700)
References: <3B8C5949.1F9C0531@home.com> <200108290938.f7T9cGe11697@chinon.cnrs-orleans.fr> <3B8CF782.68C56084@home.com>
Message-ID: <200108291413.f7TEDq105660@chinon.cnrs-orleans.fr>

> > > Is there any way to get invert() to work with complex arrays?
> > 
> > If you mean LinearAlgebra.inverse(), it should work with complex arrays,
> > although I haven't checked that for a long time.
>
> I tested a small matrix and got an error, but I will try again, since
> with my luck it was singular :)

No need to take chances:

   >>> from Numeric import *
   >>> from LinearAlgebra import inverse
   >>> a = 1.j*array([[1., 0.], [0., 1.]])
   >>> a
   array([[ 0.+1.j,  0.+0.j],
	  [ 0.+0.j,  0.+1.j]])
   >>> inverse(a)
   array([[ 0.-1.j,  0.+0.j],
	  [ 0.+0.j,  0.-1.j]])

So it works for at least one complex matrix. :-)

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------



From chrishbarker at home.net  Wed Aug 29 13:41:57 2001
From: chrishbarker at home.net (Chris Barker)
Date: Wed, 29 Aug 2001 10:41:57 -0700
Subject: [Numpy-discussion] one-offset arrays
References: <20010828220046.A4161@holler.physics.ubc.ca>
Message-ID: <3B8D2965.18095B10@home.net>

It's mostly been said already, but I can't help but add my $.02
sometimes...

Eric Nodwell wrote:

> >> Without one-offset indexing, it seems to me that Python is minimally
> >> useful for numerical computations. Many, perhaps the majority, of
> >> numerical algorithms are one-indexed. Matlab for example is one-based
> >> for this reason.  In fact it seems strange to me that a "high-level"
> >> language like Python should use zero-offset lists.

I was a heavy user of MATLAB for a long time before I discovered NumPy,
and I have to say that I like the 0-indexing scheme MUCH better!

> In my view, the most important reason to prefer 1-based indexing
> versus 0-based indexing is compatibility.  For numerical work, some of
> the languages which I use or have used are Matlab, Mathematica, Maple
> and Fortran. These are all 1-indexed.

Actually Fortran is indexed however you decide you want it:

DIMENSION array(0:9)

DIMENSION array(1:10) or DIMENSION array(10)

DIMENSION array(1900:1999)

Are all legal. This is a VERY handy feature, and I would say that I used
the 0-indexed version most often. The reason is related to C's pointer
arithmetic logic: Often the array would represent discrete points on a
continuous scale, so I could find the value of X, for instance, by
doing:

Xaxis(i) = MinX * i * DeltaX

with i-indexing, you have to subtract 1 all the time.

I suspect that the higher level nature of NumPy would make it a lot
harder to have arbitrary indexing of this fashion: if all you have to do
is access elements, it is easy, but if you have a whole collection of
array oriented operations, as NumPy does, you would probably have to
stick with one standard, and I think the 0-indexing standard is the
best.

> for experts in a
> particular field who are accustomed to certain ingrained notations, it
> is the code which breaks the conventional notation which is most
> error-prone.

This is why being able to set your own indexing notation is the best
option, but a difficult one to impliment.

> Python is otherwise such an elegant and
> natural language. Why the ugly exception of making the user conform to
> the underlying mechanism of an array being an address plus an offset?

I gave an example above, and others have too: Python's indexing scheme
is elegant and natural for MANY usages. As with many things Python
(indentation, anyone!), I found it awkward to make the transition at
first, but then found that it, in fact, made things easier in general.
For me, this is the very essence of truly usable design: it is designed
to make people most productive in the long run, not most comfortable
when they first start using it. 
 
> All this is really neither here nor there, since this debate, at least
> as far as Python is concerned, was probably settled 10 years ago and

Well, yes, and having NumPy different from the rest of Python would NOT
be a good idea either.

> I'm sure nobody wants to hear anything more about it at this point.
> As you point out, I can define my own array type with inheritance. I
> will also need my own range command and several other functions which
> haven't occured to me yet. I was hoping that there would be a standard
> module to implement this.

If it were truly generally useful, there probably would be such a
package. I imagine most people have found it easier to make the
transition than to write a whole package that would allow you not to
make the transition. If you really have a lot of code that is 1-indexed
that you want to translate, it may be worth the effort for you, and I'm
sure there are other folks that would find it useful, but remember that
it will always be incompatable with the rest of Python, which may make
it harder to use than you imagine. 

-Chris

-- 
Christopher Barker,
Ph.D.                                                           
ChrisHBarker at home.net                 ---           ---           ---
http://members.home.net/barkerlohmann ---@@       -----@@       -----@@
                                   ------@@@     ------@@@     ------@@@
Oil Spill Modeling                ------   @    ------   @   ------   @
Water Resources Engineering       -------      ---------     --------    
Coastal and Fluvial Hydrodynamics --------------------------------------
------------------------------------------------------------------------



From chrishbarker at home.net  Wed Aug 29 17:10:19 2001
From: chrishbarker at home.net (Chris Barker)
Date: Wed, 29 Aug 2001 14:10:19 -0700
Subject: [Numpy-discussion] API tricks any my new "clip" function
References: <20010828162422.A560@holler.physics.ubc.ca> <01082819421703.09959@travis>
Message-ID: <3B8D5A3B.6F5A8E09@home.net>

Hi all,

I wrote last week looking for hints for writing a new clip function,
that would be faster than the existing one. I got some suggestions, and
have written the function. It works, but I'd love to get a little code
review from the more experienced folks here. I have a few specific
questions:

A) Are there any tricks to making a function work with multiple types? I
currently have it working with only double arrays, but it would be great
for it ot be universal (and it could then be added to the main NumPy
distribution, I suppose) I seem tohave to typecast the data to do the
comparison I want, so I can't figure out how to make it general. For
example, I have this line:

if (*(double *)(Array->data + i*elementsize) > *(double *)(Max->data +
i*elementsize))
{
          *(double *)(Array->data + i*elementsize) = *(double
*)(Max->data + i*elementsize) ;
}

How could I do that without the explicit typecasts? I'm pretty sure I
could make the rest of the routine general.

B) I was trying to figure out how to loop through all the elements of a
non-contiguous array. I got two hints:
"Rob W. W. Hooft" wrote:

> Never done this in C for numpy yet, but I normally program this
> along the following lines:
> 
> n=[2,3,2,3]
> x=[0,0,0,0]
> while 1:
>     print x
>     i=len(n)-1
>     while i>=0 and x[i]+1==n[i]:
>        x[i]=0
>        i=i-1
>     if i<0:
>        break
>     x[i]=x[i]+1
> 
> It is always going to be slower than the straight loop for contiguous
> arrays. 

This is what I did, and it is a LOT slower (about a factor of ten). See
the enclosed C code. There has got to be a way to optimize this!

>So if you want to do it properly, you'd have to check whether
> the array is contiguous, and if it is, use the fast way.

Since I had already written code for the contiguous case, it was easy to
special case it.

Pete Shinners wrote:
> it's easiest to use recursion than nested loops. then you can
> support any number of dimensions. that should point you in the
> right direction :]

When I read this, I thought: "well, duh!", but then I couldn't figure
out how to do it. Would there be any advantage over the above loops
anyway?

Is there either a more elgant or faster way to loop through all the
elements of these three arrays?

C) Where is the DL_EXPORT macro defined? I havn't been able to find it
with grep yet. Does it work on all NumPy supported platforms?

D) Is there an exising function or Macro for checking if two arrays are
the same shape? I wrote one, but I imagine it exists.

F) I needed to work with input that could be any Python number, or an
Array of the right size. What I did was call: PyArray_FromObject on the
min and max inputs, so that I would get a PyArray, then I could check if
it was a rank-o array, or the right size. I havn't been able to decide
if this is a nifty trcik, or a kludge. Any other suggestions?

E) Have I got reference counting right? I tried to Py_DECREF all my temp
arrays, but I'm not sure I did it right.

Sorry to enclose the code if you have no interest in it, but it's not
really that big.

-Chris

-- 
Christopher Barker,
Ph.D.                                                           
ChrisHBarker at home.net                 ---           ---           ---
http://members.home.net/barkerlohmann ---@@       -----@@       -----@@
                                   ------@@@     ------@@@     ------@@@
Oil Spill Modeling                ------   @    ------   @   ------   @
Water Resources Engineering       -------      ---------     --------    
Coastal and Fluvial Hydrodynamics --------------------------------------
------------------------------------------------------------------------
-------------- next part --------------
#include "Python.h"
#include "arrayobject.h"

// These are little macros I use to access array values for specific rank arrays
#define ARRAYVAL0(aType,a) ( *(aType *)(a->data))	
#define ARRAYVAL1(aType,a,i) ( *(aType *)(a->data + (i)*a->strides[0]))	
#define ARRAYVAL2(aType,a,i,j) ( *(aType *)(a->data + (i)*a->strides[0] + (j)*a->strides[1]))	
#define ARRAYVAL3(aType,a,i,j,k) ( *(aType *)(a->data + (i)*a->strides[0] + (j)*a->strides[1] + (k)*a->strides[2]))	


int AreSameShape(PyArrayObject *Array1, PyArrayObject *Array2){

  int i;
  
  if (Array1->nd != Array2->nd){
    return 0;
  }
  for (i = 0; i < Array1->nd; i++){
    if (Array1->dimensions[i] != Array2->dimensions[i]){
      return 0;
    }
  }
  return 1;
}


int ComputeOffset(PyArrayObject *Array, int *indexes){

  int i, offset = 0;

  for (i = 0; i < Array->nd; i++)
    {
      offset += indexes[i]*Array->strides[i];
    }
  return offset;
}

void ReplaceValue(PyArrayObject *Array, PyArrayObject *Min, PyArrayObject *Max, int *indexes){
  // This function does the actual replacement of the values.
  // it allows Min and Max to be one element, or the same shape as Array
  // the offsets are computed separately, so as long as they are the same shape
  // it should work even if some are contiguous, and some not, for example

  double min, max;

  int offset;
    
  offset = ComputeOffset(Array,indexes);

  if (PyArray_Size((PyObject *)Min) == 1){
    min = ARRAYVAL0(double,Min);
  }
  else {
    min = *(double *)(Min->data + ComputeOffset(Min,indexes));
  }
  if (PyArray_Size((PyObject *)Max) == 1){
    max = ARRAYVAL0(double,Max);
  }
  else {
    max = *(double *)(Max->data + ComputeOffset(Max,indexes));
  }
  if (*(double *)(Array->data + offset) > max){
    *(double *)(Array->data + offset) = max ;
  }
  else if (*(double *)(Array->data + offset) < min){
    *(double *)(Array->data + offset) = min;
  }
  return;
}

static char doc_fastclip[] =
"fastclip(a,min,max):  a is clipped so that there are no values \n"
"less than min, or greater than max.\n\n"
"It only works on PyArrays of type Float.\n"
"min and max can be either scalar or the same size as a.\n"
"If a, min, and max are all contiguous (or scalar) then it is a LOT faster\n";

static PyObject * NumericExtras_fastclip(PyObject *self, PyObject *args)
{
  
  int NumArray, elementsize, i;
  int *indexes;

  PyObject *InMin;
  PyObject *InMax;

  PyArrayObject *Array;
  PyArrayObject *Min;
  PyArrayObject *Max;

  if (!PyArg_ParseTuple(args, "O!OO", 
						&PyArray_Type, &Array,
					    &InMin,
						&InMax))
    {
      return NULL;
    }  
  
  // check types of input

  if (Array->descr->type_num != PyArray_DOUBLE){
	PyErr_SetString (PyExc_ValueError,
					 "a must be a NumPy array of type Float");
    return NULL;
  }
  
  // convert min and max to double arrays:
  // if they don't convert, the input values are not valid
  // also check if they are the right size
  Min = (PyArrayObject *) PyArray_FromObject(InMin, PyArray_DOUBLE, 0, 0);
  if (Min == NULL){
    PyErr_SetString (PyExc_ValueError,
					 "min must be an object that can be converted to an array of Floats");
    return NULL;
  }

  if (!((PyArray_Size((PyObject *)Min) == 1) ||  AreSameShape(Min, Array))){
    PyErr_SetString (PyExc_ValueError,
					 "min must be either a scalar or the same size as a");
    Py_DECREF(Min);
    return NULL;
  }

  Max = (PyArrayObject *) PyArray_FromObject(InMax, PyArray_DOUBLE, 0, 0);
  if (Max == NULL){
    PyErr_SetString (PyExc_ValueError,
					 "max  must be an object that can be converted to an array of Floats");
    Py_DECREF(Min);
    return NULL;
  }

  if (!((PyArray_Size((PyObject *)Max) == 1) ||  AreSameShape(Max, Array))){
    PyErr_SetString (PyExc_ValueError,
					 "max must be either a scalar or the same size as a");
    Py_DECREF(Min);
    Py_DECREF(Max);
    return NULL;
  }

  // After all that input checking, switch between the contiguous and non-contiguous cases.
  if (PyArray_ISCONTIGUOUS(Array) && PyArray_ISCONTIGUOUS(Min) && PyArray_ISCONTIGUOUS(Max)){ //we have the contiguous case


    // This seems pretty kludgy to have the four cases, but it was the first
    // thing that came to mind that I was sure would work, and would accomidate
    // either array or scalar arguments for min and max. 
    // it's also faster than checking if they are scalar inside the loop

    NumArray = PyArray_Size((PyObject *)Array);
    elementsize = sizeof(double);

    if (PyArray_Size((PyObject *)Min) == 1 && PyArray_Size((PyObject *)Max) == 1){ // both limits are scalar
      for (i = 0; i < NumArray; i++) { // loop over each element
        if (*(double *)(Array->data + i*elementsize) > ARRAYVAL0(double,Max)){
          *(double *)(Array->data + i*elementsize) = ARRAYVAL0(double,Max);
        }
        else if (*(double *)(Array->data + i*elementsize) < ARRAYVAL0(double,Min)){
          *(double *)(Array->data + i*elementsize) = ARRAYVAL0(double,Min);
        }
      }
    }
    else if (PyArray_Size((PyObject *)Min) == 1 && PyArray_Size((PyObject *)Max) == NumArray){ // Min is scalar
      for (i = 0; i < NumArray; i++) { // loop over each element
        if (*(double *)(Array->data + i*elementsize) > *(double *)(Max->data + i*elementsize)){
          *(double *)(Array->data + i*elementsize) = *(double *)(Max->data + i*elementsize) ;
        }
        else if (*(double *)(Array->data + i*elementsize) < ARRAYVAL0(double,Min)){
          *(double *)(Array->data + i*elementsize) = ARRAYVAL0(double,Min);
        }
      }
    }    
    else if (PyArray_Size((PyObject *)Max) == 1 && PyArray_Size((PyObject *)Min) == NumArray){ // Max is scalar
      for (i = 0; i < NumArray; i++) { // loop over each element
        if (*(double *)(Array->data + i*elementsize) > ARRAYVAL0(double,Max)){
          *(double *)(Array->data + i*elementsize) = ARRAYVAL0(double,Max);
        }
        else if (*(double *)(Array->data + i*elementsize) < *(double *)(Min->data + i*elementsize)){
          *(double *)(Array->data + i*elementsize) = *(double *)(Min->data + i*elementsize) ;
        }
      }
    }    
    else if (PyArray_Size((PyObject *)Min) == NumArray && PyArray_Size((PyObject *)Max) == NumArray){ // Neither is scalar
      for (i = 0; i < NumArray; i++) { // loop over each element
        if (*(double *)(Array->data + i*elementsize) > *(double *)(Max->data + i*elementsize)){
          *(double *)(Array->data + i*elementsize) = *(double *)(Max->data + i*elementsize) ;
        }
        else if (*(double *)(Array->data + i*elementsize) < *(double *)(Min->data + i*elementsize)){
          *(double *)(Array->data + i*elementsize) = *(double *)(Min->data + i*elementsize) ;
        }
      }    
    }
    else { // The size of Min and/or Max don't match Array: we should never get here, do to previous checks.
      PyErr_SetString (PyExc_ValueError,
                       "min and max must be either scalar or the same shape as a");
      Py_DECREF(Min);
      Py_DECREF(Max);
      return NULL;
    }

  }
  else { // this now the non-contiguous case: it is much slower, but presumably could be optimised : suggestions gratefully excepted!
      
    indexes = calloc(Array->nd,sizeof(int));

    while (1){

      ReplaceValue(Array, Min, Max, indexes);

      i = (Array->nd) - 1;
      while ((i >= 0) && (indexes[i]+1 == Array->dimensions[i])){
        indexes[i] = 0;
        i -= 1;
      }
      if (i<0) {
        break;
      }
      indexes[i] = indexes[i]+1;
    } 

    free(indexes);
  }    
  Py_DECREF(Min);
  Py_DECREF(Max);
  return Py_None;
}

	
static PyMethodDef NumericExtrasMethods[] = {
  {"fastclip", NumericExtras_fastclip, METH_VARARGS, doc_fastclip},
  {NULL, NULL} /* Sentinel */
};

DL_EXPORT (void) initNumericExtras(){

  (void) Py_InitModule("NumericExtras", NumericExtrasMethods);
  import_array()
}





















From jsaenz at wm.lc.ehu.es  Wed Aug 29 17:16:19 2001
From: jsaenz at wm.lc.ehu.es (Jon Saenz)
Date: Wed, 29 Aug 2001 23:16:19 +0200 (MET DST)
Subject: [Numpy-discussion] API tricks any my new "clip" function
In-Reply-To: <3B8D5A3B.6F5A8E09@home.net>
Message-ID: <Pine.OSF.3.95.1010829230810.8194A-100000@lcdx00.wm.lc.ehu.es>

On Wed, 29 Aug 2001, Chris Barker wrote:

> A) Are there any tricks to making a function work with multiple types? I
> currently have it working with only double arrays, but it would be great
> for it ot be universal (and it could then be added to the main NumPy
> distribution, I suppose) I seem tohave to typecast the data to do the
> comparison I want, so I can't figure out how to make it general. For
> example, I have this line:
> 
> if (*(double *)(Array->data + i*elementsize) > *(double *)(Max->data +
> i*elementsize))
> {
>           *(double *)(Array->data + i*elementsize) = *(double
> *)(Max->data + i*elementsize) ;
> }
> 
> How could I do that without the explicit typecasts? I'm pretty sure I
> could make the rest of the routine general.
Some time ago, I used a union like:
typedef union {
short *sdata;
int *idata;
long *ldata;
float *fdata;
double *ddata; 
/* And so on */
} MyNumPyUnion;

Then, you can assign (and CAST) the data field of the NumPy array
according to the typecode of the array, like in:

MyNumPyUnion theunion;

theunion.fdata=(float*) thenumpyarray->data; /* If it is a Float32 */

Finally, you use  sdata, idata, ldata, fdsata and so on to iterate
dereferencing accordingly, depending on the typecode of the array.

It is a nightmare to write this code, but I can not imagine of any other
approach. May be other developers can help you with a better approach.

Hope this helps.


Jon Saenz.				| Tfno: +34 946012470
Depto. Fisica Aplicada II               | Fax:  +34 944648500
Facultad de Ciencias.   \\ Universidad del Pais Vasco \\
Apdo. 644   \\ 48080 - Bilbao  \\ SPAIN






From frankpit at erols.com  Wed Aug 29 23:43:27 2001
From: frankpit at erols.com (Bernard Frankpitt)
Date: Wed, 29 Aug 2001 23:43:27 -0400
Subject: [Numpy-discussion] cephes-1.3 install problem
References: <200108290328.f7T3SYP00391@cgpp.com>
Message-ID: <3B8DB65F.3010803@erols.com>

I came across this problem a few months ago, I hacked the indicated 
lines in what seemed,
at the time, to be the obvious fashion and the thing worked fine.  I'm 
sorry that I can't
be more helpful, but I haven't kept the file. The hack might just have 
been to comment out the offending statements.  

Bernie




From nodwell at physics.ubc.ca  Thu Aug 30 04:07:48 2001
From: nodwell at physics.ubc.ca (Eric Nodwell)
Date: Thu, 30 Aug 2001 01:07:48 -0700
Subject: [Numpy-discussion] one-offset arrays
Message-ID: <20010830010748.A28244@holler.physics.ubc.ca>

Thanks to everyone who responded to my comments about one-offset
arrays in Python.  I understand much better now why zero-offset arrays
are a good choice for Python.  For the reasons already discussed
(mostly ease of translating one-offset algorithms), one-offset arrays
would nevertheless be useful in certain situations, so I went ahead
and wrote a class for a one-offset array.

It ended up being a somewhat bigger job than I expected, and since it
is rather too much material for a mailing-list submission I created a
web site for it. Anyone who is interested in this topic please have a
look:

http://www.physics.ubc.ca/~mbelab/python/arrayone/index.html

I'd love some feedback along the following lines:

1. There are 4 outstanding problems/bugs which I could identify but
was unable to fix. Most of these seem to be due to limitations of the
UserArray class, but they could also be due to limitations of the
programmer. None of these problems make the class unusable. One of
them is the issue which Travis identified about functions taking
UserArrays but returning standard arrays. It would be nice if the
resulting discussion led to something be done... Any suggestions for
fixing the other 3 issues would be most welcome.

2. I would like this module to be as accessible as possible to others,
particularly those who are just getting started with Python and may
therefore be especially encumbered with one-offset baggage. How can I
do this?  Submit it to a Python resource such as the Vaults of
Parnassus? Get people to link to my web page? Does someone with a more
prominant website want to volunteer to host it? Is there any
possibility of having it included eventually with Numerical Python or
Scientific Python?  Probably some people will consider it too trivial
to include, but I found it less trivial than I expected once all the
cases were covered - and there are still the 4 outstanding
issues. (Yes the code sucks!  Suggestions for improvement are most
welcome!) Why make people reinvent the wheel? People coming from
MatLab for example might be inclined to give up and go back to MatLab.

Eric


P.S. In case anyone is interested in the outstanding problems but for
some reason doesn't want to or can't visit the web site, here are the
summaries:

Problem 1
---------

Description
for loops don't work with variables of type arrayone

Example

    X=arrayone((1,2,3))
    for item in X:
        print item
 This just generates an error.

Reason
In Python, a for loop works by starting at x(0) and incrementing until
an out-of-bounds error occurs. arrayone types have no 0 element.

Work-around:
Cast to type array in for loops. For example 
    X=arrayone((1,2,3))
    for item in array(X):
        print item


Possible solutions:
Is it possible to "overload" "for" so that it behaves differently for
arrayone type?


Problem 2
---------

Description
Slicing an arrayone from 0 doesn't generate an error but it should.

Example:

    X=arrayone((1,2,3))
    X[0:3]
 This returns ArrayOne.arrayone([1, 2, 3]) instead of an error.

Reason
X[:3] results in a call to __getslice__ with low=0. This cannot be
distinguished from X[0:3]. Therefore in order to deal correctly with
the X[:] case, we have to assume that low=0 means an unspecified low.

Work-around
If you don't trust your code, you have to implement specific checking
for this condition before slicing.

Possible solution
If it was possible to get access to the unparsed input (literally
'0:3' for example), this could be fixed.


Problem 3
---------

Description
Negative subscripts return a slice offset by -1 from the expected
result.

Example

    X=arrayone((1,2,3,4))
    X[-3:-2]
 This returns ArrayOne.arrayone([1, 2]) instead of
 ArrayOne.arrayone([2, 3]).

Reason
X[-3:-2] in the above example results in a call to __getslice__ with
low=1, high=2, which cannot be distinguished from X[1:2].

Work-around
Don't use negative index slicing with arrayone types.

Possible solution:
If had access to unparsed input, could fix this.

Problem 4
---------

Description
Functions which take arrayone arguments return type array.

Work-around
Have to cast back into arrayone, for example: 
    Y=arrayone(somefunc(X))

Possible solution
See http://www.geocrawler.com/lists/3/SourceForge/1329/0/6505926



From jsw at cdc.noaa.gov  Thu Aug 30 08:10:41 2001
From: jsw at cdc.noaa.gov (Jeff Whitaker)
Date: Thu, 30 Aug 2001 06:10:41 -0600 (MDT)
Subject: [Numpy-discussion] LinearAlgebra patch
Message-ID: <Pine.OSX.4.33.0108300603200.19262-100000@localhost.cdc.noaa.gov>

Has anyone tried my patch to LinearAlgebra.py and lapack_litemodule.c?
I've found major speedups (by a factor of 3-5) in all operations involving
an svd or a eigenanalysis of a symmetric matrix. Paul's initial comment
was that since it was not self-contained (it required linking external
blas and lapack libs) he could not consider putting it in.  I have
rectified that now by including new versions of f2c_lite.c, blas_lite.c
and d(z)lapack_lite.c that support the newer, faster lapack routines.

I would appreciate any reports on how it works (or doesn't) on other
platforms - I've tried it in on MacOS X and Solaris.

Grab it from the 'Patches' section of the sourceforge project page.


-Jeff

 --
Jeffrey S. Whitaker
Phone  : (303)497-6313 Meteorologist               FAX    : (303)497-6449
NOAA/OAR/CDC  R/CDC1        Email  : jsw at cdc.noaa.gov
325 Broadway                Web    : www.cdc.noaa.gov/~jsw
Boulder, CO, USA 80303-3328 Office : Skaggs Research Cntr 1D-124




From tim.hochberg at ieee.org  Thu Aug 30 10:44:07 2001
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Thu, 30 Aug 2001 07:44:07 -0700
Subject: [Numpy-discussion] one-offset arrays
References: <20010830010748.A28244@holler.physics.ubc.ca>
Message-ID: <042501c13162$393894d0$87740918@cx781526b>

Hi Eric,

Here are some comments on the problems you identify.

> Problem 1
> ---------
>
> Description
> for loops don't work with variables of type arrayone

Soon (Python 2.2?) you will be able to use the new iterator protocol to do
this correctly. So target the iterator protocol and this will begin to work
as people begin to upgrade.

> Problem 2
> ---------
>
> Description
> Slicing an arrayone from 0 doesn't generate an error but it should.

I was going to suggest removing __getslice__, and only using __getitem__.
However, the slices recieved by __getitem__ *sometimes* gets their first
value filled in with zero (in the same cases that __getslice__ is called)
instead of None. I regard this as a bug/misfeature and I just reported it on
sourceforge.


> Problem 3
> ---------
>
> Description
> Negative subscripts return a slice offset by -1 from the expected
> result.

This one _should_ be fixable just by removing __getslice__ and working with
__getitem__ only. (getitem is called instead of getslice). However it fails
for the same reason that this fix fails for problem 2. Mumble.

Note that both fixes work fine if you always include the second colon. One
approach would be to do all [] operations in getitem and have getslice raise
an error. This would force your users to always use a[-3:-2:] instead of
a[-2:-2]. The former can be made to work since it calls getitem with the raw
values [slice(-3,-2,None)] as opposed to munging them like the latter does.


-tim




From tim.hochberg at ieee.org  Thu Aug 30 11:07:48 2001
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Thu, 30 Aug 2001 08:07:48 -0700
Subject: [Numpy-discussion] one-offset arrays
References: <20010830010748.A28244@holler.physics.ubc.ca>
Message-ID: <043c01c13165$881c0480$87740918@cx781526b>

This is just a followup to my earlier message.

The getitem issues I identify there are fixed in Python 2.2, if you use new
style class (inherit from object). So, problems 1-3 can be fixed in Python
2.2. It seems that problem 4 could be fixed fairly easily in the python
layer by writing wrapper functions in Numeric.py. For example:

def cos(a):
   r = umath.cos(a)
   if isinstance(a, UserArray):
      r = a.__class__(r)
   return r

This would be potentially a bit tedious, but straightforward.

-tim




From chrishbarker at home.net  Thu Aug 30 13:12:59 2001
From: chrishbarker at home.net (Chris Barker)
Date: Thu, 30 Aug 2001 10:12:59 -0700
Subject: [Numpy-discussion] LinearAlgebra patch
References: <Pine.OSX.4.33.0108300603200.19262-100000@localhost.cdc.noaa.gov>
Message-ID: <3B8E741B.2A1073A6@home.net>

Jeff Whitaker wrote:
> Paul's initial comment
> was that since it was not self-contained (it required linking external
> blas and lapack libs) he could not consider putting it in.  I have
> rectified that now by including new versions of f2c_lite.c, blas_lite.c
> and d(z)lapack_lite.c that support the newer, faster lapack routines.

I don't know what it would take, but I would LOVE it if NumPy used a
highly optimised BLAS wherever possible. I have no idea whether there is
a good BLAS available for every major platform that has a licence that
allows it to be included with NumPy, but if so, I think it should be
included. I know this is more effor than just including a generic one,
but having optimum performace "out of the box" would be a great thing.

just my $.02, which is worth about that, as I am in littel position to
help...


-Chris



-- 
Christopher Barker,
Ph.D.                                                           
ChrisHBarker at home.net                 ---           ---           ---
http://members.home.net/barkerlohmann ---@@       -----@@       -----@@
                                   ------@@@     ------@@@     ------@@@
Oil Spill Modeling                ------   @    ------   @   ------   @
Water Resources Engineering       -------      ---------     --------    
Coastal and Fluvial Hydrodynamics --------------------------------------
------------------------------------------------------------------------



From jsw at cdc.noaa.gov  Thu Aug 30 13:08:07 2001
From: jsw at cdc.noaa.gov (Jeff Whitaker)
Date: Thu, 30 Aug 2001 11:08:07 -0600 (MDT)
Subject: [Numpy-discussion] LinearAlgebra patch
In-Reply-To: <3B8E741B.2A1073A6@home.net>
Message-ID: <Pine.OSX.4.33.0108301103000.2852-100000@crdmac10.cdc.noaa.gov>

Chris:  The ATLAS project (http://www.netlib.org/ATLAS) provides a
mechanism for creating optimized BLAS/LAPACK libs on almost any
(unixish) platform.  The tuning process is very complicated.  It would not
be feasible to include this in the Numpy distribution - however the
mechanism does exist to link these libs when compiling numpy.  Perhaps a
more realistic goal would be to modify the setup.py script to
automatically detect ATLAS and use those libs if they are installed.

-Jeff

 On Thu, 30 Aug 2001, Chris Barker wrote:

> Jeff Whitaker wrote:
> > Paul's initial comment
> > was that since it was not self-contained (it required linking external
> > blas and lapack libs) he could not consider putting it in.  I have
> > rectified that now by including new versions of f2c_lite.c, blas_lite.c
> > and d(z)lapack_lite.c that support the newer, faster lapack routines.
>
> I don't know what it would take, but I would LOVE it if NumPy used a
> highly optimised BLAS wherever possible. I have no idea whether there is
> a good BLAS available for every major platform that has a licence that
> allows it to be included with NumPy, but if so, I think it should be
> included. I know this is more effor than just including a generic one,
> but having optimum performace "out of the box" would be a great thing.
>
> just my $.02, which is worth about that, as I am in littel position to
> help...
>
>
> -Chris
>
>
>
>

-- 
Jeffrey S. Whitaker         Phone  : (303)497-6313
Meteorologist               FAX    : (303)497-6449
NOAA/OAR/CDC  R/CDC1        Email  : jsw at cdc.noaa.gov
325 Broadway                Web    : www.cdc.noaa.gov/~jsw
Boulder, CO, USA 80303-3328 Office : Skaggs Research Cntr 1D-124




From europax at home.com  Thu Aug 30 20:23:52 2001
From: europax at home.com (Rob)
Date: Thu, 30 Aug 2001 17:23:52 -0700
Subject: [Numpy-discussion] LinearAlgebra patch/ Athlon optimization
References: <Pine.OSX.4.33.0108300603200.19262-100000@localhost.cdc.noaa.gov> <3B8E741B.2A1073A6@home.net>
Message-ID: <3B8ED918.A7A3DDD6@home.com>

I'd love to have Python and Numpy optimized for the Athlon.  I will have
to wait for gcc3.0 to get absorbed into FreeBSD.  (Mandrake linux has a
beta that uses it)  It turns out that mpg encoding runs at 10x the rate
on my Athlon DDR machine compared to my laptop due to the highly Athlon
optimized "GoGo" encoding program.  Rob.  



Chris Barker wrote:
> 
> Jeff Whitaker wrote:
> > Paul's initial comment
> > was that since it was not self-contained (it required linking external
> > blas and lapack libs) he could not consider putting it in.  I have
> > rectified that now by including new versions of f2c_lite.c, blas_lite.c
> > and d(z)lapack_lite.c that support the newer, faster lapack routines.
> 
> I don't know what it would take, but I would LOVE it if NumPy used a
> highly optimised BLAS wherever possible. I have no idea whether there is
> a good BLAS available for every major platform that has a licence that
> allows it to be included with NumPy, but if so, I think it should be
> included. I know this is more effor than just including a generic one,
> but having optimum performace "out of the box" would be a great thing.
> 
> just my $.02, which is worth about that, as I am in littel position to
> help...
> 
> -Chris
> 
> --
> Christopher Barker,
> Ph.D.
> ChrisHBarker at home.net                 ---           ---           ---
> http://members.home.net/barkerlohmann ---@@       -----@@       -----@@
>                                    ------@@@     ------@@@     ------@@@
> Oil Spill Modeling                ------   @    ------   @   ------   @
> Water Resources Engineering       -------      ---------     --------
> Coastal and Fluvial Hydrodynamics --------------------------------------
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> http://lists.sourceforge.net/lists/listinfo/numpy-discussion

-- 
The Numeric Python EM Project

www.members.home.net/europax



From europax at home.com  Thu Aug 30 20:28:06 2001
From: europax at home.com (Rob)
Date: Thu, 30 Aug 2001 17:28:06 -0700
Subject: [Numpy-discussion] invert() function works just fine
Message-ID: <3B8EDA16.A0F3C134@home.com>

My problems with the invert() function has been resolved.  I found that
somewhere in my program it was trying to invert [0].  So I just added a
conditional statement ahead of it.  Evidently the C program did the same
sort of thing.  So now I've shaved another 20% of execution time off of
my program!

Rob.
-- 
The Numeric Python EM Project

www.members.home.net/europax



From fredsten at kth.se  Fri Aug 31 08:21:43 2001
From: fredsten at kth.se (Fredrik Stenberg)
Date: Fri, 31 Aug 2001 14:21:43 +0200
Subject: [Numpy-discussion] Cholesky factorization
Message-ID: <184672001853112214393@kth.se>

Hi..

Has anyone written a Cholesky factorization for general 
matrices?

Any package out there?

The only one i know of is the python 
wrappers around lapack (pylapack).

I could write it myself, but i prefer to be lazy ;))

/fredriks




From Aureli.Soria_Frisch at ipk.fhg.de  Fri Aug 31 10:54:03 2001
From: Aureli.Soria_Frisch at ipk.fhg.de (Aureli Soria Frisch)
Date: Fri, 31 Aug 2001 16:54:03 +0200
Subject: [Numpy-discussion] Help: Problem embedding Python with Numeric
In-Reply-To: <184672001853112214393@kth.se>
Message-ID: <l03010d01b7b552d9dd80@[153.97.92.109]>

Hello,

I am writing a C program, which uses a class I have written in Python. The
class has a method 'operate_image' that receives as parameters a Numeric
array, an integer and a float.

The class has been repeatedly used with no problems.

When I try to call the method I use:

pres_fi=PyObject_CallMethod(pFI,"operate_image","(Oif)",parray,1,1.0);

But the result (pres-fi) is NULL. I have tested pFI to exist and to have
the mentioned method, with no troubles.

So it seems to me that PyObject_CallMethod is casting down the parray
object, which is a PyArrayObject. That is the reason of malfunctioning.

Questions:
1) Is my interpratation right?
2) Which would be the correct way to do such an operation?

It is the first time I write an Python embedded/extension, so I am quite lost.

Thanks in advance,
Aureli


#################################
Aureli Soria Frisch
Fraunhofer IPK
Dept. Pattern Recognition

post: Pascalstr. 8-9, 10587 Berlin, Germany
e-mail:aureli at ipk.fhg.de
fon: +49 30 39006-150
fax: +49 30 3917517
#################################





From KRodgers at ryanaero.com  Fri Aug 31 12:23:36 2001
From: KRodgers at ryanaero.com (Rodgers, Kevin)
Date: Fri, 31 Aug 2001 09:23:36 -0700
Subject: [Numpy-discussion] one-offset arrays
Message-ID: <0D8C1A50C283D311ABB800508B612E53054F85F2@xcgca065.ryanaero.com>

Indices in Fortran can be defined at the time an array is declared, so
Fortran is not necessarily "one-indexed".

Kevin Rodgers
Northrop Grumman Ryan Aeronautical

-----Original Message-----
From: Eric Nodwell [mailto:nodwell at physics.ubc.ca]
Sent: Tuesday, August 28, 2001 10:01 PM
To: Numpy-discussion at lists.sourceforge.net
Subject: Re: [Numpy-discussion] one-offset arrays



>> Does anyone have a simple method of using one-offset arrays?
>> Specifically, can I define an array "a" so that a[1] refers to the
>> first element?
>>

> Inherit from the UserArray and redefine slicing to your hearts content.

>> Without one-offset indexing, it seems to me that Python is minimally
>> useful for numerical computations. Many, perhaps the majority, of
>> numerical algorithms are one-indexed. Matlab for example is one-based
>> for this reason.  In fact it seems strange to me that a "high-level"
>> language like Python should use zero-offset lists.
>>

> Wow, that is a pretty condescending-sounding statement, though I'm sure
you 
> didn't mean it that way.  Python is indeed being used for quite serious 
> numerical computations.  I have been using Python for quite some time for 
> Numerical work and find it's zero-based indexing combined with the 
> leave-last-one-out slicing notation to be much more convenient.  

Oops, what I really need is a wet-ware (i.e. brain) macro which
enforces the correct order of the pair (think,write,send)! The above
unconsidered comments arose from the sequence
(frustration,write,send,think,regret).  ;)

Obviously there is a lot of numerical work being done with python and
people are very happy with it. But for me I still think it would be
"minimally useful" without 1-indexed arrays. Here's why:

In my view, the most important reason to prefer 1-based indexing
versus 0-based indexing is compatibility.  For numerical work, some of
the languages which I use or have used are Matlab, Mathematica, Maple
and Fortran. These are all 1-indexed. (C is by nature 0-indexed
because it is so close to machine architecture, but with a little bit
of not-entirely-clean pointer manipulation, you can easily make
1-indexed arrays and matrices.) Obviously Python can't be compatible
with these languages in a strict sense, but like most people who do
some programming work, I've built up a library of my own commonly used
routines specific to my work; in general it's a trivial matter to
translate numerical routines from one language to the another if
translation is just a matter of substituting of one set of syntactical
symbols and function names for anther.  However it can be damn tricky
to convert 1-indexed code to 0-indexed code or visa versa without
introducing any errors- believe me!  (Yes it's possible to call nearly
any language from nearly any other language these days so in theory
you don't have to recode, but there are lots of reasons why often
recoding is the preferable route.)

The other reason for choosing 1-based indexing is to keep the code as
near to the standard notation as possible. This is one of the
advantages of using a high level language - it approximates the way
you think about things instead of the way the computer organizes
them. Of course, this can go either way depending on the quantity in
question: as a simple example a spatial vector (x,y,z) is
conventionally labelled 1,2,3 (1-indexed), but a relativistic
four-vector with time included (t,x,y,z) is conventionally labelled
0,1,2,3 (0-indexed). So ideally one would be able to choose the
indexing-type case-by-case. I'm sure that computer programmers will
argue vehemently that code which mixes both 0-indexed and 1-indexed
arrays is hard to understand and maintain, but for experts in a
particular field who are accustomed to certain ingrained notations, it
is the code which breaks the conventional notation which is most
error-prone. In my case, I'm dealing at the moment with crystal
structures with which are associated certain conventional sets of
vectors and tensors - all 1-indexed by convention. I find it a
delicate enough task to always get the correct vector or tensor
without having to remember that d[2] is actually d3. Defining d1,d2,d3
is not convenient because often the index itself needs to be
calculated.

I guess if I understood the reason for 0-indexed lists and tuples in
Python I would be happier. In normal, everyday usage, sets,
collections and lists are 1-indexed (the first item, the second item,
the third item, and so on). Python is otherwise such an elegant and
natural language. Why the ugly exception of making the user conform to
the underlying mechanism of an array being an address plus an offset?

All this is really neither here nor there, since this debate, at least
as far as Python is concerned, was probably settled 10 years ago and
I'm sure nobody wants to hear anything more about it at this point.
As you point out, I can define my own array type with inheritance. I
will also need my own range command and several other functions which
haven't occured to me yet. I was hoping that there would be a standard
module to implement this.


By the way, what is leave-last-one-out slicing? Is it
a[:-1]
or is it
a[0,...]
or is it something else?


Eric

_______________________________________________
Numpy-discussion mailing list
Numpy-discussion at lists.sourceforge.net
http://lists.sourceforge.net/lists/listinfo/numpy-discussion



From huaiyu_zhu at yahoo.com  Fri Aug 31 04:11:56 2001
From: huaiyu_zhu at yahoo.com (Huaiyu Zhu)
Date: Fri, 31 Aug 2001 01:11:56 -0700 (PDT)
Subject: [Numpy-discussion] Subarray with with arbitrary index?
In-Reply-To: <ADEOIFHFONCLEEPKCACCKEBNCMAA.paul@pfdubois.com>
Message-ID: <Pine.LNX.4.10.10108310048010.1394-100000@mars.localdomain>

Following up to an earlier discussion, I've written up some wrapper
functions for subarrays.  The 'takes' function only works for 2d arrays,
but the 'puts' function works for arbitrary array.  Hopefully something
like this can be included in Numeric module.

Huaiyu Zhu


"""
Wrapper functions for dealing with arbitrary subarrays.

Example: addeqs(x, [[0,2], [1,3,4], y), would add (2,3)array y to the
	(2,3)subarray of x comprised of rows 0, 2 and columns 1, 3, 4.

A more natural notation of addeqs(x, ij, y) would be x[ij] += y, but seems
difficult to do with current numpy. (How to let slice handle lists?)

"""

from Numeric import *

def puts(x, irows, icols, v):
	"""
	puts(x, i, j, v): Put v in subgrid of 2d array x given by i, j.
	"""
	nrow, ncol = x.shape
	if irows is None:		irows = arange(nrow)
	if icols is None:		icols = arange(ncol)
	if len(shape(icols)) == 1:
		icols = icols[:ncol]
	if len(shape(irows)) == 0 or len(shape(icols)) == 0:
		ii = irows*ncol + icols
		v1 = v
	else:
		ii = (irows*ncol)[:, NewAxis] + icols[NewAxis, :]
		v1 = reshape(v, shape(ii))
	put(x, ii, v1)

def takes(x, I):
	"""
	takes(x, I): Takes a subgrid from array x.
	I is a list of list of subindices.
	"""
	for i in xrange(len(I)):
		ii = I[i]
		if ii is not None:
			x = take(x, ii, i)
	return x

def addeqs(x, ij, y):
	"""
	Simulates x[ij] += y, where ij can be arbitray subarray.
	"""
	i, j = ij
	puts(x, i, j, takes(x, ij) + y)

if __name__ == "__main__":
	a5 = arange(5)
	a2 = arange(2)
	a3 = arange(3)
	d = array([a3, a3+3])
	print d; print

	b = array([a5, a5+5, a5+10, a5+15]);	print b; print
	c = b.copy();	puts(c, None, 3, a5+1000);	print c; print
	c = b.copy();	puts(c, a2*2, 3, a5+1000);	print c; print
	c = b.copy();	puts(c, 2, a2*2, a5+1000);	print c; print
	c = b.copy();	puts(c, a2*2+1, a3*2, d+1000);	print c; print
	
	c = b.copy();	d1 = takes(c, (a2*2+1, a3*2))
	c1 = c
	print d1; print
	puts(c, a2*2+1, a3*2, d+1000);	print c; print
	puts(c, a2*2+1, a3*2, d1);	print c; print

	addeqs(c, (a2*2+1, a3*2), d1*0+100);	print c; print
	print c1; print
	d1 += 20; print c; print  # Alas, this does not change c

	d2 = takes(c, (a2*2+1, None))
	print d2; print
	print shape(c), shape(a2), shape(d2)
	addeqs(c, (a2*2+1, None), -d2)
	print c; print

"""
The expected results are
[[0 1 2]
 [3 4 5]]

[[ 0  1  2  3  4]
 [ 5  6  7  8  9]
 [10 11 12 13 14]
 [15 16 17 18 19]]

[[   0    1    2 1000    4]
 [   5    6    7 1001    9]
 [  10   11   12 1002   14]
 [  15   16   17 1003   19]]

[[   0    1    2 1000    4]
 [   5    6    7    8    9]
 [  10   11   12 1001   14]
 [  15   16   17   18   19]]

[[   0    1    2    3    4]
 [   5    6    7    8    9]
 [1000   11 1001   13   14]
 [  15   16   17   18   19]]

[[   0    1    2    3    4]
 [1000    6 1001    8 1002]
 [  10   11   12   13   14]
 [1003   16 1004   18 1005]]

[[ 5  7  9]
 [15 17 19]]

[[   0    1    2    3    4]
 [1000    6 1001    8 1002]
 [  10   11   12   13   14]
 [1003   16 1004   18 1005]]

[[ 0  1  2  3  4]
 [ 5  6  7  8  9]
 [10 11 12 13 14]
 [15 16 17 18 19]]

[[  0   1   2   3   4]
 [105   6 107   8 109]
 [ 10  11  12  13  14]
 [115  16 117  18 119]]

[[  0   1   2   3   4]
 [105   6 107   8 109]
 [ 10  11  12  13  14]
 [115  16 117  18 119]]

[[  0   1   2   3   4]
 [105   6 107   8 109]
 [ 10  11  12  13  14]
 [115  16 117  18 119]]

[[105   6 107   8 109]
 [115  16 117  18 119]]

(4, 5) (2,) (2, 5)
[[ 0  1  2  3  4]
 [ 0  0  0  0  0]
 [10 11 12 13 14]
 [ 0  0  0  0  0]]
"""




From jsw at cdc.noaa.gov  Fri Aug 31 16:51:13 2001
From: jsw at cdc.noaa.gov (Jeff Whitaker)
Date: Fri, 31 Aug 2001 14:51:13 -0600 (MDT)
Subject: [Numpy-discussion] Cholesky factorization
In-Reply-To: <633420018531173032787@kth.se>
Message-ID: <Pine.OSX.4.33.0108311448560.7556-100000@crdmac10.cdc.noaa.gov>

I've uploaded a new version of my LinearAlgebra patch that adds a Cholesky
decomposition routine (cholesky_decomposition), as per Frederik's request.

-Jeff

On Fri, 31 Aug 2001, Fredrik Stenberg wrote:

> You read my mind......
>
> ;))
>
> /fredriks
>
>
> ------- Ursprungligt meddelande -------
> Fr?n: jsw at cdc.noaa.gov
> Datum: Fri, 31 Aug 2001 08:32:01 -0700
>
> This message was sent from Geocrawler.com by "jeff whitaker"
> <jsw at cdc.noaa.gov>
>
>
> Fredrik:  I could easily add this to the
> LinearAlgebra patches I've just submitted.  Are
> you thinking of an interface to the lapack routine
> dpotrf (cholesky factorization of a real symmteric
> pos definite matrix), or something else?
>
> -Jeff
>
> ---------------------------------------
> Hi..
>
> Has anyone written a Cholesky factorization for
> general
> matrices?
>
> Any package out there?
>
> The only one i know of is the python
> wrappers around lapack (pylapack).
>
> I could write it myself, but i prefer to be lazy ;))
>
> /fredriks
>
>
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> http://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
>
>
> Geocrawler.com - The Knowledge Archive
>
>
>
>

-- 
Jeffrey S. Whitaker         Phone  : (303)497-6313
Meteorologist               FAX    : (303)497-6449
NOAA/OAR/CDC  R/CDC1        Email  : jsw at cdc.noaa.gov
325 Broadway                Web    : www.cdc.noaa.gov/~jsw
Boulder, CO, USA 80303-3328 Office : Skaggs Research Cntr 1D-124




From milgram at cgpp.com  Fri Aug 31 08:32:11 2001
From: milgram at cgpp.com (J. Milgram)
Date: Fri, 31 Aug 2001 08:32:11 -0400
Subject: [Numpy-discussion] cephes-1.3 install problem 
In-Reply-To: Your message of "Wed, 29 Aug 2001 23:43:27 EDT."
             <3B8DB65F.3010803@erols.com> 
Message-ID: <200108311233.f7VCWCm16235@cgpp.com>


> I came across this problem a few months ago, I hacked the indicated
> lines in what seemed, at the time, to be the obvious fashion and the
> thing worked fine.  I'm sorry that I can't be more helpful, but I
> haven't kept the file. The hack might just have been to comment out
> the offending statements.

Yes, I tried just that (also suggested by Travis Oliphant), and it
compiled just fine.

Thanks

Judah


Judah Milgram                             milgram at cgpp.com
College Park Press                        http://www.cgpp.com
P.O. Box 143, College Park MD, USA 20741  +001 (301) 422-4626 (422-3047 fax)



From europax at home.com  Fri Aug 31 19:43:03 2001
From: europax at home.com (Rob)
Date: Fri Aug 31 19:43:03 2001
Subject: [Numpy-discussion] matrix invert() problems
Message-ID: <3B904AF6.D0857135@home.com>

I found a slight discrepancy in my Python EM code output when I use the
invert() function.  It may be that the C routine that I am comparing it
to actually has the bug.  For now I don't know.  It would be cool to
have a large two dimensional array that has a known inverse.  I think
NIST keeps an array collection, but I don't know if they have the
inverses.  Rob.


-- 
The Numeric Python EM Project

www.members.home.net/europax



From europax at home.com  Fri Aug 31 22:48:02 2001
From: europax at home.com (Rob)
Date: Fri Aug 31 22:48:02 2001
Subject: [Numpy-discussion] matrix invert() problems
References: <3B904AF6.D0857135@home.com>
Message-ID: <3B907642.A0A9D3EE@home.com>

I'm pretty certain now that this is a numeric precision problem.  The
discrepancy only shows up for very small numbers.  For now I'll ignore
it.  Rob.

Rob wrote:
> 
> I found a slight discrepancy in my Python EM code output when I use the
> invert() function.  It may be that the C routine that I am comparing it
> to actually has the bug.  For now I don't know.  It would be cool to
> have a large two dimensional array that has a known inverse.  I think
> NIST keeps an array collection, but I don't know if they have the
> inverses.  Rob.
> 
> --
> The Numeric Python EM Project
> 
> www.members.home.net/europax
> 
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion

-- 
The Numeric Python EM Project

www.members.home.net/europax