From hinsen@cnrs-orleans.fr Fri Sep 4 13:10:56 1998
From: hinsen@cnrs-orleans.fr (Konrad Hinsen)
Date: Fri, 4 Sep 1998 14:10:56 +0200
Subject: [Matrix-SIG] Compiling for Windows
Message-ID: <199809041210.OAA18842@dirac.cnrs-orleans.fr>
This is not exactly about NumPy, but I think that I reach the right
people here, so...
I just finished my first scientific Python program meant for end
users, and I would like to make it available for the Windows world as
well. It uses the Python standard library, NumPy, and of course its
own modules, including a few C extension modules, which call the NumPy
C modules. I suppose a Windows version should build on the existing
Python and NumPy binary distributions, but that's about all I know :-(
I don't care about fancy installation, but it should not require a C
compiler and it should result in something that can be run by clicking
on an icon. Is that doable? (I *really* don't know anything about
Python under Windows!) I suspect the biggest problem is getting the
C modules compiled, could anyone help with that?
For those interested in the program, there's a full description at
http://dirac.cnrs-orleans.fr/DomainFinder/
But unless you are interested in protein dynamics, you probably
can't do much with it!
Konrad.
--
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsen@cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.55.69
Rue Charles Sadron | Fax: +33-2.38.63.15.17
45071 Orleans Cedex 2 | Deutsch/Esperanto/English/
France | Nederlands/Francais
-------------------------------------------------------------------------------
From krodgers@tdyryan.com Fri Sep 4 18:52:25 1998
From: krodgers@tdyryan.com (Kevin Rodgers)
Date: Fri, 4 Sep 1998 10:52:25 -0700
Subject: [Matrix-SIG] Windows NumPy installer problem
Message-ID: <000101bdd82c$c74ffa70$3228bbcd@krodgers.gate.tdyryan.com>
With the current (and previous) version of the NumPy installer, on all of
the NT machines (NT 4 SP3) on which I have installed NumPy (6 so far), the
registry key that gets set to specify the PythonPath always ends up with the
"Program Files" directory munged to the DOS name (PROGRA~1). Would somebody
please fix this? I don't know if it is a "real" problems or not, but it's
ugly as hell and bugs more naīve users (well, it bugs me too!). Thanks in
advance . . .
--------------------------------
Kevin Rodgers Teledyne Ryan Aeronautical krodgers@tdyryan.com
"This one goes up to eleven." -- Nigel Tufnel
-------------------------------
From ransom@cfa.harvard.edu Sat Sep 5 17:22:53 1998
From: ransom@cfa.harvard.edu (Scott M. Ransom)
Date: Sat, 05 Sep 1998 16:22:53 +0000
Subject: [Matrix-SIG] Freeing malloc'd array in Python
Message-ID: <35F1655D.1BCA3B61@cfa.harvard.edu>
Hello,
I am trying to wrap a large pulsar data analysis library written in C so
that I can use it in Python. I am using the Numeric extension to
represent many vectors and matrices that I use.
I am new to Python and do not (yet!) have a good grasp of Python's
reference counting method of garbage collection -- as you will probably
see from my code snippet below... ;)
The problem that I am running into is that many of my routines allocate
arrays using malloc() from 'C' and then return these newly created
arrays to the user. In my 'C' routines I simple free() the array in the
main program when I don't need it anymore.
When I wrap these functions onto Numeric arrays, though, Python does not
seem to want to free the data that I malloc'd. For example, using the
code attached at the bottom of this note and compiled (on Linux) using:
gcc -shared -I/usr/local/include/python1.5 -Wall -fpic -g -O my_range.c
-o my_range.so
and then imported into Python, I get the following:
Python 1.5.1 (#8, Sep 3 1998, 19:02:41) [GCC egcs-2.90.29 980515 (egc
on linux2
Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
>>> from Numeric import *
>>> from my_range import *
>>> from sys import *
>>> a = my_range(10)
>>> getrefcount(a)
2
>>> a
array([ 0., 1., 2., 3., 4., 5., 6., 7., 8., 9.])
>>> del(a)
>>> getrefcount(a)
Traceback (innermost last):
File "<stdin>", line 1, in ?
NameError: a
>>> a = my_range(100000)
>>> del(a)
>>>
del(a) obviously deletes the PyArrayObject, but examining the memory
usage before and after the a = my_range(100000) and the corresponding
del(a), it is obvious that Python is not touching the actual data in the
PyArrayObject -- a bigtime memory leak.
I examined the code in arrayobject.c and arrayobject.h and my initial
impressions are that I need to somehow manually free the data portion of
the PyArrayObject when I am finished with it in Python -- before I del()
the PyArrayObject. I unfortunately have no idea how to do this. Do I
need another helper function in my wrapper code to free() the data in
the PyArrayObject()? Or am I missing something even easier?
It seems that I have also been warned (from arrayobject.h) not to use
PyArray_FromDimsAndData() to create my PyArrayObject (any ideas on how
to use something else?)
Thanks in advance for your help,
Scott Ransom
---------my_range.c------------
#include <stdlib.h>
#include <stdio.h>
#include "Python.h"
#include "arrayobject.h"
/* Here is an example library function that returns an array (a 1D
vector). This is just representative -- the actual functions in my
library are much more complicated. They are all malloc'd,
though. */
double * my_range(long n)
{
double *v;
long i;
v = (double *) malloc((size_t) (sizeof(double) * n));
if (!v) {
printf("\nAllocation error in my_range()\n");
exit(1);
}
for (i=0; i<n; i++) v[i] = (double) i;
return v;
}
/* Here is a representative wrapper function that I am using to
interface the above routine with Python (using Numeric arrays). */
static PyObject *wrap_my_range(PyObject *self, PyObject *args)
{
PyObject *obj;
PyArrayObject *arr;
double *result;
long n;
self = self;
if(!PyArg_ParseTuple(args,"O:wrap_my_range",&obj))
return NULL;
n = PyInt_AsLong((PyObject *)obj);
result = my_range(n);
arr = (PyArrayObject *)PyArray_FromDimsAndData(1, (int *)&n,
PyArray_DOUBLE, \
(char *) result);
if (arr == NULL) return NULL;
return (PyObject *)arr;
}
/* Here are the module initialization functions */
static PyObject *ErrorObject;
static PyMethodDef my_range_method[] = {
{ "my_range", wrap_my_range, 1 },
{ NULL, NULL }
};
void initmy_range()
{
PyObject *m, *d;
m = Py_InitModule("my_range", my_range_method);
d = PyModule_GetDict(m);
import_array();
ErrorObject = PyString_FromString("my_range.error");
PyDict_SetItemString(d, "error", ErrorObject);
if (PyErr_Occurred())
Py_FatalError("can't initialize module my_range");
}
--
Scott M. Ransom
Phone: (580) 536-7215 Address: 703 SW Chaucer Cir.
email: ransom@cfa.harvard.edu Lawton, OK 73505
PGP Fingerprint: D2 0E D0 10 CD 95 06 DA EF 78 FE 2B CB 3A D3 53
From aaron@cs.rutgers.edu Sat Sep 5 19:27:13 1998
From: aaron@cs.rutgers.edu (Aaron Watters)
Date: Sat, 05 Sep 1998 14:27:13 -0400
Subject: [Matrix-SIG] Freeing malloc'd array in Python
References: <35F1655D.1BCA3B61@cfa.harvard.edu>
Message-ID: <35F18281.70AC63EA@cs.rutgers.edu>
I might be wrong, but i think the problem is that
Python keeps a last value reference named _ (underscore);
the del a deletes the *name* a but the object formerly
named a still has another name _, maybe try this instead
a = my_range(10000)
a = None
1
Now the object should be gone. Another thing that can
happen is the object can get stuck on a traceback object
import sys
sys.last_traceback = None
If you really have a memory leak this should cause problems
from time import sleep
for i in range(100):
x = my_range(10000)
print "sleeping"; sleep(2)
...watch the process size from another window. If it really
is leaking, maybe recomplain. -- Aaron Watters
Scott M. Ransom wrote:
> Hello,
>
> I am trying to wrap a large pulsar data analysis library written in C so
> that I can use it in Python. I am using the Numeric extension to
> represent many vectors and matrices that I use.
>
> I am new to Python and do not (yet!) have a good grasp of Python's
> reference counting method of garbage collection -- as you will probably
> see from my code snippet below... ;)
>
> The problem that I am running into is that many of my routines allocate
> arrays using malloc() from 'C' and then return these newly created
> arrays to the user. In my 'C' routines I simple free() the array in the
> main program when I don't need it anymore.
>
> When I wrap these functions onto Numeric arrays, though, Python does not
> seem to want to free the data that I malloc'd. For example, using the
> code attached at the bottom of this note and compiled (on Linux) using:
>
> gcc -shared -I/usr/local/include/python1.5 -Wall -fpic -g -O my_range.c
> -o my_range.so
>
> and then imported into Python, I get the following:
>
> Python 1.5.1 (#8, Sep 3 1998, 19:02:41) [GCC egcs-2.90.29 980515 (egc
> on linux2
> Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
> >>> from Numeric import *
> >>> from my_range import *
> >>> from sys import *
> >>> a = my_range(10)
> >>> getrefcount(a)
> 2
> >>> a
> array([ 0., 1., 2., 3., 4., 5., 6., 7., 8., 9.])
> >>> del(a)
> >>> getrefcount(a)
> Traceback (innermost last):
> File "<stdin>", line 1, in ?
> NameError: a
> >>> a = my_range(100000)
> >>> del(a)
> >>>
>
> del(a) obviously deletes the PyArrayObject, but examining the memory
> usage before and after the a = my_range(100000) and the corresponding
> del(a), it is obvious that Python is not touching the actual data in the
> PyArrayObject -- a bigtime memory leak.
>
> I examined the code in arrayobject.c and arrayobject.h and my initial
> impressions are that I need to somehow manually free the data portion of
> the PyArrayObject when I am finished with it in Python -- before I del()
> the PyArrayObject. I unfortunately have no idea how to do this. Do I
> need another helper function in my wrapper code to free() the data in
> the PyArrayObject()? Or am I missing something even easier?
>
> It seems that I have also been warned (from arrayobject.h) not to use
> PyArray_FromDimsAndData() to create my PyArrayObject (any ideas on how
> to use something else?)
>
> Thanks in advance for your help,
>
> Scott Ransom
>
> ---------my_range.c------------
> #include <stdlib.h>
> #include <stdio.h>
> #include "Python.h"
> #include "arrayobject.h"
>
> /* Here is an example library function that returns an array (a 1D
> vector). This is just representative -- the actual functions in my
> library are much more complicated. They are all malloc'd,
> though. */
>
> double * my_range(long n)
> {
> double *v;
> long i;
>
> v = (double *) malloc((size_t) (sizeof(double) * n));
> if (!v) {
> printf("\nAllocation error in my_range()\n");
> exit(1);
> }
> for (i=0; i<n; i++) v[i] = (double) i;
> return v;
> }
>
> /* Here is a representative wrapper function that I am using to
> interface the above routine with Python (using Numeric arrays). */
>
> static PyObject *wrap_my_range(PyObject *self, PyObject *args)
> {
> PyObject *obj;
> PyArrayObject *arr;
> double *result;
> long n;
>
> self = self;
> if(!PyArg_ParseTuple(args,"O:wrap_my_range",&obj))
> return NULL;
> n = PyInt_AsLong((PyObject *)obj);
> result = my_range(n);
> arr = (PyArrayObject *)PyArray_FromDimsAndData(1, (int *)&n,
> PyArray_DOUBLE, \
> (char *) result);
> if (arr == NULL) return NULL;
> return (PyObject *)arr;
> }
>
> /* Here are the module initialization functions */
>
> static PyObject *ErrorObject;
> static PyMethodDef my_range_method[] = {
> { "my_range", wrap_my_range, 1 },
> { NULL, NULL }
> };
>
> void initmy_range()
> {
> PyObject *m, *d;
>
> m = Py_InitModule("my_range", my_range_method);
> d = PyModule_GetDict(m);
> import_array();
> ErrorObject = PyString_FromString("my_range.error");
> PyDict_SetItemString(d, "error", ErrorObject);
> if (PyErr_Occurred())
> Py_FatalError("can't initialize module my_range");
> }
>
> --
> Scott M. Ransom
> Phone: (580) 536-7215 Address: 703 SW Chaucer Cir.
> email: ransom@cfa.harvard.edu Lawton, OK 73505
> PGP Fingerprint: D2 0E D0 10 CD 95 06 DA EF 78 FE 2B CB 3A D3 53
>
> _______________________________________________
> Matrix-SIG maillist - Matrix-SIG@python.org
> http://www.python.org/mailman/listinfo/matrix-sig
From ransom@cfa.harvard.edu Sat Sep 5 22:08:21 1998
From: ransom@cfa.harvard.edu (Scott M. Ransom)
Date: Sat, 05 Sep 1998 21:08:21 +0000
Subject: [Matrix-SIG] Freeing malloc'd array in Python
References: <35F1655D.1BCA3B61@cfa.harvard.edu> <35F18281.70AC63EA@cs.rutgers.edu>
Message-ID: <35F1A844.66B1A989@cfa.harvard.edu>
Aaron Watters wrote:
> ...maybe try this instead
>
> a = my_range(10000)
> a = None
> 1
>
> Now the object should be gone.
The object is gone, but the data memory is still allocated.
> Another thing that can
> happen is the object can get stuck on a traceback object
>
> import sys
> sys.last_traceback = None
This didn't help either.
> If you really have a memory leak this should cause problems
>
> from time import sleep
> for i in range(100):
> x = my_range(10000)
> print "sleeping"; sleep(2)
>
> ...watch the process size from another window. If it really
> is leaking, maybe recomplain. -- Aaron Watters
OK. I'm recomplaining. It is leaking like a seive....
Any other ideas?
Is there an easy way to free a malloc'd array from Python?
--
Scott M. Ransom
Phone: (580) 536-7215 Address: 703 SW Chaucer Cir.
email: ransom@cfa.harvard.edu Lawton, OK 73505
PGP Fingerprint: D2 0E D0 10 CD 95 06 DA EF 78 FE 2B CB 3A D3 53
From aaron@cs.rutgers.edu Sat Sep 5 22:52:51 1998
From: aaron@cs.rutgers.edu (Aaron Watters)
Date: Sat, 05 Sep 1998 17:52:51 -0400
Subject: [Matrix-SIG] Freeing malloc'd array in Python
References: <35F1655D.1BCA3B61@cfa.harvard.edu> <35F18281.70AC63EA@cs.rutgers.edu> <35F1A844.66B1A989@cfa.harvard.edu>
Message-ID: <35F1B2B3.ECA546CA@cs.rutgers.edu>
> Is there an easy way to free a malloc'd array from Python?
Sorry for the last message.
In general if you allocate memory and hand it to Python you
need to "wrap" the allocated structure in a python object with
its own "delete" function which does any finalization required
including deallocating the wrapped structure. Please see either
of the current Python books for discussion.
I'm not acquainted with the C level of numpy but it might be that
what is happening in your case is that you allocate the array
and then *copy* it into a python array and then never deallocate it.
But I haven't looked into it. If you just need fresh memory maybe
you could allocate a "blob" as a numpy array and then use the
internal buffer as storage, perhaps obviating the need to do a full
pyobject implementation... If you need to wrap preallocated
structures, then you gotta do the pyobject thing -- but be careful
if the structure is referenced outside of the "python domain".
Also maybe look into SWIG.
sorry again.
-- Aaron Watters
From ransom@cfa.harvard.edu Sun Sep 6 00:32:42 1998
From: ransom@cfa.harvard.edu (Scott M. Ransom)
Date: Sat, 05 Sep 1998 23:32:42 +0000
Subject: [Matrix-SIG] Freeing malloc'd array in Python
References: <35F1655D.1BCA3B61@cfa.harvard.edu> <35F18281.70AC63EA@cs.rutgers.edu> <35F1A844.66B1A989@cfa.harvard.edu> <35F1B2B3.ECA546CA@cs.rutgers.edu>
Message-ID: <35F1CA1A.8381049D@cfa.harvard.edu>
Aaron Watters wrote:
> ...If you just need fresh memory maybe
> you could allocate a "blob" as a numpy array and then use the
> internal buffer as storage, perhaps obviating the need to do a full
> pyobject implementation... If you need to wrap preallocated
> structures, then you gotta do the pyobject thing -- but be careful
> if the structure is referenced outside of the "python domain".
Well I got rid of the memory leak by using a different routine to
allocate the PyArrayObject. Instead of doing the following in the
wrapper function:
arr = (PyArrayObject *)PyArray_FromDimsAndData(1, (int *)&n,
PyArray_DOUBLE, \
(char *) result);
I used the following to allocate a PyArrayObect of known size and then
copied the data into the data portion of the PyArrayObject. Finally, I
lose the memory leak by freeing the now useless array that came from my
'C' function. This works (including proper memory management of the
array in Python), but seems kind of inelegant since I have to copy a
perfectly good block of memory into a different location.
Note: 'result' is a double precision vector that was allocated in a 'C'
function
arr = (PyArrayObject *)PyArray_FromDims(1, (int *)&n, PyArray_DOUBLE);
if (arr == NULL) return NULL;
/* Have to copy the data, until I figure out a better way... */
memcpy(arr->data, (char *)result, n*arr->descr->elsize);
/* Free the malloced array from the 'C' routine... */
free(result);
PyArray_INCREF(arr);
return (PyObject *)arr;
Does anyone know if I can perform a free(arr->data) and then a arr->data
= result instead? That would get rid of the memcpy() and keep the
original result array.
> Also maybe look into SWIG.
I already am. The reason I am doing this whole thing is to generate
SWIG templates for Numeric Python for my code...
Thanks for the help.
Scott
--
Scott M. Ransom
Phone: (580) 536-7215 Address: 703 SW Chaucer Cir.
email: ransom@cfa.harvard.edu Lawton, OK 73505
PGP Fingerprint: D2 0E D0 10 CD 95 06 DA EF 78 FE 2B CB 3A D3 53
From Paul F. Dubois" <dubois1@llnl.gov Sun Sep 6 04:45:04 1998
From: Paul F. Dubois" <dubois1@llnl.gov (Paul F. Dubois)
Date: Sat, 5 Sep 1998 20:45:04 -0700
Subject: [Matrix-SIG] Freeing malloc'd array in Python
Message-ID: <001901bdd948$bcb393c0$52120f80@dubois1-1.llnl.gov>
The intent of PyArray_FromDimsAndData is to make a Python object that uses
space you own. Python is NOT supposed to ever release it. Normally one would
create the NumPy array and then fill it up with your calculation.
-----Original Message-----
From: Scott M. Ransom <ransom@cfa.harvard.edu>
To: Aaron Watters <aaron@cs.rutgers.edu>; matrix-sig <matrix-sig@python.org>
Date: Saturday, September 05, 1998 4:34 PM
Subject: Re: [Matrix-SIG] Freeing malloc'd array in Python
>Aaron Watters wrote:
>
>> ...If you just need fresh memory maybe
>> you could allocate a "blob" as a numpy array and then use the
>> internal buffer as storage, perhaps obviating the need to do a full
>> pyobject implementation... If you need to wrap preallocated
>> structures, then you gotta do the pyobject thing -- but be careful
>> if the structure is referenced outside of the "python domain".
>
>Well I got rid of the memory leak by using a different routine to
>allocate the PyArrayObject. Instead of doing the following in the
>wrapper function:
>
> arr = (PyArrayObject *)PyArray_FromDimsAndData(1, (int *)&n,
>PyArray_DOUBLE, \
> (char *) result);
>
>I used the following to allocate a PyArrayObect of known size and then
>copied the data into the data portion of the PyArrayObject. Finally, I
>lose the memory leak by freeing the now useless array that came from my
>'C' function. This works (including proper memory management of the
>array in Python), but seems kind of inelegant since I have to copy a
>perfectly good block of memory into a different location.
>
>Note: 'result' is a double precision vector that was allocated in a 'C'
>function
>
> arr = (PyArrayObject *)PyArray_FromDims(1, (int *)&n, PyArray_DOUBLE);
>
> if (arr == NULL) return NULL;
>
> /* Have to copy the data, until I figure out a better way... */
>
> memcpy(arr->data, (char *)result, n*arr->descr->elsize);
>
> /* Free the malloced array from the 'C' routine... */
>
> free(result);
>
> PyArray_INCREF(arr);
> return (PyObject *)arr;
>
>Does anyone know if I can perform a free(arr->data) and then a arr->data
>= result instead? That would get rid of the memcpy() and keep the
>original result array.
>
>> Also maybe look into SWIG.
>
>I already am. The reason I am doing this whole thing is to generate
>SWIG templates for Numeric Python for my code...
>
>Thanks for the help.
>
>Scott
>
>--
>Scott M. Ransom
>Phone: (580) 536-7215 Address: 703 SW Chaucer Cir.
>email: ransom@cfa.harvard.edu Lawton, OK 73505
>PGP Fingerprint: D2 0E D0 10 CD 95 06 DA EF 78 FE 2B CB 3A D3 53
>
>
>
>
>_______________________________________________
>Matrix-SIG maillist - Matrix-SIG@python.org
>http://www.python.org/mailman/listinfo/matrix-sig
>
>
From hinsen@cnrs-orleans.fr Mon Sep 7 10:04:07 1998
From: hinsen@cnrs-orleans.fr (Konrad Hinsen)
Date: Mon, 7 Sep 1998 11:04:07 +0200
Subject: [Matrix-SIG] Freeing malloc'd array in Python
In-Reply-To: <35F1CA1A.8381049D@cfa.harvard.edu> (ransom@cfa.harvard.edu)
References: <35F1655D.1BCA3B61@cfa.harvard.edu> <35F18281.70AC63EA@cs.rutgers.edu> <35F1A844.66B1A989@cfa.harvard.edu> <35F1B2B3.ECA546CA@cs.rutgers.edu> <35F1CA1A.8381049D@cfa.harvard.edu>
Message-ID: <199809070904.LAA16258@dirac.cnrs-orleans.fr>
> I used the following to allocate a PyArrayObect of known size and then
> copied the data into the data portion of the PyArrayObject. Finally, I
> lose the memory leak by freeing the now useless array that came from my
> 'C' function. This works (including proper memory management of the
> array in Python), but seems kind of inelegant since I have to copy a
> perfectly good block of memory into a different location.
There is no other solution, as far as I know, at least no easy one.
PyArray_FromDimsAndData() is meant for wrapping static arrays,
i.e. arrays that cannot be freed.
If you want to avoid copying, and don't mind some added complexity,
you could try the following: in addition to the array object (created
with PyArray_FromDimsAndData()), return a Python CObject from your C
module which wraps the pointer to the C array and a pointer to a
function called upon deletion which frees the array. Then make sure
that both Python objects are deleted at the same time.
Yet another way, if you don't care messing around with undocumented
data structures, is creating a small Python array with its
data space, freeing the data space, substituting a pointer to the
data space you have, and patch the other entries in the array data
structure to reflect the correct size.
But unless you are working with really large arrays, just stick
to the solution you have!
I don't know if there are many C libraries out there that return
allocated memory blocks to the caller (which I think is not a good
approach for a variety of reasons). If there are, then NumPy should
offer some way to handle freeing.
--
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsen@cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.55.69
Rue Charles Sadron | Fax: +33-2.38.63.15.17
45071 Orleans Cedex 2 | Deutsch/Esperanto/English/
France | Nederlands/Francais
-------------------------------------------------------------------------------
From ransom@cfa.harvard.edu Mon Sep 7 15:28:21 1998
From: ransom@cfa.harvard.edu (Scott M. Ransom)
Date: Mon, 07 Sep 1998 14:28:21 +0000
Subject: [Matrix-SIG] Freeing malloc'd array in Python
References: <35F1655D.1BCA3B61@cfa.harvard.edu> <35F18281.70AC63EA@cs.rutgers.edu> <35F1A844.66B1A989@cfa.harvard.edu> <35F1B2B3.ECA546CA@cs.rutgers.edu> <35F1CA1A.8381049D@cfa.harvard.edu> <199809070904.LAA16258@dirac.cnrs-orleans.fr>
Message-ID: <35F3ED85.8DFF37CA@cfa.harvard.edu>
Konrad Hinsen wrote:
> Yet another way, if you don't care messing around with undocumented
> data structures, is creating a small Python array with its
> data space, freeing the data space, substituting a pointer to the
> data space you have, and patch the other entries in the array data
> structure to reflect the correct size.
I think I will play around with this.
> But unless you are working with really large arrays, just stick
> to the solution you have!
This is the problem. I am often working with arrays that are approx. equal to
about half of the real memory size of my machine. (Pulsar observations generate
_a lot_ of data!) So any method that has me copying memory I would like to try
and avoid.
> I don't know if there are many C libraries out there that return
> allocated memory blocks to the caller (which I think is not a good
> approach for a variety of reasons). If there are, then NumPy should
> offer some way to handle freeing.
I agree. I wrote the library myself and the reason why I have functions returning
memory blocks is that for many of the calls, the size of the block is unknown when
the block is requested. So the routine calculates the size of the memory needed,
allocates it, works with it, and returns the memory block along with its size.
All I have to do is free it.
I could avoid this by breaking the code up into individual pieces that do each of
the above, but that makes the code much less readable, IMHO.
I don't know. Now that I have "found" Python and NumPy (about a week ago -- and I
must say that I am quite amazed by it....) maybe I'll have to do some library
re-writing to make it a little more Python/NumPy/SWIG friendly.
Thanks for the help,
Scott
--
Scott M. Ransom
Phone: (580) 536-7215 Address: 703 SW Chaucer Cir.
email: ransom@cfa.harvard.edu Lawton, OK 73505
PGP Fingerprint: D2 0E D0 10 CD 95 06 DA EF 78 FE 2B CB 3A D3 53
From zcm@llnl.gov Tue Sep 8 16:55:41 1998
From: zcm@llnl.gov (Zane Motteler)
Date: Tue, 8 Sep 1998 07:55:41 -0800
Subject: [Matrix-SIG] Freeing malloc'd array in Python
In-Reply-To: <35F3ED85.8DFF37CA@cfa.harvard.edu>
References: <35F1655D.1BCA3B61@cfa.harvard.edu>
<35F18281.70AC63EA@cs.rutgers.edu> <35F1A844.66B1A989@cfa.harvard.edu>
<35F1B2B3.ECA546CA@cs.rutgers.edu> <35F1CA1A.8381049D@cfa.harvard.edu>
<199809070904.LAA16258@dirac.cnrs-orleans.fr>
Message-ID: <v03007801b21b0142723a@[128.115.36.161]>
Hi Scott,
Unlike a lot of workaholics, I took Labor Day off. ;-} So I've
read the correspondence about freeing a malloc'ed array in
Python, by which I understand you to mean that you want Python
to do the freeing when the array's reference count reaches zero.
I hate to expose myself as an unregenerate hacker, but there
is a way to do it (and you'll see it done shamelessly in
gistCmodule.c). The thing is, when you create an array with
PyArray_FromDimsAndData, there is a flag in the struct which
says, in effect, I don't own this data, so don't free it when
my reference count reaches zero. You can fool mother nature
by setting this flag to the opposite setting. Suppose op
is the name of the array you created. Then simply do
op->flags |= OWN_DATA
That makes Python think that *it* owns the data, so when
the reference count reaches zero, it will be freed.
Paul and Konrad are absolutely correct when they say that
PyArray_FromDimsAndData was intended to create either an
array to hold static data, or else one to hold data which you
are responsible for freeing. However, in gistCmodule we
found the above trick indispensable. I agree with you that
it seems wasteful to copy the data if you really don't
have to.
Hope this helps.
Zane
-----------------------------------------------------------------
Zane C. Motteler, Ph. D.
Computer Scientist | Professor Emeritus of
Lawrence Livermore National | Computer Science and Engineering
Laboratory | California Polytechnic
P. O. Box 808, L-038 | State University
Livermore, CA 94551-0808 | San Luis Obispo, CA 93407
925/423-2143, FAX 925/423-9969 | zmottel@hornet.csc.calpoly.edu
zcm@llnl.gov, motteler@icf.llnl.gov
(For FedEx, UPS, etc. use 7000 East Avenue, zip 94550-9234)
From cgw@pgt.com Thu Sep 10 18:59:26 1998
From: cgw@pgt.com (Charles G Waldman)
Date: Thu, 10 Sep 1998 13:59:26 -0400 (EDT)
Subject: [Matrix-SIG] [PATCH] fix typo in error message
Message-ID: <13816.4990.485785.926485@janus.pgt.com>
I was taken aback to see a misspelled error message come out of NumPy.
It looks kind of unprofessional... unless "deap" is a technical term
I've never heard of...
===================================================================
RCS file: arrayobject.c,v
retrieving revision 1.1
diff -u -r1.1 arrayobject.c
--- arrayobject.c 1998/09/10 17:56:47 1.1
+++ arrayobject.c 1998/09/10 17:56:55
@@ -1724,7 +1724,7 @@
if (max_depth != 0 && ((PyArrayObject *)r)->nd > max_depth) {
Py_DECREF(r);
PyErr_SetString(PyExc_ValueError,
- "Object too deap for desired array");
+ "Object too deep for desired array");
return NULL;
}
return r;
From zcm@llnl.gov Fri Sep 11 18:43:00 1998
From: zcm@llnl.gov (Zane Motteler)
Date: Fri, 11 Sep 1998 09:43:00 -0800
Subject: [Matrix-SIG] [PATCH] fix typo in error message
In-Reply-To: <13816.4990.485785.926485@janus.pgt.com>
Message-ID: <v03007800b21f1138a681@[128.115.36.161]>
Hi Charles,
You wrote:
>I was taken aback to see a misspelled error message come out of NumPy.
>It looks kind of unprofessional... unless "deap" is a technical term
>I've never heard of...
Shucks, you mean you want us to destroy our little bit of history? ;-}
Actually, we have fixed the problem and that message shouldn't
be misspelled in future versions.
Zane
-----------------------------------------------------------------
Zane C. Motteler, Ph. D.
Computer Scientist | Professor Emeritus of
Lawrence Livermore National | Computer Science and Engineering
Laboratory | California Polytechnic
P. O. Box 808, L-038 | State University
Livermore, CA 94551-0808 | San Luis Obispo, CA 93407
925/423-2143, FAX 925/423-9969 | zmottel@hornet.csc.calpoly.edu
zcm@llnl.gov, motteler@icf.llnl.gov
(For FedEx, UPS, etc. use 7000 East Avenue, zip 94550-9234)
From cgw@pgt.com Fri Sep 11 21:02:24 1998
From: cgw@pgt.com (Charles G Waldman)
Date: 11 Sep 1998 16:02:24 -0400
Subject: [Matrix-SIG] Perplexing comparisons
Message-ID: <x44suewh7z.fsf@janus.pgt.com>
The following message is a courtesy copy of an article
that has been posted to comp.lang.python as well.
Can anyone enlighten me as to what's going on here?
Python 1.5 (#13, Sep 9 1998, 15:58:11) [GCC 2.7.2.3] on linux2
Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
>>> t1 = (150,0,1)
>>> t2 = (149,1,0)
>>> t1<t2
0
>>> t2<t1
1
>>> t1>t2
1
>>> t2>t1
0
>>> a1,a2 = array(t1),array(t2)
>>> a1<a2
1
>>> a2<a1
1
>>> a1>a2
0
>>> a2>a1
0
From zcm@llnl.gov Fri Sep 11 23:25:17 1998
From: zcm@llnl.gov (Zane Motteler)
Date: Fri, 11 Sep 1998 14:25:17 -0800
Subject: [Matrix-SIG] Perplexing comparisons
In-Reply-To: <x44suewh7z.fsf@janus.pgt.com>
Message-ID: <v03007804b21f52bb0ab3@[128.115.36.161]>
Charles,
You said:
>The following message is a courtesy copy of an article
>that has been posted to comp.lang.python as well.
>
>
>Can anyone enlighten me as to what's going on here?
>
>Python 1.5 (#13, Sep 9 1998, 15:58:11) [GCC 2.7.2.3] on linux2
>Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
>>>> t1 = (150,0,1)
>>>> t2 = (149,1,0)
>>>> t1<t2
>0
>>>> t2<t1
>1
>>>> t1>t2
>1
>>>> t2>t1
>0
>>>> a1,a2 = array(t1),array(t2)
>>>> a1<a2
>1
>>>> a2<a1
>1
>>>> a1>a2
>0
>>>> a2>a1
>0
What's probably being compared here is something esoteric,
e. g. the addresses of the objects or some such. If you actually
want to do element-by-element comparisons of arrays, use
the functions 'greater', 'less', etc.
>>> less (a1, a2)
array([0, 1, 0])
>>> greater (a1, a2)
array([1, 0, 1])
>>>
Cheers,
Zane
-----------------------------------------------------------------
Zane C. Motteler, Ph. D.
Computer Scientist | Professor Emeritus of
Lawrence Livermore National | Computer Science and Engineering
Laboratory | California Polytechnic
P. O. Box 808, L-038 | State University
Livermore, CA 94551-0808 | San Luis Obispo, CA 93407
925/423-2143, FAX 925/423-9969 | zmottel@hornet.csc.calpoly.edu
zcm@llnl.gov, motteler@icf.llnl.gov
(For FedEx, UPS, etc. use 7000 East Avenue, zip 94550-9234)
From da@skivs.ski.org Fri Sep 11 22:31:13 1998
From: da@skivs.ski.org (David Ascher)
Date: Fri, 11 Sep 1998 14:31:13 -0700 (PDT)
Subject: [Matrix-SIG] Perplexing comparisons
In-Reply-To: <v03007804b21f52bb0ab3@[128.115.36.161]>
Message-ID: <Pine.SUN.3.96.980911142929.9645D-100000@skivs.ski.org>
> >Can anyone enlighten me as to what's going on here?
> >>>> a2<a1
> >1
> >>>> a1>a2
> >0
> >>>> a2>a1
> >0
>
> What's probably being compared here is something esoteric,
> e. g. the addresses of the objects or some such. If you actually
> want to do element-by-element comparisons of arrays, use
> the functions 'greater', 'less', etc.
Note that this is somewhat moot -- the latest versions of NumPy should
prevent this anyway:
>>> a1, a2 = array(t1), array(t2)
>>> a1 < a2
Traceback (innermost last):
File "<stdin>", line 1, in ?
TypeError: Comparison of multiarray objects is not implemented.
There are plans for a 'rich comparison' which would return in your case
the array (0,0,0) for a1 < a2, and (1,0,0) for (a2 < a1).
--david
From cgw@pgt.com Mon Sep 14 22:32:06 1998
From: cgw@pgt.com (Charles G Waldman)
Date: Mon, 14 Sep 1998 17:32:06 -0400 (EDT)
Subject: [Matrix-SIG] NumPy crashes
Message-ID: <13821.35670.396138.772123@janus.pgt.com>
I just upgraded to the latest (?) NumPy, from the LLNLPython5
distribution. I'm finding it distressingly easy to get Python to dump
core when reshaping arrays:
Python 1.5 (#5, Sep 14 1998, 17:26:48) [GCC 2.7.2.3] on linux2
Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
>>> import Numeric
>>> Numeric.__version__
'1.4'
>>> w,h = 512,512
>>> wh = w*h
>>> x,y = Numeric.indices((w,h))
>>> x.shape = wh,1
Segmentation fault (core dumped)
From zcm@llnl.gov Tue Sep 15 00:41:22 1998
From: zcm@llnl.gov (Zane Motteler)
Date: Mon, 14 Sep 1998 15:41:22 -0800
Subject: [Matrix-SIG] NumPy crashes
In-Reply-To: <13821.35670.396138.772123@janus.pgt.com>
Message-ID: <v03007802b22358ecdcb9@[128.115.36.161]>
Charles,
You wrote:
>I just upgraded to the latest (?) NumPy, from the LLNLPython5
>distribution. I'm finding it distressingly easy to get Python to dump
>core when reshaping arrays:
>
>Python 1.5 (#5, Sep 14 1998, 17:26:48) [GCC 2.7.2.3] on linux2
>Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
>>>> import Numeric
>>>> Numeric.__version__
>'1.4'
>>>> w,h = 512,512
>>>> wh = w*h
>>>> x,y = Numeric.indices((w,h))
>>>> x.shape = wh,1
>Segmentation fault (core dumped)
Hmm. I am unable to reproduce this behavior on unix. The
Numeric.reshape function also works.
All I can suggest is running python -v to be sure that you
are loading all of the latest modules.
Zane
-----------------------------------------------------------------
Zane C. Motteler, Ph. D.
Computer Scientist | Professor Emeritus of
Lawrence Livermore National | Computer Science and Engineering
Laboratory | California Polytechnic
P. O. Box 808, L-038 | State University
Livermore, CA 94551-0808 | San Luis Obispo, CA 93407
925/423-2143, FAX 925/423-9969 | zmottel@hornet.csc.calpoly.edu
zcm@llnl.gov, motteler@icf.llnl.gov
(For FedEx, UPS, etc. use 7000 East Avenue, zip 94550-9234)
From mhagger@blizzard.harvard.edu Thu Sep 17 20:20:29 1998
From: mhagger@blizzard.harvard.edu (Michael Haggerty)
Date: Thu, 17 Sep 1998 15:20:29 -0400
Subject: [Matrix-SIG] NumPy crashes
In-Reply-To: <v03007802b22358ecdcb9@[128.115.36.161]> (message from Zane
Motteler on Mon, 14 Sep 1998 15:41:22 -0800)
References: <v03007802b22358ecdcb9@[128.115.36.161]>
Message-ID: <9809171920.AA12738@tsunami.harvard.edu>
> >I just upgraded to the latest (?) NumPy, from the LLNLPython5
> >distribution. I'm finding it distressingly easy to get Python to dump
> >core when reshaping arrays:
> >
> >Python 1.5 (#5, Sep 14 1998, 17:26:48) [GCC 2.7.2.3] on linux2
> >Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
> >>>> import Numeric
> >>>> Numeric.__version__
> >'1.4'
> >>>> w,h = 512,512
> >>>> wh = w*h
> >>>> x,y = Numeric.indices((w,h))
> >>>> x.shape = wh,1
> >Segmentation fault (core dumped)
>
> Hmm. I am unable to reproduce this behavior on unix. The
> Numeric.reshape function also works.
When I do this on DEC Alpha under Digital Unix (64 bit machine), I get
two Unaligned access faults. (These don't cause a core dump, but they
may on cgw's machine. Perhaps this is the clue you are looking for.
Michael
--
Michael Haggerty
mhagger@blizzard.harvard.edu
From ca@germanlloyd.org Thu Sep 24 20:24:38 1998
From: ca@germanlloyd.org (Christian Cabos)
Date: 24 Sep 1998 19:24:38 -0000
Subject: [Matrix-SIG] lexicographic sort and non-conservative argsort
Message-ID: <19980924192438.20814.qmail@findmail.com>
I am trying to sort a Numeric-array lexicographically, i.e. sort by
values in first row. Then, if some values are equal in the first row, use
second row as secondary key. I think the following algorithm is fairly
standard (sort by secondary key first then by primary key),
a=array([[2,2,0,2],
[1,2,3,4]])
# first sort by secondary key
ind2=argsort(a[1,:])
a=take(a,ind2,1)
# then sort by primary key (and hope that the order
# of the secondary key is preserved)
ind1=argsort(a[0,:])
a=take(a,ind1,1)
but under Solaris it yields
[[0 2 2 2]
[3 2 1 4]]
for 'a' which is wrong. Under Linux it yields
[[0 2 2 2]
[3 1 2 4]]
which is correct. The reason is that argsort is not conservative
under Solaris:
argsort(array([2,2,0,2])) is [2 1 0 3]
instead of [2,0,1,3].
Is there any workaround for lexicogrphic sorting in Numeric except for
sorting something like a[0,:]*1000000+a[1,:] which is of very limited
use ?
Can argsort be made conservative on all platforms in the future ?
Christian
Python 1.5 (#3, Jan 12 1998, 08:56:36) [GCC 2.7.2.2] on sunos5
Python 1.5 (#3, Apr 15 1998, 21:17:05) [GCC 2.7.2.1] on linux2
-----
Free e-mail group hosting at http://www.eGroups.com/
From ffjhl@uaf.edu Mon Sep 28 07:21:42 1998
From: ffjhl@uaf.edu (Jonah Lee)
Date: Sun, 27 Sep 1998 22:21:42 -0800 (AKDT)
Subject: [Matrix-SIG] Numpy array.tolist() memory leak ?
Message-ID: <Pine.OSF.3.95.980927221916.10284A-100000@mealpha.engr.uaf.edu>
Hi,
There seems to be a memory leak with the following example:
from Numeric import *
a=array([[1,2,3],[4,5,6],[7,8,9]], Float)
for i in range(100000):
for j in range(100000):
c=a.tolist()
If a=array([1,2,3,4]), then there is no leak.
Any advice on this would be appreciated. Thanks.
Regards,
Jonah
From da@skivs.ski.org Mon Sep 28 07:50:44 1998
From: da@skivs.ski.org (David Ascher)
Date: Sun, 27 Sep 1998 23:50:44 -0700 (Pacific Daylight Time)
Subject: [Matrix-SIG] Numpy array.tolist() memory leak ?
In-Reply-To: <Pine.OSF.3.95.980927221916.10284A-100000@mealpha.engr.uaf.edu>
Message-ID: <Pine.WNT.4.05.9809272346300.142-100000@david.ski.org>
On Sun, 27 Sep 1998, Jonah Lee wrote:
>
> There seems to be a memory leak with the following example:
>
> from Numeric import *
> a=array([[1,2,3],[4,5,6],[7,8,9]], Float)
> for i in range(100000):
> for j in range(100000):
> c=a.tolist()
>
> If a=array([1,2,3,4]), then there is no leak.
>
> Any advice on this would be appreciated. Thanks.
I could be wrong (reference counting was never my forte), but I believe
that arrayobject.c's PyArray_ToList() should be modified to read:
...
for (i=0; i<sz; i++) {
! PyArrayObject *elt;
! elt = array_item((PyArrayObject *)self, i);
! PyList_SetItem(lp, i, PyArray_ToList(elt));
! Py_DECREF(elt);
}
...
otherwise the subarrays ([1,2,3] etc. in the example above) never get
DECREF'ed out.
--david ascher
From ryszard@moldyn.com Mon Sep 28 15:58:58 1998
From: ryszard@moldyn.com (Ryszard Czerminski)
Date: Mon, 28 Sep 1998 10:58:58 -0400 (EDT)
Subject: [Matrix-SIG] how to contribute new code ?
Message-ID: <Pine.LNX.3.95.980928105327.361J-100000@mpc2>
I was looking recently for routine which would solve linear
least square problem A*X = B with constrains X >= 0.
I have found fortran code for this
and I have written Python interface
to it using lapack_litemodule from NumPy as a template.
It seems to me that access to this code via python
may be of general interest.
What is the usual way of contributing
such a code to Numpy ?
Ryszard Czerminski phone : (617)354-3124 x 10
Moldyn, Inc. fax : (617)491-4522
955 Massachusetts Avenue e-mail: ryszard@moldyn.com
Cambridge MA, 02139-3180 or ryszard@photon.com
From da@skivs.ski.org Mon Sep 28 17:27:42 1998
From: da@skivs.ski.org (David Ascher)
Date: Mon, 28 Sep 1998 09:27:42 -0700 (Pacific Daylight Time)
Subject: [Matrix-SIG] how to contribute new code ?
In-Reply-To: <Pine.LNX.3.95.980928105327.361J-100000@mpc2>
Message-ID: <Pine.WNT.4.04.9809280914100.269-100000@rigoletto.ski.org>
On Mon, 28 Sep 1998, Ryszard Czerminski wrote:
>
> I was looking recently for routine which would solve linear
> least square problem A*X = B with constrains X >= 0.
>
> I have found fortran code for this
> and I have written Python interface
> to it using lapack_litemodule from NumPy as a template.
>
> It seems to me that access to this code via python
> may be of general interest.
Certainly.
> What is the usual way of contributing
> such a code to Numpy ?
There is no 'usual'. There are two possible ways to distribute such code
a) within the LLNL distribution, and b) on the net. Currently, I think
the LLNL folks don't want to take the responsibility to distribute code
they don't 'own'. The maintenance and release process is too cumbersome
to allow for that (e.g. if the author of a contributed package decides to
update his package every day, there's a problem). That leaves b), on the
net. There is a project in development to develop a generic architecture
for dealing with software archives (the Trove project, led by Eric
Raymond). It is not at a useable stage, however. In the meantime, if
someone wants to publish on the web and doesn't have a website for it,
I'll gladly put things up on my starship account (assuming not-to-frequent
releases), as I did for two modules from Travis Oliphant
(http://starship.skyport.net/~da/Travis/).
This doesn't solve some of the other problems, such as how to categorize
NumPy-related software. There are standard schemes for catogization, but
I believe they won't work without a skilled 'librarian' who knows enough
about all of the various topics to do the categorization right. Using a
completely informal approach, I have started a "Topic Guide", which is
available at http://www.python.org/topics/scicomp/. It is rough at this
point, but I intend to have it cleaned up before the Python Conference --
feel free to email me comments and suggestions. Also note that there is
the framework for a NumPy FAQ at http://www.python.org/cgi-bin/numpy-faq,
which I encourage people to add entries to (no password is necessary --
leave the password field blank).
Cheers,
--david
From cgw@pgt.com Tue Sep 29 18:12:06 1998
From: cgw@pgt.com (Charles G Waldman)
Date: 29 Sep 1998 13:12:06 -0400
Subject: [Matrix-SIG] Syntactic sugar for constructing arrays
Message-ID: <x47lymu9ll.fsf@janus.pgt.com>
The following message is a courtesy copy of an article
that has been posted to comp.lang.python as well.
The more I use Numeric Python, the more I appreciate the expressive
power it adds to Python. I do a fair amount of geometric work, and I
find it very elegant to be able to write things like:
point = point+offset
or
def distance(p1,p2):
return sqrt(sum((p1-p2)**2))
where point, offset, p1, p2 are 1-dimensional NumPy arrays. This
takes Python, IMO, to a level of sophistication well beyond a "nifty
easy-to-use scripting language"; it becomes a very attractive
environment for doing math/scientific work.
I used to write X-Y coordinates as 2-tuples, but things like vector
addition are much less elegant this way than they are if I use NumPy
arrays. The more I use Numeric Python, the more I realize that for
lots of mathematical structures like points, a 1-D NumPy array is a
much better representation than a tuple.
The downside of this is that my code is now littered with the construction:
center = array((256,256))
point = array((10,-20))
etc.
Since the array is such a useful type, I'd like to see a
cleaner-looking syntax for creating array objects. I'd like to be
able to write, for instance,
center = <256,256>
point = <10,-20>
What do people think of this suggestion?
From Paul F. Dubois" <dubois1@llnl.gov Tue Sep 29 18:46:20 1998
From: Paul F. Dubois" <dubois1@llnl.gov (Paul F. Dubois)
Date: Tue, 29 Sep 1998 10:46:20 -0700
Subject: [Matrix-SIG] how to contribute new code ?
Message-ID: <001601bdebd1$14b8f5c0$f4160218@c1004579-C.plstn1.sfba.home.com>
David correctly summarizes our view: we don't think having us holding these
packages is a good idea. All other ideas have problems, too. The starship
seems like one solution in the long run, with each author managing their own
bit.
-----Original Message-----
From: David Ascher <da@skivs.ski.org>
To: Ryszard Czerminski <ryszard@moldyn.com>
Cc: matrix-sig@python.org <matrix-sig@python.org>
Date: Monday, September 28, 1998 9:33 AM
Subject: Re: [Matrix-SIG] how to contribute new code ?
>
>
>On Mon, 28 Sep 1998, Ryszard Czerminski wrote:
>>
>> I was looking recently for routine which would solve linear
>> least square problem A*X = B with constrains X >= 0.
>>
>> I have found fortran code for this
>> and I have written Python interface
>> to it using lapack_litemodule from NumPy as a template.
>>
>> It seems to me that access to this code via python
>> may be of general interest.
>
>Certainly.
>
>> What is the usual way of contributing
>> such a code to Numpy ?
>
>There is no 'usual'. There are two possible ways to distribute such code
>a) within the LLNL distribution, and b) on the net. Currently, I think
>the LLNL folks don't want to take the responsibility to distribute code
>they don't 'own'. The maintenance and release process is too cumbersome
>to allow for that (e.g. if the author of a contributed package decides to
>update his package every day, there's a problem). That leaves b), on the
>net. There is a project in development to develop a generic architecture
>for dealing with software archives (the Trove project, led by Eric
>Raymond). It is not at a useable stage, however. In the meantime, if
>someone wants to publish on the web and doesn't have a website for it,
>I'll gladly put things up on my starship account (assuming not-to-frequent
>releases), as I did for two modules from Travis Oliphant
>(http://starship.skyport.net/~da/Travis/).
>
>This doesn't solve some of the other problems, such as how to categorize
>NumPy-related software. There are standard schemes for catogization, but
>I believe they won't work without a skilled 'librarian' who knows enough
>about all of the various topics to do the categorization right. Using a
>completely informal approach, I have started a "Topic Guide", which is
>available at http://www.python.org/topics/scicomp/. It is rough at this
>point, but I intend to have it cleaned up before the Python Conference --
>feel free to email me comments and suggestions. Also note that there is
>the framework for a NumPy FAQ at http://www.python.org/cgi-bin/numpy-faq,
>which I encourage people to add entries to (no password is necessary --
>leave the password field blank).
>
>Cheers,
>
>--david
>
>
>
>
>
>
>_______________________________________________
>Matrix-SIG maillist - Matrix-SIG@python.org
>http://www.python.org/mailman/listinfo/matrix-sig
>
>
From ryszard@moldyn.com Tue Sep 29 19:10:46 1998
From: ryszard@moldyn.com (Ryszard Czerminski)
Date: Tue, 29 Sep 1998 14:10:46 -0400 (EDT)
Subject: [Matrix-SIG] how to contribute new code ?
In-Reply-To: <001601bdebd1$14b8f5c0$f4160218@c1004579-C.plstn1.sfba.home.com>
Message-ID: <Pine.LNX.3.95.980929135453.363C-100000@mpc2>
Well, this approach (each author managing their own
bit) makes perfect sense for larger and
distinct modules/packages.
In case like this one, i.e. well defined
addition (*) to already existing and well defined
package (in this case LinearAlgebra) keeping it
separetely seems to be somewhat disadvantageous.
Some obvious disadvantages are:
(1) harder to find for somebody looking for it
(2) harder to install
With best regards,
Ryszard
(*)
c The original version of this code was developed by
c Charles L. Lawson and Richard J. Hanson at Jet Propulsion Laboratory
c 1973 JUN 15, and published in the book
c "SOLVING LEAST SQUARES PROBLEMS", Prentice-HalL, 1974.
Ryszard Czerminski phone : (617)354-3124 x 10
Moldyn, Inc. fax : (617)491-4522
955 Massachusetts Avenue e-mail: ryszard@moldyn.com
Cambridge MA, 02139-3180 or ryszard@photon.com
On Tue, 29 Sep 1998, Paul F. Dubois wrote:
> David correctly summarizes our view: we don't think having us holding these
> packages is a good idea. All other ideas have problems, too. The starship
> seems like one solution in the long run, with each author managing their own
> bit.
>
> -----Original Message-----
> From: David Ascher <da@skivs.ski.org>
> To: Ryszard Czerminski <ryszard@moldyn.com>
> Cc: matrix-sig@python.org <matrix-sig@python.org>
> Date: Monday, September 28, 1998 9:33 AM
> Subject: Re: [Matrix-SIG] how to contribute new code ?
>
>
> >
> >
> >On Mon, 28 Sep 1998, Ryszard Czerminski wrote:
> >>
> >> I was looking recently for routine which would solve linear
> >> least square problem A*X = B with constrains X >= 0.
> >>
> >> I have found fortran code for this
> >> and I have written Python interface
> >> to it using lapack_litemodule from NumPy as a template.
> >>
> >> It seems to me that access to this code via python
> >> may be of general interest.
> >
> >Certainly.
> >
> >> What is the usual way of contributing
> >> such a code to Numpy ?
> >
> >There is no 'usual'. There are two possible ways to distribute such code
> >a) within the LLNL distribution, and b) on the net. Currently, I think
> >the LLNL folks don't want to take the responsibility to distribute code
> >they don't 'own'. The maintenance and release process is too cumbersome
> >to allow for that (e.g. if the author of a contributed package decides to
> >update his package every day, there's a problem). That leaves b), on the
> >net. There is a project in development to develop a generic architecture
> >for dealing with software archives (the Trove project, led by Eric
> >Raymond). It is not at a useable stage, however. In the meantime, if
> >someone wants to publish on the web and doesn't have a website for it,
> >I'll gladly put things up on my starship account (assuming not-to-frequent
> >releases), as I did for two modules from Travis Oliphant
> >(http://starship.skyport.net/~da/Travis/).
> >
> >This doesn't solve some of the other problems, such as how to categorize
> >NumPy-related software. There are standard schemes for catogization, but
> >I believe they won't work without a skilled 'librarian' who knows enough
> >about all of the various topics to do the categorization right. Using a
> >completely informal approach, I have started a "Topic Guide", which is
> >available at http://www.python.org/topics/scicomp/. It is rough at this
> >point, but I intend to have it cleaned up before the Python Conference --
> >feel free to email me comments and suggestions. Also note that there is
> >the framework for a NumPy FAQ at http://www.python.org/cgi-bin/numpy-faq,
> >which I encourage people to add entries to (no password is necessary --
> >leave the password field blank).
> >
> >Cheers,
> >
> >--david
> >
> >
> >
> >
> >
> >
> >_______________________________________________
> >Matrix-SIG maillist - Matrix-SIG@python.org
> >http://www.python.org/mailman/listinfo/matrix-sig
> >
> >
>
>
> _______________________________________________
> Matrix-SIG maillist - Matrix-SIG@python.org
> http://www.python.org/mailman/listinfo/matrix-sig
>
From arb@connect.com.au Wed Sep 30 10:15:58 1998
From: arb@connect.com.au (Anthony Baxter)
Date: Wed, 30 Sep 1998 19:15:58 +1000
Subject: [Matrix-SIG] LLNLPython5 on Solaris with Sparcworks compiler...
Message-ID: <199809300915.TAA28952@koro.off.connect.com.au>
In the CXX directory, "python makethis.py" fails on the first file...
CC -g -O2 -I/opt/python/include/python1.5 -I/opt/python/include/python1.5 -DHAVE_CONFIG_H -I./Include -I./Demo -c ./Demo/r.cxx
"./Include/CXX_Exception.h", line 12: Error: Could not open include file <string>.
"./Include/CXX_Exception.h", line 13: Error: Could not open include file <iostream>.
"./Include/CXX_Exception.h", line 15: Error: Type name expected instead of "namespace".
"./Include/CXX_Exception.h", line 25: Error: Type name expected instead of "Exception".
"./Include/CXX_Exception.h", line 25: Error: std is not defined.
"./Include/CXX_Exception.h", line 25: Error: string must be initialized.
"./Include/CXX_Exception.h", line 25: Error: "," expected instead of "&".
"./Include/CXX_Exception.h", line 27: Error: Unexpected "}".
"./Include/CXX_Exception.h", line 34: Error: Unexpected "}".
"./Include/CXX_Exception.h", line 36: Error: Exception is not a class name as required for a qualifer.
"./Include/CXX_Exception.h", line 36: Error: Type name expected instead of "{".
"./Include/CXX_Exception.h", line 38: Error: No direct declarator preceding "(".
"./Include/CXX_Exception.h", line 45: Error: Exception is not a class name as required for a qualifer.
"./Include/CXX_Exception.h", line 45: Error: Type name expected instead of "{".
"./Include/CXX_Exception.h", line 47: Error: No direct declarator preceding "(".
"./Include/CXX_Exception.h", line 54: Error: Exception is not a class name as required for a qualifer.
"./Include/CXX_Exception.h", line 54: Error: Type name expected instead of "{".
"./Include/CXX_Exception.h", line 56: Error: No direct declarator preceding "(".
"./Include/CXX_Exception.h", line 63: Error: Exception is not a class name as required for a qualifer.
"./Include/CXX_Exception.h", line 63: Error: Type name expected instead of "{".
"./Include/CXX_Exception.h", line 65: Error: No direct declarator preceding "(".
"./Include/CXX_Exception.h", line 72: Error: Exception is not a class name as required for a qualifer.
"./Include/CXX_Exception.h", line 72: Error: Type name expected instead of "{".
"./Include/CXX_Exception.h", line 74: Error: No direct declarator preceding "(".
"./Include/CXX_Exception.h", line 81: Error: Exception is not a class name as required for a qualifer.
Compilation aborted, too many Error messages.
*** Error code 1
make: Fatal error: Command failed for target `r.o'
From Anthony Baxter <arb@connect.com.au> Wed Sep 30 10:44:48 1998
From: Anthony Baxter <arb@connect.com.au> (Anthony Baxter)
Date: Wed, 30 Sep 1998 19:44:48 +1000
Subject: [Matrix-SIG] LLNLPython5 on Solaris with Sparcworks compiler...
In-Reply-To: Your message of "Wed, 30 Sep 1998 19:15:58 +1000."
<199809300915.TAA28952@koro.off.connect.com.au>
Message-ID: <199809300944.TAA11383@koro.off.connect.com.au>
>>> Anthony Baxter wrote
> In the CXX directory, "python makethis.py" fails on the first file...
D'oh. I was too quick to post. I found the COMPILER_SUPPORTS_NAMESPACES
option in the Include/CXX_Config.h, but still no joy. Will keep working
at it...
From Paul F. Dubois" <dubois1@llnl.gov Wed Sep 30 17:23:49 1998
From: Paul F. Dubois" <dubois1@llnl.gov (Paul F. Dubois)
Date: Wed, 30 Sep 1998 09:23:49 -0700
Subject: [Matrix-SIG] LLNLPython5 on Solaris with Sparcworks compiler...
Message-ID: <007701bdec8e$b5982480$f4160218@c1004579-C.plstn1.sfba.home.com>
The no namespaces option is one I have no personal way to test, so I put it
in for someone who told me they got it to work. However, in making recent
changes I may have messed it up again, and of course I have no idea what
will happen with the Sparcworks compiler. I don't know how old your compiler
is but you might want to see if they have a new one closer to the standard.
If you have any fixes I'd be happy to get them.
-----Original Message-----
From: Anthony Baxter <arb@connect.com.au>
Cc: matrix-sig@python.org <matrix-sig@python.org>
Date: Wednesday, September 30, 1998 2:47 AM
Subject: Re: [Matrix-SIG] LLNLPython5 on Solaris with Sparcworks compiler...
>
>>>> Anthony Baxter wrote
>> In the CXX directory, "python makethis.py" fails on the first file...
>
>D'oh. I was too quick to post. I found the COMPILER_SUPPORTS_NAMESPACES
>option in the Include/CXX_Config.h, but still no joy. Will keep working
>at it...
>
>_______________________________________________
>Matrix-SIG maillist - Matrix-SIG@python.org
>http://www.python.org/mailman/listinfo/matrix-sig
>
>