Mailman 3 November 2005 - NumPy-Discussion

A small glitch remains in CVS numarray
by Francesc Altet 12 Nov '05

12 Nov '05

Hi, I've checked CVS numarray today and yes, the problem with strided arrays seems to be gone. It does remain some small (I think) issue: >>> import Numeric;Numeric.__version__ '24.1' >>> import numarray;numarray.__version__ '1.4.2' >>> num=Numeric.array([0.,1.]) >>> na=numarray.zeros(shape=2) >>> na[...] = num Traceback (most recent call last): File "<stdin>", line 1, in ? TypeError: NA_DescrFromType: unknown type: -1 and also: >>> numarray.array(num) Traceback (most recent call last): File "<stdin>", line 1, in ? File "/usr/lib/python2.4/site-packages/numarray/numarraycore.py", line 359, in array a = _numarray._array_from_array_struct(sequence.__array_struct__) TypeError: NA_DescrFromType: unknown type: -1 Regards, -- >0,0< Francesc Altet http://www.carabos.com/ V V Cárabos Coop. V. Enjoy Data "-"

2 1

SciPy SVN altered so that newscipy and newcore are now the trunk.
by Travis Oliphant 10 Nov '05

10 Nov '05

I've finished altering the subversion repository of SciPy so that the new development is taking place on the trunk of both scipy and scipy_core. The old versions are under branches named oldcore and oldscipy. Get the new repositor(y,ies) using: *Core*: svn co http://svn.scipy.org/svn/scipy_core/trunk core *Full SciPy*: svn co http://svn.scipy.org/svn/scipy/trunk scipy Doing both will place two directories named core and scipy in your current directory containing the current state of both repositories. python setup.py install should work in each directory. The Freeze is now over. I want to track down the bug that Christopher Hanley noted and another f2py-related bug before making a release, which I expect to happen by the weekend. -Travis

2 1

Re: [Numpy-discussion] Writing/reading a numeric array to a file
by Niklas Volbers 10 Nov '05

10 Nov '05

First of all, thanks for your answers. Travis Oliphant <oliphant(a)ee.byu.edu> schrieb am 09.11.05 20:23:16: > What unneeded functionality is there. SciPy Core is just a Numeric > replacement. It does not NEED Atlas, it just uses it if you have it --- > exactly the same as Numeric and numarray. > It is a misconception to say scipy core needs any thing else but Python > installed. So, let's not let that rumor kill convergence to a single > package. I am sorry for this faulty assumption of mine which you have now clarified. I had assumed that ATLAS was needed, just because scipy_core would not build on my system w/o a prior install of these libraries. With the knowledge, that it _should_ work, I have just retried to build scipy_core 0.4.2 on my rather fresh installation of Slackware Linux 10.2 (python 2.4.1) and still couldn't get it to build. I have attached the log files out.txt and err.txt to this e-mail, which I got by typing $ python setup.py build 1>out.txt 2>err.txt in the scipy_core main directory. The two files are compressed into a single tar.gz archive (5k), because I do not know if 60k attachments are o.k. on this list. Maybe you can help me solve my problem? > > (4) What solutions do you use to save/load data to files? > > SciPy core arrays have tofile methods and a fromfile function. They are > raw reading and writing --- nothing fancy. You need to use Pickling if > you want to recognize endian-ness among platforms. What is your > opposition to Pickling? >From how I understood pickling, it converts the python objects to a binary representation. What I want is a data representation that is independent of python. Anyway, I now have a few possibilities to explore besides netcdf (with the alternative file formats FITS and with pytables, i.e. HDF5, proposed here). > I think you should take a look at PyTables for more elegant solutions. I wasn't sure if it will stay compatible to scipy_core, because right now it uses numarray. But from what you said I am sure that eventually pytables might also switch to scipy_core. Best regards, and thanks for your efforts, Niklas. ______________________________________________________________________ XXL-Speicher, PC-Virenschutz, Spartarife & mehr: Nur im WEB.DE Club! Jetzt gratis testen! http://freemail.web.de/home/landingpad/?mc=021130

4 3

BUG!
by Sune Rastad Bahn 10 Nov '05

10 Nov '05

Dear devs, Thanks a lot for a very nice and extremely useful python module. I recommend all my colleagues to try you package whenever they ask for a matlab like environment. This week one of these friends came by me and showed me a bug in RandomArray. try the following python -c "from RandomArray import *; seed(1874764637, 1152239787);print randint(0,1000)" and notice how the result is the upper limit (1000) even though the documentation clearly states that the number should be strictly below 1000. My guess is that this is an issue with the underlying ranlib library, but perhaps someone can clarify? best, Sune -- Software Application Manager Sune Rastad Bahn Phone: +45 35 320 638 Mobile: +45 23 455 997 Email: bahn(a)atomistix.com

3 2

Re: [Numpy-discussion] Yet another problem in CVS numarray
by Todd Miller 10 Nov '05

10 Nov '05

Francesc Altet wrote: >El dv 04 de 11 del 2005 a les 15:35 -0500, en/na Todd Miller va >escriure: > > >>This turned out to be a problem with the way numarray handles Numeric's >>multi-segment buffer protocol. I worked around this by implementing >>David Cooke's __array_struct__ array interface for numarray. >> >> > >Did you commit the changes in CVS? I'm getting the same problems with >the current CVS version: > > > >>>>Numeric.__version__ >>>> >>>> >'24.1' > > >>>>import numarray >>>>numarray.__version__ >>>> >>>> >'1.4.2' > > >>>>num=Numeric.array([1,2,3,4]) >>>>numarray.array(num) >>>> >>>> >array([1, 2, 3, 4]) > > >>>>num2=num[::2] >>>>numarray.array(num2) >>>> >>>> >Traceback (most recent call last): > File "<stdin>", line 1, in ? > File "/usr/lib/python2.4/site-packages/numarray/numarraycore.py", line >380, in array > a = a.astype(type) > File "/usr/lib/python2.4/site-packages/numarray/numarraycore.py", line >867, in astype > return self.copy() > File "/usr/lib/python2.4/site-packages/numarray/numarraycore.py", line >927, in copy > c = _gen.NDArray.copy(self) > File "/usr/lib/python2.4/site-packages/numarray/generic.py", line 724, >in copy > arr._itemsize) >numarray.libnumarray.error: copy4bytes: access beyond buffer. offset=11 >buffersize=8 > > This turned out to be a bug in numarray buffer size determination... striding wasn't accounted for so the buffer appeared to be too small. It's fixed now in CVS. There's still a backward compatibility problem for old Numerics which don't implement __array_struct__ and get multi-segment buffers through __array_data__. Regards, Todd

2 3

Writing/reading a numeric array to a file
by N. Volbers 10 Nov '05

10 Nov '05

Hello everyone, I have been using Numeric and/or numarray for a while now and I now I have a question concerning the reading and writing data to a file. Until now I had used pycdf, which is an interface to the netcdf library, to save and load my numeric arrays. However, since my application already has too many dependencies, I would like to cut down on these two and replace the saving and loading by a method intrinisc to Numeric or numarray. Pickling is not an alternative to me, because I need to have a representation that can be read by other programs (and yet is faster than ASCII). NetCDF was a good choice for that, but I would really like to not depend on it. So my questions are: (1) Are the numarray functions 'fromfile' and 'tofile' 100% portable among platforms, i.e. do they automatically recognize endian-ness and such? (2) Does it make sense to still use numarray? I know Travis would say "use scipy_core". However, for me this would provide much unneeded functionality and I have not yet found an easy way to install scipy_core (it seems to require ATLAS and such, which are not so easy to install if you don't have it prepackaged). And after all, my goal is to cut down dependencies... (3) Let me restate question (2): Will numarray still be maintained? Or is it also deprecated? What would you advice someone who just needs the array interface? And of course, (4) What solutions do you use to save/load data to files? I would deeply appreciate any help on this subject, Niklas Volbers.

5 4

Bug or feature?
by Sune Rastad Bahn 10 Nov '05

10 Nov '05

Dear devs, Thanks a lot for a very nice and extremely useful python module. I recommend all my colleagues to try you package whenever they ask for a matlab like environment. This week one of these friends came by me with an oddity relating to the RandomArray module. try the following on your linux command prompt: while true; do python -c "from RandomArray import *;seed();print randint(100000)"; done and check the output. I get a steadily but slowly decreasing series of numbers. e.g: 4013 4013 4012 4012 4012 4012 4012 4012 4012 4012 4010 4010 4010 According to the documentation seed() should initiatilize from the system clock. If one checks the seed (using get_seed) it turns out to be the case indeed, but that it is only very slowly changing (once a second), and only on the second seed. e.g python -c "from RandomArray import *;seed();print get_seed()" (113161, 3079) (113161, 3079) (113161, 3080) (113161, 3080) (113161, 3080) (113161, 3080) My best guess is that this is the cause for the behavior seen above. This could be filed as a feature, but I believe it is not the behavior people would expect. At least the documentation should emphasize that two runs started within a short time ( upto 1s) will give the exact same result (experts on pseudo random numbers will know this, but the length of the time 1sec is surprisingly long). that the first random number is slowly varying (strongly correlated) with time (this is more surprising). One fix is to take one randomnumber and throw away, since the second number is not correlated with time (except from noted above). Best, Sune -- Software Application Manager Sune Rastad Bahn Phone: +45 35 320 638 Mobile: +45 23 455 997 Email: bahn(a)atomistix.com

1 0

Python egg support in Numeric 24.2 in CVS
by David M. Cooke 09 Nov '05

09 Nov '05

I've added support to Numeric to allow a Python egg [1] to be made easily. Basically, run $ python setup.py bdist_egg and an .egg file should be laid in dist/, which you can install with easy_install. This will only work if you have setuptools installed. I've taken steps so that usual 'setup.py build; setup.py install' will still install Numeric the old way (with a .pth file). The header files for Numeric are now also installed in a package Numeric_headers, which contains a function get_numeric_include(). This returns the directory in which the headers are installed in, so you can use it in your setup.py. The docstring for the function explains how to use it. It's modelled after get_scipy_include() in scipy.base. This way, the header files are included in the egg, and can be used. I'd appreciate it if someone with eggperience (Robert?) could take a look at it before Travis collects Numeric for 24.2. [1] http://peak.telecommunity.com/DevCenter/PythonEggs -- |>|\/|< /------------------------------------------------------------------\ |David M. Cooke http://arbutus.physics.mcmaster.ca/dmc/ |cookedm(a)physics.mcmaster.ca

1 0

Numeric CVS __array_struct__ interface is broken on 64 bit platforms
by Andrew Straw 09 Nov '05

09 Nov '05

Hi, A couple bugs have been reported (including mine last night) which indicate a problem with the following bit of code in Numerical/Src/arrayobject.c (near line 2200) on 64 bit platforms. I think it'll be a lot faster for someone else to fix it, so I'll leave it at this for now. if (strcmp(name, "__array_struct__") == 0) { PyArrayInterface *inter; inter = (PyArrayInterface *)malloc(sizeof(PyArrayInterface)); inter->version = 2; inter->nd = self->nd; if ((inter->nd == 0) || (sizeof(int) == sizeof(Py_intptr_t))) { inter->shape = (Py_intptr_t *)self->dimensions; inter->strides = (Py_intptr_t *)self->strides; } else { int i; for (i=0; i<self->nd; i++) { inter->shape[i] = self->dimensions[i]; inter->strides[i] = self->strides[i]; } }

1 0

initializing new ufuncs
by Stefan Kuzminski 08 Nov '05

08 Nov '05

Hi, I'm writing some new ufuncs, and I'm wondering where I can insert some initialization and finalization code for each ufunc invocation. I can see where it would go in the generated code, but I don't know how to do it through the code generation API. thanks, Stefan Kuzminski __________________________________ Start your day with Yahoo! - Make it your home page! http://www.yahoo.com/r/hs

1 0