From b.hall at irl.cri.nz Thu Jun 5 16:23:07 2003 From: b.hall at irl.cri.nz (Blair Hall) Date: Thu Jun 5 16:23:07 2003 Subject: [Numpy-discussion] Having difficulty with Matrix.py Message-ID: <5.2.0.9.1.20030606111256.01e6f248@pop.wgtn.irl.cri.nz> I have been caught out writing a simple function that manipulates the elements of a matrix. I have distilled the problem into the code below. I find that 'sum' works with a Numeric array but fails with a Matrix. Am I doing something wrong? If not, what should I do to patch it? (I am working under Windows with Numpy 23.0) Here is the code: # 'a' is a two dimensional array. The # function simply places the sum of # the first column in a[0,0] def sum(a): N = a.shape[0] sum = 0 for i in range(N): sum += a[i,0] # This is the criticial line a[0,0] = sum return a[0,0] #------------------------ if(__name__ == '__main__'): import Numeric import Matrix a = Numeric.ones( (9,1) ) print sum( a ) # Ok print sum( Matrix.Matrix( a ) ) # Fails From oliphant.travis at ieee.org Fri Jun 6 10:11:08 2003 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Fri Jun 6 10:11:08 2003 Subject: [Numpy-discussion] Matrix.py has been fixed in CVS References: <5.2.0.9.1.20030606111256.01e6f248@pop.wgtn.irl.cri.nz> Message-ID: <3EE0CAED.60100@ieee.org> This is a bug in Matrix.py which has been fixed. Here is the relevant section of code to replace the current __setitem__ command in the Matrix class with: def __setitem__(self, index, value): value = asarray(value, self._typecode) self.array[index] = squeeze(value) Good luck, Travis O. > > I find that 'sum' works with a Numeric > array but fails with a Matrix. Am I doing something > wrong? If not, what should I do to patch it? > From pete at shinners.org Sun Jun 8 11:41:03 2003 From: pete at shinners.org (Pete Shinners) Date: Sun Jun 8 11:41:03 2003 Subject: [Numpy-discussion] convert to numarray Message-ID: <3EE38292.4030205@shinners.org> in the pygame project i've had the ability to map images into Numeric data arrays onto image pixel data. This has worked excellently for me, but in the near future i'd also like to support numarray. early in numarray's development it looked like this was not going to be possible at all. i've been following numarray loosely, and it sure looks like things have 'loosened' up a bit. still there are a few things i am doing that i'm unsure if numarray is ready to handle yet? what is going to make this tricky is i'm doing a bit of 'attribute mangling' to the Numeric array structure. this is necessary as the image data is extremely 'non-flat'. also, since i am referencing data held in another python object, i need to make sure the array holds a reference to the original object. these are the things i'm afraid i'll be stuck on. here is pretty much what i am doing now, simplified quite a bit... PyObject* pixelsarray(SDL_Surface *surf) { int dim[3]; PyObject *array; dim[0] = surf->w; dim[1] = surf->h; dim[2] = 3; array = PyArray_FromDimsAndData(3, dim, PyArray_UBYTE, surf->pixels); if(array) { PyArrayObject *a = (PyArrayObject*)array a->flags = OWN_DIMENSIONS|OWN_STRIDES; a->strides[2] = 1; a->strides[1] = surf->pitch; a->strides[0] = surf->format->BytesPerPixel; a->base = _pyobject_to_surf_; } return array; } note that depending on pixel packing and endianess, the strides[2] can become -1. smiley. also pretend a->base is pointing to a real python object, which it does in the real version. there is likely a way to workaround the "base" requirement with weakrefs i suppose, but i'd rather not jump through the extra hoops. the real necessity is setting the strides how i want. i didn't see any array creation functions that allow me to pick my own strides. once i create the array here i never change any of the array attributes. if this looks doable then it's time for me to sit down with the numarray docs and see what new and exciting things await me :] From jmiller at stsci.edu Mon Jun 9 06:41:02 2003 From: jmiller at stsci.edu (Todd Miller) Date: Mon Jun 9 06:41:02 2003 Subject: [Numpy-discussion] convert to numarray In-Reply-To: <3EE38292.4030205@shinners.org> References: <3EE38292.4030205@shinners.org> Message-ID: <1055165976.9644.55.camel@halloween.stsci.edu> On Sun, 2003-06-08 at 14:38, Pete Shinners wrote: > still there are a few things i am > doing that i'm unsure if numarray is ready to handle yet? > > what is going to make this tricky is i'm doing a bit of 'attribute > mangling' to the Numeric array structure. this is necessary as the image > data is extremely 'non-flat'. also, since i am referencing data held in > another python object, i need to make sure the array holds a reference > to the original object. these are the things i'm afraid i'll be stuck on. > > here is pretty much what i am doing now, simplified quite a bit... > > PyObject* pixelsarray(SDL_Surface *surf) > { > int dim[3]; > PyObject *array; > > dim[0] = surf->w; > dim[1] = surf->h; > dim[2] = 3; > array = PyArray_FromDimsAndData(3, dim, > PyArray_UBYTE, surf->pixels); > if(array) > { > PyArrayObject *a = (PyArrayObject*)array > a->flags = OWN_DIMENSIONS|OWN_STRIDES; > a->strides[2] = 1; > a->strides[1] = surf->pitch; > a->strides[0] = surf->format->BytesPerPixel; > a->base = _pyobject_to_surf_; > } > return array; > } > > note that depending on pixel packing and endianess, the strides[2] can > become -1. smiley. Despite old documentation to the contrary, the strides array is now writable. The -1 is more problematic. Anytime a ufunc is called, there is "safety code" which evaluates the shape, strides, byteoffset, buffer base and buffer size to ensure that the ufunc won't attempt an access outside the buffer or generate a misaligned pointer. negative strides are possible *provided* that the byteoffset is chosen so that everything works out. > also pretend a->base is pointing to a real python > object, which it does in the real version. numarray has a "base" pointer which is currently unused and intended for the purpose of backward compatibility. Since numarray does not yet use the base pointer at all, it does not deallocate it at array destruction time. > there is likely a way to workaround the "base" requirement with weakrefs > i suppose, but i'd rather not jump through the extra hoops. the real > necessity is setting the strides how i want. i didn't see any array > creation functions that allow me to pick my own strides. New in numarray-0.5 is the API function NA_NewAllStrides which allows you to pass in a pointer to a strides array. NA_NewAllStrides does not currently check the validity of the strides array. > once i create > the array here i never change any of the array attributes. > > if this looks doable then it's time for me to sit down with the numarray > docs and see what new and exciting things await me :] It looks close to doable to me. There are probably still a few loose ends. Todd -- Todd Miller jmiller at stsci.edu STSCI / ESS / SSB From felix at fbreuer.de Wed Jun 11 14:32:06 2003 From: felix at fbreuer.de (Felix Breuer) Date: Wed Jun 11 14:32:06 2003 Subject: [Numpy-discussion] OverflowError when importing RandomArray Message-ID: <1055366499.25112.65.camel@tapir> Hello! I am using Numeric 0.23 with Python 2.2 on my Gentoo Linux box. Problem: >>> from Numeric import * >>> import RandomArray Traceback (most recent call last): File "", line 1, in ? File "/usr/lib/python2.2/site-packages/Numeric/RandomArray.py", line 30, in ? seed() File "/usr/lib/python2.2/site-packages/Numeric/RandomArray.py", line 26, in seed x = int(t/base) OverflowError: float too large to convert Is this a bug, a misconfiguration on my system or a misusage on my part? Opinions, anyone? Thanks, Felix From 237506 at mail.com Fri Jun 13 01:07:02 2003 From: 237506 at mail.com (237506 at mail.com) Date: Fri Jun 13 01:07:02 2003 Subject: [Numpy-discussion] 237506 E-mail recording 1117571091 Message-ID: 12159CDA-243323A0-58FA3720-30BD2546-63FB8503@hotmail.com An HTML attachment was scrubbed... URL: From jmiller at stsci.edu Fri Jun 13 07:48:13 2003 From: jmiller at stsci.edu (Todd Miller) Date: Fri Jun 13 07:48:13 2003 Subject: [Numpy-discussion] numarray repackaging... as it turned out Message-ID: <1055515666.30709.80.camel@halloween.stsci.edu> Our efforts to repackage numarray as a package rather than a collection of modules have taken form, so I'm looking for a (final) final round of comments before we declare this to be the "new interface". ================= basic repackaging ================= The core numarray routines will be distributed as the package "numarray" and because the new package __init__.py imports the contents of the old numarray.py module, simple import phrases involving numarray will continue to work like they always did: import numarray # These will continue to work unchanged from numarray import * For everything else, a "numarray." prefix and possibly a new name are required: recarray --> numarray.records chararray --> numarray.strings ndarray --> numarray.generic numtestall --> numarray.testall * --> numarray.* The 4 standard add-on packages have been renamed and will be distributed as a single unit called "numarray-addons" and will install directly in the numarray package. This has most effect for Windows binary users who should now be able to install with two clicks: one for numarray, the other for the addons. UNIX users still have the setupall one liner or the option to use the two independent setups however they like. FFT2 --> numarray.fft LinearAlgebra2 --> numarray.linear_algebra Convolve --> numarray.convolve Convolve.Image --> numarray.image ======= numcomp ======= There is a new "compatibility distribution" called "numcomp" which installs backward compatibility modules in a directory separate from numarray. Install numcomp if you want to run numarray-0.6 without changing your existing numarray applications for the new naming scheme (and if you directly import anything other than numarray itself, e.g. recarray or FFT2). We need numcomp at STSCI as a pragmatic issue, to prevent needing simultaneous changes of numarray and software depending on it. numcomp issues a single deprecation warning when you import your first compatibility module. numcomp is a sort of veneer, not a complete distribution; you still need the numarray-0.6 package in order to use numcomp. The plan is to phase numcomp out of existence at the release of numarray-1.0. The warning can be eliminated either by a simple hack of numcomp's repackage.py module, or from the Python command line. Thus, without numcomp but with numarray-0.6: >>> import recarray # can't do this anymore ... ImportError >>> import numarray.records # have to do this now >>> but with numcomp: >>> import recarray # this still works, but with a warning ...NumarrayRepackagingWarning... # too big a mouthful for email ================ numarray.numeric ================ There is now a submodule of numarray, numarray.numeric, which is more Numeric compatbile than numarray itself. It is part of the main numarray distribution. numarray.numeric contains everything the (old) numarray module does, but also redefines things for better compatibility. Currently, it just redefines nonzero() to handle 1D arrays the same as Numeric does. numarray.nonzero() remains unchanged. Any future Numeric compatibility changes will be added to numarray.numeric as well. So... if you've already written numarray code using nonzero() and it works, you're done. If you want to port Numeric code to numarray, consider importing numarray.numeric instead of numarray. The current plan is to maintain numarray.numeric indefinitely. >>> import numarray >>> import numarray.numeric >>> a=numarray.arange(5) >>> numarray.nonzero(a) (array([1, 2, 3, 4]),) # this format handles multi-D arrays >>> numarray.numeric.nonzero(a) array([1, 2, 3, 4]) # this format doesn't Comments? Todd -- Todd Miller jmiller at stsci.edu STSCI / ESS / SSB From falted at openlc.org Fri Jun 13 09:34:01 2003 From: falted at openlc.org (Francesc Alted) Date: Fri Jun 13 09:34:01 2003 Subject: [Numpy-discussion] numarray repackaging... as it turned out In-Reply-To: <1055515666.30709.80.camel@halloween.stsci.edu> References: <1055515666.30709.80.camel@halloween.stsci.edu> Message-ID: <200306131832.45201.falted@openlc.org> Hi, The new interface looks pretty clear. I don't think numarray.numcomp veneer would be very elegant, because after all, it took to me only 30 minuts to migrate pytables from numarray 0.5 to the forthcoming numarray 0.6 (and it used the old recarray, charray and ndarray modules intensively), although I recognize that it may be useful in some circunstances, of course. On the contrary, I find the numarray.numeric module a *great* thing as it will easy a lot the migration of applications from Numeric to numarray. Cheers, -- Francesc Alted From jmiller at stsci.edu Fri Jun 13 10:34:17 2003 From: jmiller at stsci.edu (Todd Miller) Date: Fri Jun 13 10:34:17 2003 Subject: [Numpy-discussion] numarray repackaging... as it turned out In-Reply-To: <200306131832.45201.falted@openlc.org> References: <1055515666.30709.80.camel@halloween.stsci.edu> <200306131832.45201.falted@openlc.org> Message-ID: <1055525626.30709.104.camel@halloween.stsci.edu> On Fri, 2003-06-13 at 12:32, Francesc Alted wrote: > Hi, > > The new interface looks pretty clear. I don't think numarray.numcomp veneer > would be very elegant, because after all, it took to me only 30 minuts to > migrate pytables from numarray 0.5 to the forthcoming numarray 0.6 (and it > used the old recarray, charray and ndarray modules intensively), although I > recognize that it may be useful in some circunstances, of course. numcomp is a free-standing collection of compatibility modules. It is throw-away code to help mitigate the need to simultaneously change several pieces of software. It's not required or recommended, but is there as a tool to ease the transition from the old names to the new if the transition presents any sticky problems. Thanks looking it over, Todd From magnus at hetland.org Sat Jun 14 05:39:03 2003 From: magnus at hetland.org (Magnus Lie Hetland) Date: Sat Jun 14 05:39:03 2003 Subject: [Numpy-discussion] Missing _confmodule.c in .5 and Solaris compilation trouble Message-ID: <20030614123743.GA16773@idi.ntnu.no> When I try to build numarray 0.5, it seems to look for Src/_convmodule.c, which isn't there (although it is there in 0.4). I couldn't find anything about this in the bug tracker, which somehow surprised me... Am I missing something obvious? Another thing: I'm having trouble using setupall.py in Solaris with Python 2.3b1 (it worked with earlier versions, including some pre-2.3a1 CVS versions, IIRC). Things compile, but when the tests run, I get a segmentation fault. I can import numarray (and everything from it) but if I try something basic, such as calling the array function, I again get a segfault. Things *do* work, however, if I simply use python setup.py install, so I suppose there must be some problem with some extension...? I don't quite understand how that would affect the basic array function, for example... Is there an interaction effect there? I couldn't find anything about this in the bug tracker either, when searching for "build failure". Tips on how to find out more so I can give a more precise bug report would be appreciated. I really hope I can eventually use 0.6 (can't seem to be able to get anything with anonymous CVS :() with Py2.3 and Solaris... And -- I really think the numarray restructuring looks _great_! :) Thanks for any input, - Magnus -- Magnus Lie Hetland "In this house we obey the laws of http://hetland.org thermodynamics!" Homer Simpson From camartin at snet.net Tue Jun 24 16:06:01 2003 From: camartin at snet.net (Cliff Martin) Date: Tue Jun 24 16:06:01 2003 Subject: [Numpy-discussion] Use of fromfile Message-ID: <3EF8D980.2010101@snet.net> Dear Group, I've just started to use numarray as I have an imaging program I want to port. I noticed in the differences document that one can read in data using fromfile into an array. This is a great savings over using the standard , string to integer floats. For some strange reason I can't get it to work correctly with my interferometer file (512,512) array. So I made up a small set of data in a file.(Attached test.txt). The code I used was simple, inp = open('c:/transfer/test.txt','r') y=fromfile(inp,Int32,[4,4]) with the following error message(s) Traceback (most recent call last): File "", line 1, in ? File "C:\Python22\Lib\site-packages\numarray\numarray.py", line 380, in fromfile raise ValueError( ValueError: Not enough bytes left in file for specified shape and type Well I thought maybe Int32 is not the correct specification so I tried Int16 and Int8. These both worked but the numbers read in were wrong. So what am I not understanding here? Thanks Cliff Martin -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: test.txt URL: From herbalpills at att.net Wed Jun 25 01:56:42 2003 From: herbalpills at att.net (Herbal Pills Online) Date: Wed Jun 25 01:56:42 2003 Subject: [Numpy-discussion] Become A New Man - Herbal Pills Manhood Growth ww Message-ID: An HTML attachment was scrubbed... URL: From jmiller at stsci.edu Wed Jun 25 04:02:38 2003 From: jmiller at stsci.edu (Todd Miller) Date: Wed Jun 25 04:02:38 2003 Subject: [Numpy-discussion] Use of fromfile In-Reply-To: <3EF8D980.2010101@snet.net> References: <3EF8D980.2010101@snet.net> Message-ID: <1056538881.3464.47.camel@localhost.localdomain> On Tue, 2003-06-24 at 19:06, Cliff Martin wrote: > Dear Group, > > I've just started to use numarray as I have an imaging program I want to > port. I noticed in the differences document that one can read in data > using fromfile into an array. This is a great savings over using the > standard , string to integer floats. For some strange reason I can't > get it to work correctly with my interferometer file (512,512) array. So > I made up a small set of data in a file.(Attached test.txt). > Looking over your test data, it looks like it is in ASCII. fromfile() works with binary data. Reading in your data can be done with a few lines of Python code: >>> import numarray >>> n = numarray.array(shape=(16,), type=numarray.Int32) >>> f = open("test.txt") >>> s = f.read() # Read the whole file as one string >>> words = s.split(" ")[:-1] # split on spaces; discard trailing junk >>> for i in range(len(words)): n[i] = eval(words[i]) # convert ASCII to Python Ints and assign >>> n.shape=(4,4) # Add the "real" shape >>> n array([[ 0, -32678, 14, 85], [ 342, 12, 14, 15], [ 16, 45, 67, 98], [ 38, 256, 234, 234]]) Binary files are easier and more efficient, but are not portable unless you remember the byte order. Todd -- Todd Miller From eric at enthought.com Fri Jun 27 15:11:03 2003 From: eric at enthought.com (eric jones) Date: Fri Jun 27 15:11:03 2003 Subject: [Numpy-discussion] [ANN] SciPy '03 -- The 2nd Annual Python for Scientific Computing Workshop Message-ID: <015701c33cf8$96be3670$8901a8c0@ERICDESKTOP> Hey folks, I've been postponing this announcement because the registration page isn't active yet. It's getting late though, and I thought I'd at least let you know SciPy '03 is happening. I'll repost when registration is open. Thanks, Eric ------------------------------------------------------- SciPy '03 The 2nd Annual Python for Scientific Computing Workshop ------------------------------------------------------- CalTech, Pasadena, CA September 11-12, 2003 http://www.scipy.org/site_content/scipy03 This workshop provides a unique opportunity to learn and affect what is happening in the realm of scientific computing with Python. Attendees will have the opportunity to review the available tools and how they apply to specific problems. By providing a forum for developers to share their Python expertise with the wider industrial, academic, and research communities, this workshop will foster collaboration and facilitate the sharing of software components, techniques and a vision for high level language use in scientific computing. The cost of the workshop is $100.00 and includes 2 breakfasts and 2 lunches on Sept. 11th and 12th, one dinner on Sept. 11th, and snacks during breaks. Online registration is not available yet, but will be soon. We would like to have a wide variety of presenters this year. If you have a paper you would like to present, please contact eric at enthought.com. Discussion about the conference may be directed to the SciPy-user mailing list: Mailing list page: http://www.scipy.org/MailList Mailinbg list address: scipy-user at scipy.org Please forward this announcement to anyone/list that might be interested. ------------- Co-Hosted By: ------------- The National Biomedical Computation Resource (NBCR, SDSC, San Diego, CA) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ http://nbcr.sdsc.edu The mission of the National Biomedical Computation Resource at the San Diego Supercomputer Center is to conduct, catalyze, and enable biomedical research by harnessing advanced computational technology. The Center for Advanced Computing Research (CACR, CalTech, Pasadena, CA) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ http://nbcr.sdsc.edu CACR is dedicated to the pursuit of excellence in the field of high-performance computing, communication, and data engineering. Major activities include carrying out large-scale scientific and engineering applications on parallel supercomputers and coordinating collaborative research projects on high-speed network technologies, distributed computing and database methodologies, and related topics. Our goal is to help further the state of the art in scientific computing. Enthought, Inc. (Austin, TX) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ http://enthought.com Enthought, Inc. provides scientific and business computing solutions through software development, consulting and training. ---------------------------------------------- eric jones 515 Congress Ave www.enthought.com Suite 1614 512 536-1057 Austin, Tx 78701 From b.hall at irl.cri.nz Thu Jun 5 16:23:07 2003 From: b.hall at irl.cri.nz (Blair Hall) Date: Thu Jun 5 16:23:07 2003 Subject: [Numpy-discussion] Having difficulty with Matrix.py Message-ID: <5.2.0.9.1.20030606111256.01e6f248@pop.wgtn.irl.cri.nz> I have been caught out writing a simple function that manipulates the elements of a matrix. I have distilled the problem into the code below. I find that 'sum' works with a Numeric array but fails with a Matrix. Am I doing something wrong? If not, what should I do to patch it? (I am working under Windows with Numpy 23.0) Here is the code: # 'a' is a two dimensional array. The # function simply places the sum of # the first column in a[0,0] def sum(a): N = a.shape[0] sum = 0 for i in range(N): sum += a[i,0] # This is the criticial line a[0,0] = sum return a[0,0] #------------------------ if(__name__ == '__main__'): import Numeric import Matrix a = Numeric.ones( (9,1) ) print sum( a ) # Ok print sum( Matrix.Matrix( a ) ) # Fails From oliphant.travis at ieee.org Fri Jun 6 10:11:08 2003 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Fri Jun 6 10:11:08 2003 Subject: [Numpy-discussion] Matrix.py has been fixed in CVS References: <5.2.0.9.1.20030606111256.01e6f248@pop.wgtn.irl.cri.nz> Message-ID: <3EE0CAED.60100@ieee.org> This is a bug in Matrix.py which has been fixed. Here is the relevant section of code to replace the current __setitem__ command in the Matrix class with: def __setitem__(self, index, value): value = asarray(value, self._typecode) self.array[index] = squeeze(value) Good luck, Travis O. > > I find that 'sum' works with a Numeric > array but fails with a Matrix. Am I doing something > wrong? If not, what should I do to patch it? > From pete at shinners.org Sun Jun 8 11:41:03 2003 From: pete at shinners.org (Pete Shinners) Date: Sun Jun 8 11:41:03 2003 Subject: [Numpy-discussion] convert to numarray Message-ID: <3EE38292.4030205@shinners.org> in the pygame project i've had the ability to map images into Numeric data arrays onto image pixel data. This has worked excellently for me, but in the near future i'd also like to support numarray. early in numarray's development it looked like this was not going to be possible at all. i've been following numarray loosely, and it sure looks like things have 'loosened' up a bit. still there are a few things i am doing that i'm unsure if numarray is ready to handle yet? what is going to make this tricky is i'm doing a bit of 'attribute mangling' to the Numeric array structure. this is necessary as the image data is extremely 'non-flat'. also, since i am referencing data held in another python object, i need to make sure the array holds a reference to the original object. these are the things i'm afraid i'll be stuck on. here is pretty much what i am doing now, simplified quite a bit... PyObject* pixelsarray(SDL_Surface *surf) { int dim[3]; PyObject *array; dim[0] = surf->w; dim[1] = surf->h; dim[2] = 3; array = PyArray_FromDimsAndData(3, dim, PyArray_UBYTE, surf->pixels); if(array) { PyArrayObject *a = (PyArrayObject*)array a->flags = OWN_DIMENSIONS|OWN_STRIDES; a->strides[2] = 1; a->strides[1] = surf->pitch; a->strides[0] = surf->format->BytesPerPixel; a->base = _pyobject_to_surf_; } return array; } note that depending on pixel packing and endianess, the strides[2] can become -1. smiley. also pretend a->base is pointing to a real python object, which it does in the real version. there is likely a way to workaround the "base" requirement with weakrefs i suppose, but i'd rather not jump through the extra hoops. the real necessity is setting the strides how i want. i didn't see any array creation functions that allow me to pick my own strides. once i create the array here i never change any of the array attributes. if this looks doable then it's time for me to sit down with the numarray docs and see what new and exciting things await me :] From jmiller at stsci.edu Mon Jun 9 06:41:02 2003 From: jmiller at stsci.edu (Todd Miller) Date: Mon Jun 9 06:41:02 2003 Subject: [Numpy-discussion] convert to numarray In-Reply-To: <3EE38292.4030205@shinners.org> References: <3EE38292.4030205@shinners.org> Message-ID: <1055165976.9644.55.camel@halloween.stsci.edu> On Sun, 2003-06-08 at 14:38, Pete Shinners wrote: > still there are a few things i am > doing that i'm unsure if numarray is ready to handle yet? > > what is going to make this tricky is i'm doing a bit of 'attribute > mangling' to the Numeric array structure. this is necessary as the image > data is extremely 'non-flat'. also, since i am referencing data held in > another python object, i need to make sure the array holds a reference > to the original object. these are the things i'm afraid i'll be stuck on. > > here is pretty much what i am doing now, simplified quite a bit... > > PyObject* pixelsarray(SDL_Surface *surf) > { > int dim[3]; > PyObject *array; > > dim[0] = surf->w; > dim[1] = surf->h; > dim[2] = 3; > array = PyArray_FromDimsAndData(3, dim, > PyArray_UBYTE, surf->pixels); > if(array) > { > PyArrayObject *a = (PyArrayObject*)array > a->flags = OWN_DIMENSIONS|OWN_STRIDES; > a->strides[2] = 1; > a->strides[1] = surf->pitch; > a->strides[0] = surf->format->BytesPerPixel; > a->base = _pyobject_to_surf_; > } > return array; > } > > note that depending on pixel packing and endianess, the strides[2] can > become -1. smiley. Despite old documentation to the contrary, the strides array is now writable. The -1 is more problematic. Anytime a ufunc is called, there is "safety code" which evaluates the shape, strides, byteoffset, buffer base and buffer size to ensure that the ufunc won't attempt an access outside the buffer or generate a misaligned pointer. negative strides are possible *provided* that the byteoffset is chosen so that everything works out. > also pretend a->base is pointing to a real python > object, which it does in the real version. numarray has a "base" pointer which is currently unused and intended for the purpose of backward compatibility. Since numarray does not yet use the base pointer at all, it does not deallocate it at array destruction time. > there is likely a way to workaround the "base" requirement with weakrefs > i suppose, but i'd rather not jump through the extra hoops. the real > necessity is setting the strides how i want. i didn't see any array > creation functions that allow me to pick my own strides. New in numarray-0.5 is the API function NA_NewAllStrides which allows you to pass in a pointer to a strides array. NA_NewAllStrides does not currently check the validity of the strides array. > once i create > the array here i never change any of the array attributes. > > if this looks doable then it's time for me to sit down with the numarray > docs and see what new and exciting things await me :] It looks close to doable to me. There are probably still a few loose ends. Todd -- Todd Miller jmiller at stsci.edu STSCI / ESS / SSB From felix at fbreuer.de Wed Jun 11 14:32:06 2003 From: felix at fbreuer.de (Felix Breuer) Date: Wed Jun 11 14:32:06 2003 Subject: [Numpy-discussion] OverflowError when importing RandomArray Message-ID: <1055366499.25112.65.camel@tapir> Hello! I am using Numeric 0.23 with Python 2.2 on my Gentoo Linux box. Problem: >>> from Numeric import * >>> import RandomArray Traceback (most recent call last): File "", line 1, in ? File "/usr/lib/python2.2/site-packages/Numeric/RandomArray.py", line 30, in ? seed() File "/usr/lib/python2.2/site-packages/Numeric/RandomArray.py", line 26, in seed x = int(t/base) OverflowError: float too large to convert Is this a bug, a misconfiguration on my system or a misusage on my part? Opinions, anyone? Thanks, Felix From 237506 at mail.com Fri Jun 13 01:07:02 2003 From: 237506 at mail.com (237506 at mail.com) Date: Fri Jun 13 01:07:02 2003 Subject: [Numpy-discussion] 237506 E-mail recording 1117571091 Message-ID: 12159CDA-243323A0-58FA3720-30BD2546-63FB8503@hotmail.com An HTML attachment was scrubbed... URL: From jmiller at stsci.edu Fri Jun 13 07:48:13 2003 From: jmiller at stsci.edu (Todd Miller) Date: Fri Jun 13 07:48:13 2003 Subject: [Numpy-discussion] numarray repackaging... as it turned out Message-ID: <1055515666.30709.80.camel@halloween.stsci.edu> Our efforts to repackage numarray as a package rather than a collection of modules have taken form, so I'm looking for a (final) final round of comments before we declare this to be the "new interface". ================= basic repackaging ================= The core numarray routines will be distributed as the package "numarray" and because the new package __init__.py imports the contents of the old numarray.py module, simple import phrases involving numarray will continue to work like they always did: import numarray # These will continue to work unchanged from numarray import * For everything else, a "numarray." prefix and possibly a new name are required: recarray --> numarray.records chararray --> numarray.strings ndarray --> numarray.generic numtestall --> numarray.testall * --> numarray.* The 4 standard add-on packages have been renamed and will be distributed as a single unit called "numarray-addons" and will install directly in the numarray package. This has most effect for Windows binary users who should now be able to install with two clicks: one for numarray, the other for the addons. UNIX users still have the setupall one liner or the option to use the two independent setups however they like. FFT2 --> numarray.fft LinearAlgebra2 --> numarray.linear_algebra Convolve --> numarray.convolve Convolve.Image --> numarray.image ======= numcomp ======= There is a new "compatibility distribution" called "numcomp" which installs backward compatibility modules in a directory separate from numarray. Install numcomp if you want to run numarray-0.6 without changing your existing numarray applications for the new naming scheme (and if you directly import anything other than numarray itself, e.g. recarray or FFT2). We need numcomp at STSCI as a pragmatic issue, to prevent needing simultaneous changes of numarray and software depending on it. numcomp issues a single deprecation warning when you import your first compatibility module. numcomp is a sort of veneer, not a complete distribution; you still need the numarray-0.6 package in order to use numcomp. The plan is to phase numcomp out of existence at the release of numarray-1.0. The warning can be eliminated either by a simple hack of numcomp's repackage.py module, or from the Python command line. Thus, without numcomp but with numarray-0.6: >>> import recarray # can't do this anymore ... ImportError >>> import numarray.records # have to do this now >>> but with numcomp: >>> import recarray # this still works, but with a warning ...NumarrayRepackagingWarning... # too big a mouthful for email ================ numarray.numeric ================ There is now a submodule of numarray, numarray.numeric, which is more Numeric compatbile than numarray itself. It is part of the main numarray distribution. numarray.numeric contains everything the (old) numarray module does, but also redefines things for better compatibility. Currently, it just redefines nonzero() to handle 1D arrays the same as Numeric does. numarray.nonzero() remains unchanged. Any future Numeric compatibility changes will be added to numarray.numeric as well. So... if you've already written numarray code using nonzero() and it works, you're done. If you want to port Numeric code to numarray, consider importing numarray.numeric instead of numarray. The current plan is to maintain numarray.numeric indefinitely. >>> import numarray >>> import numarray.numeric >>> a=numarray.arange(5) >>> numarray.nonzero(a) (array([1, 2, 3, 4]),) # this format handles multi-D arrays >>> numarray.numeric.nonzero(a) array([1, 2, 3, 4]) # this format doesn't Comments? Todd -- Todd Miller jmiller at stsci.edu STSCI / ESS / SSB From falted at openlc.org Fri Jun 13 09:34:01 2003 From: falted at openlc.org (Francesc Alted) Date: Fri Jun 13 09:34:01 2003 Subject: [Numpy-discussion] numarray repackaging... as it turned out In-Reply-To: <1055515666.30709.80.camel@halloween.stsci.edu> References: <1055515666.30709.80.camel@halloween.stsci.edu> Message-ID: <200306131832.45201.falted@openlc.org> Hi, The new interface looks pretty clear. I don't think numarray.numcomp veneer would be very elegant, because after all, it took to me only 30 minuts to migrate pytables from numarray 0.5 to the forthcoming numarray 0.6 (and it used the old recarray, charray and ndarray modules intensively), although I recognize that it may be useful in some circunstances, of course. On the contrary, I find the numarray.numeric module a *great* thing as it will easy a lot the migration of applications from Numeric to numarray. Cheers, -- Francesc Alted From jmiller at stsci.edu Fri Jun 13 10:34:17 2003 From: jmiller at stsci.edu (Todd Miller) Date: Fri Jun 13 10:34:17 2003 Subject: [Numpy-discussion] numarray repackaging... as it turned out In-Reply-To: <200306131832.45201.falted@openlc.org> References: <1055515666.30709.80.camel@halloween.stsci.edu> <200306131832.45201.falted@openlc.org> Message-ID: <1055525626.30709.104.camel@halloween.stsci.edu> On Fri, 2003-06-13 at 12:32, Francesc Alted wrote: > Hi, > > The new interface looks pretty clear. I don't think numarray.numcomp veneer > would be very elegant, because after all, it took to me only 30 minuts to > migrate pytables from numarray 0.5 to the forthcoming numarray 0.6 (and it > used the old recarray, charray and ndarray modules intensively), although I > recognize that it may be useful in some circunstances, of course. numcomp is a free-standing collection of compatibility modules. It is throw-away code to help mitigate the need to simultaneously change several pieces of software. It's not required or recommended, but is there as a tool to ease the transition from the old names to the new if the transition presents any sticky problems. Thanks looking it over, Todd From magnus at hetland.org Sat Jun 14 05:39:03 2003 From: magnus at hetland.org (Magnus Lie Hetland) Date: Sat Jun 14 05:39:03 2003 Subject: [Numpy-discussion] Missing _confmodule.c in .5 and Solaris compilation trouble Message-ID: <20030614123743.GA16773@idi.ntnu.no> When I try to build numarray 0.5, it seems to look for Src/_convmodule.c, which isn't there (although it is there in 0.4). I couldn't find anything about this in the bug tracker, which somehow surprised me... Am I missing something obvious? Another thing: I'm having trouble using setupall.py in Solaris with Python 2.3b1 (it worked with earlier versions, including some pre-2.3a1 CVS versions, IIRC). Things compile, but when the tests run, I get a segmentation fault. I can import numarray (and everything from it) but if I try something basic, such as calling the array function, I again get a segfault. Things *do* work, however, if I simply use python setup.py install, so I suppose there must be some problem with some extension...? I don't quite understand how that would affect the basic array function, for example... Is there an interaction effect there? I couldn't find anything about this in the bug tracker either, when searching for "build failure". Tips on how to find out more so I can give a more precise bug report would be appreciated. I really hope I can eventually use 0.6 (can't seem to be able to get anything with anonymous CVS :() with Py2.3 and Solaris... And -- I really think the numarray restructuring looks _great_! :) Thanks for any input, - Magnus -- Magnus Lie Hetland "In this house we obey the laws of http://hetland.org thermodynamics!" Homer Simpson From camartin at snet.net Tue Jun 24 16:06:01 2003 From: camartin at snet.net (Cliff Martin) Date: Tue Jun 24 16:06:01 2003 Subject: [Numpy-discussion] Use of fromfile Message-ID: <3EF8D980.2010101@snet.net> Dear Group, I've just started to use numarray as I have an imaging program I want to port. I noticed in the differences document that one can read in data using fromfile into an array. This is a great savings over using the standard , string to integer floats. For some strange reason I can't get it to work correctly with my interferometer file (512,512) array. So I made up a small set of data in a file.(Attached test.txt). The code I used was simple, inp = open('c:/transfer/test.txt','r') y=fromfile(inp,Int32,[4,4]) with the following error message(s) Traceback (most recent call last): File "", line 1, in ? File "C:\Python22\Lib\site-packages\numarray\numarray.py", line 380, in fromfile raise ValueError( ValueError: Not enough bytes left in file for specified shape and type Well I thought maybe Int32 is not the correct specification so I tried Int16 and Int8. These both worked but the numbers read in were wrong. So what am I not understanding here? Thanks Cliff Martin -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: test.txt URL: From herbalpills at att.net Wed Jun 25 01:56:42 2003 From: herbalpills at att.net (Herbal Pills Online) Date: Wed Jun 25 01:56:42 2003 Subject: [Numpy-discussion] Become A New Man - Herbal Pills Manhood Growth ww Message-ID: An HTML attachment was scrubbed... URL: From jmiller at stsci.edu Wed Jun 25 04:02:38 2003 From: jmiller at stsci.edu (Todd Miller) Date: Wed Jun 25 04:02:38 2003 Subject: [Numpy-discussion] Use of fromfile In-Reply-To: <3EF8D980.2010101@snet.net> References: <3EF8D980.2010101@snet.net> Message-ID: <1056538881.3464.47.camel@localhost.localdomain> On Tue, 2003-06-24 at 19:06, Cliff Martin wrote: > Dear Group, > > I've just started to use numarray as I have an imaging program I want to > port. I noticed in the differences document that one can read in data > using fromfile into an array. This is a great savings over using the > standard , string to integer floats. For some strange reason I can't > get it to work correctly with my interferometer file (512,512) array. So > I made up a small set of data in a file.(Attached test.txt). > Looking over your test data, it looks like it is in ASCII. fromfile() works with binary data. Reading in your data can be done with a few lines of Python code: >>> import numarray >>> n = numarray.array(shape=(16,), type=numarray.Int32) >>> f = open("test.txt") >>> s = f.read() # Read the whole file as one string >>> words = s.split(" ")[:-1] # split on spaces; discard trailing junk >>> for i in range(len(words)): n[i] = eval(words[i]) # convert ASCII to Python Ints and assign >>> n.shape=(4,4) # Add the "real" shape >>> n array([[ 0, -32678, 14, 85], [ 342, 12, 14, 15], [ 16, 45, 67, 98], [ 38, 256, 234, 234]]) Binary files are easier and more efficient, but are not portable unless you remember the byte order. Todd -- Todd Miller From eric at enthought.com Fri Jun 27 15:11:03 2003 From: eric at enthought.com (eric jones) Date: Fri Jun 27 15:11:03 2003 Subject: [Numpy-discussion] [ANN] SciPy '03 -- The 2nd Annual Python for Scientific Computing Workshop Message-ID: <015701c33cf8$96be3670$8901a8c0@ERICDESKTOP> Hey folks, I've been postponing this announcement because the registration page isn't active yet. It's getting late though, and I thought I'd at least let you know SciPy '03 is happening. I'll repost when registration is open. Thanks, Eric ------------------------------------------------------- SciPy '03 The 2nd Annual Python for Scientific Computing Workshop ------------------------------------------------------- CalTech, Pasadena, CA September 11-12, 2003 http://www.scipy.org/site_content/scipy03 This workshop provides a unique opportunity to learn and affect what is happening in the realm of scientific computing with Python. Attendees will have the opportunity to review the available tools and how they apply to specific problems. By providing a forum for developers to share their Python expertise with the wider industrial, academic, and research communities, this workshop will foster collaboration and facilitate the sharing of software components, techniques and a vision for high level language use in scientific computing. The cost of the workshop is $100.00 and includes 2 breakfasts and 2 lunches on Sept. 11th and 12th, one dinner on Sept. 11th, and snacks during breaks. Online registration is not available yet, but will be soon. We would like to have a wide variety of presenters this year. If you have a paper you would like to present, please contact eric at enthought.com. Discussion about the conference may be directed to the SciPy-user mailing list: Mailing list page: http://www.scipy.org/MailList Mailinbg list address: scipy-user at scipy.org Please forward this announcement to anyone/list that might be interested. ------------- Co-Hosted By: ------------- The National Biomedical Computation Resource (NBCR, SDSC, San Diego, CA) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ http://nbcr.sdsc.edu The mission of the National Biomedical Computation Resource at the San Diego Supercomputer Center is to conduct, catalyze, and enable biomedical research by harnessing advanced computational technology. The Center for Advanced Computing Research (CACR, CalTech, Pasadena, CA) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ http://nbcr.sdsc.edu CACR is dedicated to the pursuit of excellence in the field of high-performance computing, communication, and data engineering. Major activities include carrying out large-scale scientific and engineering applications on parallel supercomputers and coordinating collaborative research projects on high-speed network technologies, distributed computing and database methodologies, and related topics. Our goal is to help further the state of the art in scientific computing. Enthought, Inc. (Austin, TX) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ http://enthought.com Enthought, Inc. provides scientific and business computing solutions through software development, consulting and training. ---------------------------------------------- eric jones 515 Congress Ave www.enthought.com Suite 1614 512 536-1057 Austin, Tx 78701