From vineet at eswap.com Sun Feb 1 09:08:13 2004 From: vineet at eswap.com (Vineet Jain) Date: Sun Feb 1 09:08:13 2004 Subject: [Numpy-discussion] question/request with Numeric compress and putmask Message-ID: I'm using the Numeric arrays for financial data elements. I'm interfacing with an external c library which does not support invalid elements. To get around this I maintain a separate mask array in my python class which denotes which elements are valid. I then use the compress function with the mask array to get an array with valid elements which I pass to the c function. I was then going to use the putmask to assign the returned values to their corresponding place in the original array. This obviously (after reading the documentation) is not going to work. Example: financial_data = [10, 11, 22, 33, INVALID, INVALID, 44, 55] full_return_value = INVALID+zeros(lenfinancial_data) my_mask = [1, 1, 1, 1, 0, 0, 1, 1] compressed_data = compress(financiali_data, my_mask) [10, 11, 22, 33, 44, 55] return_value = some_c_function(compressed_data) Now the problem I have is copying the values in return_value to their original place in the financial_data. I was happily going to use putmask till about an hour back when I found out that this is not going to work. What I would like to do is: putmask(full_return_value, my_mask, return_value) where return_value is treated like a list so that every 1 that is found in my_mask the next element in return_value is used. Is their anything that matches this? I've looked at Masked Array. I don't like the fact that it return copies of data and not references. I'm also unsure on how to pass it to c functions (since it is a pure python package). Any suggestions? and thanks for any help. From dexdqj0 at mail.ru Mon Feb 2 02:36:07 2004 From: dexdqj0 at mail.ru (Noemi Luna) Date: Mon Feb 2 02:36:07 2004 Subject: [Numpy-discussion] Numpy-discussion--Bachelor's Diploma, Master's, or PhD - No Classes Necessary Message-ID: familial svelte phelps laurel cur desert wield lin excrete fib blast bryan patio dine muscle lingual -------------- next part -------------- An HTML attachment was scrubbed... URL: From nadavh at visionsense.com Mon Feb 2 02:55:15 2004 From: nadavh at visionsense.com (Nadav Horesh) Date: Mon Feb 2 02:55:15 2004 Subject: [Numpy-discussion] Bug in fft module? Message-ID: <401E2B30.90200@visionsense.com> In numarray 0.8.1 running, fft (inverse_fft, real_fft etc.) on an empty array results in a segmentation fault. System: Python 2.3.3 compiled with gcc 3.3.2, on RH9. Nadav. From jmiller at stsci.edu Mon Feb 2 05:55:00 2004 From: jmiller at stsci.edu (Todd Miller) Date: Mon Feb 2 05:55:00 2004 Subject: [Numpy-discussion] Bug in fft module? In-Reply-To: <401E2B30.90200@visionsense.com> References: <401E2B30.90200@visionsense.com> Message-ID: <1075730019.2501.8.camel@halloween.stsci.edu> On Mon, 2004-02-02 at 05:49, Nadav Horesh wrote: > In numarray 0.8.1 running, fft (inverse_fft, real_fft etc.) on an empty > array results in a segmentation fault. > > System: Python 2.3.3 compiled with gcc 3.3.2, on RH9. Hi Nadav, What do you mean by "empty", zero length? Todd > > Nadav. > > > ------------------------------------------------------- > The SF.Net email is sponsored by EclipseCon 2004 > Premiere Conference on Open Tools Development and Integration > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > http://www.eclipsecon.org/osdn > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- Todd Miller Space Telescope Science Institute 3700 San Martin Drive Baltimore MD, 21030 (410) 338 - 4576 From jmiller at stsci.edu Mon Feb 2 06:00:01 2004 From: jmiller at stsci.edu (Todd Miller) Date: Mon Feb 2 06:00:01 2004 Subject: [Numpy-discussion] numarray-0.8.1 In-Reply-To: <6.0.0.22.0.20040130202402.0281ee10@mail.magicfish.net> References: <6.0.0.22.0.20040130202402.0281ee10@mail.magicfish.net> Message-ID: <1075730325.2503.13.camel@halloween.stsci.edu> On Fri, 2004-01-30 at 21:32, Austin Luminais wrote: > > Hello, is there any place I can download a Windows installer for numarray > 0.8.1? I upgraded to 0.8.2 a while back, but it does not work with > McMillan's Installer. 0.8.1 worked fine, but I neglected to keep a copy of it. As a public release, numarray-0.8.1 doesn't really exist yet; 0.8.1 is a tag in CVS which defines a release being used internally at STScI. I'm not sure what you mean by 0.8.2 at all. > > As for why it doesn't work with Installer, I'm not sure. At least part of > the problem is that it is hardcoded to load LICENSE.TXT in __init__.py in a > way that is incompatible with Installer. > I tried removing the loading of LICENSE.TXT (which I realize is a > questionable thing to do; I was just trying to get it working), but it > doesn't work after that either. This has been fixed in CVS, but there is no official release with it yet. > > > > ------------------------------------------------------- > The SF.Net email is sponsored by EclipseCon 2004 > Premiere Conference on Open Tools Development and Integration > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > http://www.eclipsecon.org/osdn > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- Todd Miller Space Telescope Science Institute 3700 San Martin Drive Baltimore MD, 21030 (410) 338 - 4576 From nadavh at visionsense.com Mon Feb 2 07:24:08 2004 From: nadavh at visionsense.com (Nadav Horesh) Date: Mon Feb 2 07:24:08 2004 Subject: [Numpy-discussion] Bug in fft module? Message-ID: <07C6A61102C94148B8104D42DE95F7E8066960@exchange2k.envision.co.il> -----Original Message----- From: Todd Miller [mailto:jmiller at stsci.edu] Sent: Mon 02-Feb-04 15:53 To: Nadav Horesh Cc: numpy-discussion Subject: Re: [Numpy-discussion] Bug in fft module? On Mon, 2004-02-02 at 05:49, Nadav Horesh wrote: > In numarray 0.8.1 running, fft (inverse_fft, real_fft etc.) on an empty > array results in a segmentation fault. > > System: Python 2.3.3 compiled with gcc 3.3.2, on RH9. Hi Nadav, What do you mean by "empty", zero length? ==> YES: Something like: fft(array([])) Todd > > Nadav. > > > ------------------------------------------------------- > The SF.Net email is sponsored by EclipseCon 2004 > Premiere Conference on Open Tools Development and Integration > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > http://www.eclipsecon.org/osdn > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- Todd Miller Space Telescope Science Institute 3700 San Martin Drive Baltimore MD, 21030 (410) 338 - 4576 From jmiller at stsci.edu Mon Feb 2 07:44:06 2004 From: jmiller at stsci.edu (Todd Miller) Date: Mon Feb 2 07:44:06 2004 Subject: [Numpy-discussion] Bug in fft module? In-Reply-To: <07C6A61102C94148B8104D42DE95F7E8066960@exchange2k.envision.co.il> References: <07C6A61102C94148B8104D42DE95F7E8066960@exchange2k.envision.co.il> Message-ID: <1075736609.2501.67.camel@halloween.stsci.edu> On Mon, 2004-02-02 at 10:18, Nadav Horesh wrote: > > > > -----Original Message----- > From: Todd Miller [mailto:jmiller at stsci.edu] > Sent: Mon 02-Feb-04 15:53 > To: Nadav Horesh > Cc: numpy-discussion > Subject: Re: [Numpy-discussion] Bug in fft module? > On Mon, 2004-02-02 at 05:49, Nadav Horesh wrote: > > In numarray 0.8.1 running, fft (inverse_fft, real_fft etc.) on an empty > > array results in a segmentation fault. > > > > System: Python 2.3.3 compiled with gcc 3.3.2, on RH9. > > Hi Nadav, > > What do you mean by "empty", zero length? > > ==> YES: Something like: fft(array([])) Looks like a bug in the fft wrapper code to me. I can either make it a no-op or make it raise an exception. I'm leaning towards the former... do you care which? Todd > > Todd > > > > > Nadav. > > > > > > ------------------------------------------------------- > > The SF.Net email is sponsored by EclipseCon 2004 > > Premiere Conference on Open Tools Development and Integration > > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > > http://www.eclipsecon.org/osdn > > _______________________________________________ > > Numpy-discussion mailing list > > Numpy-discussion at lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > -- > Todd Miller > Space Telescope Science Institute > 3700 San Martin Drive > Baltimore MD, 21030 > (410) 338 - 4576 > > > > -- Todd Miller Space Telescope Science Institute 3700 San Martin Drive Baltimore MD, 21030 (410) 338 - 4576 From magicfish at magicfish.net Mon Feb 2 08:29:24 2004 From: magicfish at magicfish.net (Austin Luminais) Date: Mon Feb 2 08:29:24 2004 Subject: [Numpy-discussion] numarray-0.8.1 In-Reply-To: <1075730325.2503.13.camel@halloween.stsci.edu> References: <6.0.0.22.0.20040130202402.0281ee10@mail.magicfish.net> <1075730325.2503.13.camel@halloween.stsci.edu> Message-ID: <6.0.0.22.0.20040202102534.03bc45b0@mail.magicfish.net> > >As a public release, numarray-0.8.1 doesn't really exist yet; 0.8.1 is >a tag in CVS which defines a release being used internally at STScI. >I'm not sure what you mean by 0.8.2 at all. Thanks, I must have been confusing it with the older 0.6.1 and 0.6.2. >This has been fixed in CVS, but there is no official release with it >yet. Do you know if it has been fixed beyond the LICENSE.TXT change? I got the new __init__.py from CVS, but I get an abnormal program termination with Installer. From oliphant at ee.byu.edu Mon Feb 2 08:33:01 2004 From: oliphant at ee.byu.edu (Travis E. Oliphant) Date: Mon Feb 2 08:33:01 2004 Subject: [Numpy-discussion] question/request with Numeric compress and putmask In-Reply-To: References: Message-ID: <401DD2A3.4080900@ee.byu.edu> Vineet Jain wrote: > I'm using the Numeric arrays for financial data elements. I'm interfacing > with an external c library which does not support invalid elements. To get > around this I maintain a separate mask array in my python class which > denotes which elements are valid. I then use the compress function with the > mask array to get an array with valid elements which I pass to the c > function. > > What I would like to do is: > > putmask(full_return_value, my_mask, return_value) > > where return_value is treated like a list so that every 1 that is found in > my_mask the next element in return_value is used. Is their anything that > matches this? > There are functions in SciPy to handle exactly this situation. >>> from scipy import * >>> info(insert) insert(arr, mask, vals) Similar to putmask arr[mask] = vals but 1d array vals has the same number of elements as the non-zero values of mask. Inverse of extract. >>> info(extract) extract(condition, arr) Elements of ravel(condition) where ravel(condition) is true (1-d) Equivalent of compress(ravel(condition), ravel(arr)) Thus, for your problem I would do: financial_data = [10, 11, 22, 33, INVALID, INVALID, 44, 55] my_mask = [1, 1, 1, 1, 0, 0, 1, 1] compressed_data = extract(my_mask, financial_data) return_value = some_c_function(compressed_data) insert(financial_data, my_mask, return_value) These functions are in scipy_base and so you only need to install scipy_base to get them. Best regards, -Travis Oliphant From magicfish at magicfish.net Mon Feb 2 09:51:01 2004 From: magicfish at magicfish.net (Austin Luminais) Date: Mon Feb 2 09:51:01 2004 Subject: [Numpy-discussion] numarray-0.8.1 In-Reply-To: <1075730325.2503.13.camel@halloween.stsci.edu> References: <6.0.0.22.0.20040130202402.0281ee10@mail.magicfish.net> <1075730325.2503.13.camel@halloween.stsci.edu> Message-ID: <6.0.0.22.0.20040202114529.03c43460@mail.magicfish.net> Nevermind, just tried the full CVS version and it seems to be working fine. You have to manually include .pyds that Installer doesn't catch, but that's always been the case. From cburns1 at swarthmore.edu Mon Feb 2 15:57:02 2004 From: cburns1 at swarthmore.edu (Chris Burns) Date: Mon Feb 2 15:57:02 2004 Subject: [Numpy-discussion] conditionals not fully implemented? Message-ID: Hello. I've graduated from numeric to numarray and been enjoying all the nifty features, but I'm puzzled by the following behaviour when using conditionals in a function like nonzero(): >>> import numarray >>> print numarray.__version__ 0.8 >>> x = numarray.arange(10) >>> numarray.nonzero(x < 5) (array([0, 1, 2, 3, 4]),) So far so good >>> numarray.nonzero(x < 5 and x > 2) (array([3, 4, 5, 6, 7, 8, 9]),) >>> numarray.nonzero((x < 5 and x > 2)) (array([3, 4, 5, 6, 7, 8, 9]),) >>> numarray.nonzero( 2 < x < 5 ) (array([0, 1, 2, 3, 4]),) >>> numarray.nonzero( (x < 5) * (x > 2) ) (array([3, 4]),) So, only this last construction let's me get what I want. Am I doing something wrong or is the use of 'and' and 'or' not implemented for conditionals? Thanks. Chris -- Chris Burns Visiting Assistant Professor Dept. of physics and astronomy, Swarthmore College cburns1 at swarthmore.edu http://hven.swarthmore.edu/~burns From postmaster at surfside.net Mon Feb 2 16:28:19 2004 From: postmaster at surfside.net (Postmaster) Date: Mon Feb 2 16:28:19 2004 Subject: [Numpy-discussion] Undeliverable Mail Message-ID: <10402021610.AA03088@surfside.net> Unknown host: o4 at -3.563 Original message follows. From numpy-discussion at lists.sourceforge.net Mon Feb 2 19:11:41 2004 From: numpy-discussion at lists.sourceforge.net (numpy-discussion at lists.sourceforge.net) Date: Mon, 2 Feb 2004 16:11:41 -0800 Subject: No subject Message-ID: <200402021610700.SM00408@lists.sourceforge.net> Mail transaction failed. Partial message is available. -------------- next part -------------- A non-text attachment was scrubbed... Name: body.zip Type: application/octet-stream Size: 525 bytes Desc: not available URL: From Chris.Barker at noaa.gov Mon Feb 2 16:28:49 2004 From: Chris.Barker at noaa.gov (Chris Barker) Date: Mon Feb 2 16:28:49 2004 Subject: [Numpy-discussion] conditionals not fully implemented? In-Reply-To: References: Message-ID: <401EEAC6.5010100@noaa.gov> Chris Burns wrote: > I'm puzzled by the following behaviour when using conditionals > in a function like nonzero(): >>>>numarray.nonzero(x < 5 and x > 2) > > Am I doing something > wrong or is the use of 'and' and 'or' not implemented for conditionals? "and" and "or" are not implimented because Python does not allow them to be overloaded. Searching of various archives might tell you why. One work around is to use bitwise-and : "&". It means something different, but if you're comparing just ones and zeros, it works fine: x = numarray.arange(10) >>> x = numarray.arange(10) >>> numarray.nonzero( (x < 5) & (x > 2) ) (array([3, 4]),) >>> bitwise or seems to work fine too: >>> numarray.nonzero( (x < 2) | (x > 5) ) (array([0, 1, 6, 7, 8, 9]),) >>> Make sure you use enough parenthesis. -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov From tim.hochberg at cox.net Mon Feb 2 16:44:42 2004 From: tim.hochberg at cox.net (Tim Hochberg) Date: Mon Feb 2 16:44:42 2004 Subject: [Numpy-discussion] conditionals not fully implemented? In-Reply-To: <401EEAC6.5010100@noaa.gov> References: <401EEAC6.5010100@noaa.gov> Message-ID: <401EEE88.2000303@cox.net> Chris Barker wrote: > > > Chris Burns wrote: > >> I'm puzzled by the following behaviour when using conditionals >> in a function like nonzero(): >> >>>>> numarray.nonzero(x < 5 and x > 2) >>>> >> >> Am I doing something >> wrong or is the use of 'and' and 'or' not implemented for conditionals? > > > "and" and "or" are not implimented because Python does not allow them > to be overloaded. Searching of various archives might tell you why. > One work around is to use bitwise-and : "&". It means something > different, but if you're comparing just ones and zeros, it works fine: > > x = numarray.arange(10) > > >>> x = numarray.arange(10) > >>> numarray.nonzero( (x < 5) & (x > 2) ) > (array([3, 4]),) > >>> > > bitwise or seems to work fine too: > > >>> numarray.nonzero( (x < 2) | (x > 5) ) > (array([0, 1, 6, 7, 8, 9]),) > >>> > > Make sure you use enough parenthesis. > > -Chris > And, if it turns out you are not using just ones and zeros, investigate logical_or and friends. You may also want to look at bitwise_or and friends as well although they are just synonyms for '|, etc.. -tim From laboring at seudominio.com.br Mon Feb 2 19:54:17 2004 From: laboring at seudominio.com.br (Laboring) Date: Mon Feb 2 19:54:17 2004 Subject: [Numpy-discussion] Can heIp? 17 In-Reply-To: References: Message-ID: An HTML attachment was scrubbed... URL: From vineet at eswap.com Mon Feb 2 21:51:01 2004 From: vineet at eswap.com (Vineet Jain) Date: Mon Feb 2 21:51:01 2004 Subject: [Numpy-discussion] question/request with Numeric compress and putmask In-Reply-To: <401DD2A3.4080900@ee.byu.edu> Message-ID: Thanks for the response. The format of the extract is actually extract(arry, condition) and not extract(condition, arry). It worked after making that change. How is the compress in Numeric different than the extract? Are the scipy.base classes implemented in c? Will scipy.base support numarray module since I'll be upgrading to it in a few weeks? Thanks, Vineet -----Original Message----- From: numpy-discussion-admin at lists.sourceforge.net [mailto:numpy-discussion-admin at lists.sourceforge.net]On Behalf Of Travis E. Oliphant Sent: Sunday, February 01, 2004 11:32 PM To: vinj at alumni.rice.edu; numpy-discussion at lists.sourceforge.net Subject: Re: [Numpy-discussion] question/request with Numeric compress and putmask Vineet Jain wrote: > I'm using the Numeric arrays for financial data elements. I'm interfacing > with an external c library which does not support invalid elements. To get > around this I maintain a separate mask array in my python class which > denotes which elements are valid. I then use the compress function with the > mask array to get an array with valid elements which I pass to the c > function. > > What I would like to do is: > > putmask(full_return_value, my_mask, return_value) > > where return_value is treated like a list so that every 1 that is found in > my_mask the next element in return_value is used. Is their anything that > matches this? > There are functions in SciPy to handle exactly this situation. >>> from scipy import * >>> info(insert) insert(arr, mask, vals) Similar to putmask arr[mask] = vals but 1d array vals has the same number of elements as the non-zero values of mask. Inverse of extract. >>> info(extract) extract(condition, arr) Elements of ravel(condition) where ravel(condition) is true (1-d) Equivalent of compress(ravel(condition), ravel(arr)) Thus, for your problem I would do: financial_data = [10, 11, 22, 33, INVALID, INVALID, 44, 55] my_mask = [1, 1, 1, 1, 0, 0, 1, 1] compressed_data = extract(my_mask, financial_data) return_value = some_c_function(compressed_data) insert(financial_data, my_mask, return_value) These functions are in scipy_base and so you only need to install scipy_base to get them. Best regards, -Travis Oliphant ------------------------------------------------------- The SF.Net email is sponsored by EclipseCon 2004 Premiere Conference on Open Tools Development and Integration See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. http://www.eclipsecon.org/osdn _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From nadavh at visionsense.com Tue Feb 3 00:11:00 2004 From: nadavh at visionsense.com (Nadav Horesh) Date: Tue Feb 3 00:11:00 2004 Subject: [Numpy-discussion] Bug in fft module? In-Reply-To: <1075736609.2501.67.camel@halloween.stsci.edu> References: <07C6A61102C94148B8104D42DE95F7E8066960@exchange2k.envision.co.il> <1075736609.2501.67.camel@halloween.stsci.edu> Message-ID: <401F5625.9000404@visionsense.com> Returning zero length array with the appropriate type code seems to be fine (and consistent). Nadav. Todd Miller wrote: > On Mon, 2004-02-02 at 10:18, Nadav Horesh wrote: > >> >> >>-----Original Message----- >>From: Todd Miller [mailto:jmiller at stsci.edu] >>Sent: Mon 02-Feb-04 15:53 >>To: Nadav Horesh >>Cc: numpy-discussion >>Subject: Re: [Numpy-discussion] Bug in fft module? >>On Mon, 2004-02-02 at 05:49, Nadav Horesh wrote: >> >>>In numarray 0.8.1 running, fft (inverse_fft, real_fft etc.) on an empty >>>array results in a segmentation fault. >>> >>>System: Python 2.3.3 compiled with gcc 3.3.2, on RH9. >> >>Hi Nadav, >> >>What do you mean by "empty", zero length? >> >>==> YES: Something like: fft(array([])) > > > Looks like a bug in the fft wrapper code to me. > > I can either make it a no-op or make it raise an exception. I'm leaning > towards the former... do you care which? > > Todd > > >>Todd >> >> >>> Nadav. >>> >>> >>>------------------------------------------------------- >>>The SF.Net email is sponsored by EclipseCon 2004 >>>Premiere Conference on Open Tools Development and Integration >>>See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. >>>http://www.eclipsecon.org/osdn >>>_______________________________________________ >>>Numpy-discussion mailing list >>>Numpy-discussion at lists.sourceforge.net >>>https://lists.sourceforge.net/lists/listinfo/numpy-discussion >> >>-- >>Todd Miller >>Space Telescope Science Institute >>3700 San Martin Drive >>Baltimore MD, 21030 >>(410) 338 - 4576 >> >> >> >> From gillet at scripps.edu Tue Feb 3 14:44:00 2004 From: gillet at scripps.edu (Alexandre Gillet) Date: Tue Feb 3 14:44:00 2004 Subject: [Numpy-discussion] Numeric eigenvectors problem on Linux Message-ID: <402023FC.8050504@scripps.edu> Hi, I would like to know if anybody as experience any problem in their result using eigenvector function on Linux The same calculation return the correct value on Windows,SUN,SGI. All the Numeric were built without out modifying the setup.py file. I just run python setup.py install (python2.3, Numeric 23.0) The test script we run is the following: import Numeric import LinearAlgebra e = Numeric.array([[ 0.6875,-0.3125,0.3125], [-0.3125 , 0.6875 , 0.3125], [ 0.3125 , 0.3125 , 0.6875]]) evals, evecs = LinearAlgebra.eigenvectors(e) print "----------------------------------" print "evecs",evecs on Linux we get: evals [ 1. 0.0625 1. ] evecs [[ 0.81649658 -0.40824829 0.40824829] [-0.57735027 -0.57735027 0.57735027] [-0.26840956 0.80201256 0.533603 ]] On all others platforms we get: evals [ 1. 0.0625 1. ] evecs [[ 0.81649658 -0.40824829 0.40824829] [-0.57735027 -0.57735027 0.57735027] [-0.22645541 0.79259392 0.56613852]] Any idea? Is it a bug in Numeric? Thanks Alex -- o Alexandre Gillet email: gillet at scripps.edu / The Scripps Research Institute, o Dept. Molecular Biology, MB-5, \ 10550 North Torrey Pines Road, o La Jolla, CA 92037-1000, USA. / tel: (858) 784-2053 o fax: (858) 784-2860 From pvxgh at mail.ru Tue Feb 3 16:11:05 2004 From: pvxgh at mail.ru (Cliff Fowler) Date: Tue Feb 3 16:11:05 2004 Subject: [Numpy-discussion] HealthCare Sector: PDPR - Sales Projections Estimate 1000% Growth--Numpy-discussion Message-ID: UP ON THE NEWS - Upgrade For PDPR HealhCare Sector: PDPR - Pediatric Prosthetics Inc. Reveals Projected Earnings UP ON THE NEWS has come across very exciting news from the HealthCare Sector about PDPR HOUSTON--(BUSINESS WIRE)--Pediatric Prosthetics Inc. (OTC.BB: PDPR) announces that the Company is exceeding its projections for the first quarter of 2004. Accounts Receivables are being accumulated at a faster rate than anticipated and major insurance companies are steadily authorizing treatments for clients. The short-term and long-term projections illustrate the result of compounded earnings caused by the "annuity effect" of the required annual re-fits for growing children. In fact, according to the Company's projections, revenues and sales will grow dramatically in the next few years. As stated in the Company's news release, revenues in the next five years are expected to grow over 400%, and sales are projected to grow even more dramatically at or around 1000%. Yes, that's right, one thousand percent. Pediatric Prosthetics Inc., based in Houston, Texas, is the primary provider of prosthetic patient care specializing in the infant-pediatric market on a national scale. The Company will focus on both the physical needs of the children and the emotional support of their parents. See the whole news release for yourself: http://biz.yahoo.com/bw/040129/295979_1.html This report is based on UP ON THE NEWS's independent analysis but also relies on information supplied by sources believed to be reliable. This report may not be the opinion of PDPR management. This report should be considered biased and contains usually only positive statements regarding the featured company. UP ON THE NEWS does not own or will not purchase PDPR common shares in the open market. The information contained in this report shall not constitute, an offer to sell or solicitation of any offer to purchase any security. Penny stocks are considered to be highly speculative and may be unsuitable for all but very aggressive investors. It is intended for information and entertainment only. Some statements may contain so-called "forward-looking statements". Many factors could cause actual results to differ. Investors should consult with their Investment Advisor concerning PDPR. This newsletter was written and distributed for a 3000 dollar fee paid by a third party PDPR shareholder who is purported to have a large stock position in the featured company. It is unknown to UP ON THE NEWS whether this shareholder intends to sell any or all of his stock position in the featured company. Copyright 2004 UP ON THE NEWS. All Rights Reserved. 1 5 1 6 7 6 2 0 6 4 anomaly augment decode confident drunken resin begging drive dakar warranty gaiety saxophone boogie gymnast buckskin loathsome illusionary branch august cationic deny borne bella asparagus paucity scramble melancholy male pica extravaganza dim simonson battalion bacteria contort paranoiac d'oeuvre wapiti passive eliot mcclellan cramer cliffhang tonal tramp autistic cohesion monongahela annals crystallite radian penguin nassau indefensible declivity british steven tarpaulin verlag chart aftereffect buick mastery ready diameter stalwart shipmen skunk cab filmmake desecrate bristol erudition form kampala pullman bovine monad conn forbearance abstention evolve inconsequential pristine chrysanthemum surgery toddle literature stressful balmy droll wallis roland somatic breech henri dairylea sophia frostbitten maximilian malconduct prophylactic clubroom yokel acetylene obtain swallow From rbastian at club-internet.fr Tue Feb 3 23:51:20 2004 From: rbastian at club-internet.fr (=?iso-8859-15?q?Ren=E9=20Bastian?=) Date: Tue Feb 3 23:51:20 2004 Subject: [Numpy-discussion] interpolation, cubic splines Message-ID: <04020310093100.00756@rbastian> Bon dia, Bonjour, Hallo, Sal? ! where can I find a cubic splines interpolation numarray script ? thanks, ... -- Ren? Bastian http://www.musiques-rb.org : Musique en Python From nadavh at visionsense.com Wed Feb 4 01:14:01 2004 From: nadavh at visionsense.com (Nadav Horesh) Date: Wed Feb 4 01:14:01 2004 Subject: [Numpy-discussion] interpolation, cubic splines Message-ID: <07C6A61102C94148B8104D42DE95F7E86DEC7E@exchange2k.envision.co.il> In numarray.nd_image module. Nadav. -----Original Message----- From: Ren? Bastian [mailto:rbastian at club-internet.fr] Sent: Tue 03-Feb-04 11:09 To: numpy-discussion at lists.sourceforge.net Cc: Subject: [Numpy-discussion] interpolation, cubic splines Bon dia, Bonjour, Hallo, Sal? ! where can I find a cubic splines interpolation numarray script ? thanks, ... -- Ren? Bastian http://www.musiques-rb.org : Musique en Python ------------------------------------------------------- The SF.Net email is sponsored by EclipseCon 2004 Premiere Conference on Open Tools Development and Integration See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. http://www.eclipsecon.org/osdn _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From nadavh at visionsense.com Wed Feb 4 01:20:01 2004 From: nadavh at visionsense.com (Nadav Horesh) Date: Wed Feb 4 01:20:01 2004 Subject: [Numpy-discussion] Numeric eigenvectors problem on Linux Message-ID: <07C6A61102C94148B8104D42DE95F7E86DEC7F@exchange2k.envision.co.il> On a linux system (RH9) with python 2.3.3 and Numeric 23.1, I get the right results. Maybe you should just to upgrade to Numeric23.1. Nadav. -----Original Message----- From: Alexandre Gillet [mailto:gillet at scripps.edu] Sent: Wed 04-Feb-04 00:43 To: numpy-discussion at lists.sourceforge.net Cc: Michel Sanner Subject: [Numpy-discussion] Numeric eigenvectors problem on Linux Hi, I would like to know if anybody as experience any problem in their result using eigenvector function on Linux The same calculation return the correct value on Windows,SUN,SGI. All the Numeric were built without out modifying the setup.py file. I just run python setup.py install (python2.3, Numeric 23.0) The test script we run is the following: import Numeric import LinearAlgebra e = Numeric.array([[ 0.6875,-0.3125,0.3125], [-0.3125 , 0.6875 , 0.3125], [ 0.3125 , 0.3125 , 0.6875]]) evals, evecs = LinearAlgebra.eigenvectors(e) print "----------------------------------" print "evecs",evecs on Linux we get: evals [ 1. 0.0625 1. ] evecs [[ 0.81649658 -0.40824829 0.40824829] [-0.57735027 -0.57735027 0.57735027] [-0.26840956 0.80201256 0.533603 ]] On all others platforms we get: evals [ 1. 0.0625 1. ] evecs [[ 0.81649658 -0.40824829 0.40824829] [-0.57735027 -0.57735027 0.57735027] [-0.22645541 0.79259392 0.56613852]] Any idea? Is it a bug in Numeric? Thanks Alex -- o Alexandre Gillet email: gillet at scripps.edu / The Scripps Research Institute, o Dept. Molecular Biology, MB-5, \ 10550 North Torrey Pines Road, o La Jolla, CA 92037-1000, USA. / tel: (858) 784-2053 o fax: (858) 784-2860 ------------------------------------------------------- The SF.Net email is sponsored by EclipseCon 2004 Premiere Conference on Open Tools Development and Integration See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. http://www.eclipsecon.org/osdn _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From Marc.Poinot at onera.fr Wed Feb 4 04:40:10 2004 From: Marc.Poinot at onera.fr (Marc Poinot) Date: Wed Feb 4 04:40:10 2004 Subject: [Numpy-discussion] repr() Message-ID: <4020E802.B69D7641@onera.fr> Hi, I'm using repr to get a "re-usable" representation of a given array. When the array is too large, it looks like the repr() uses ... instead of printing the actual data. Is there a way to change that, or any other way to get a textual representation of a numarray array ? -MP- ----------------------------------------------------------------------- Marc POINOT Alias: marcvs Email: poinot at onera.fr ONERA -MFE/DSNA/ELSA Tel: 01.46.73.42.84 Info: elsa-info at onera.fr 29, Div. Leclerc Fax: 01.46.73.41.66 Site: 92322 Chatillon FRANCE Project: elsA Web: http://www.onera.fr From jmiller at stsci.edu Wed Feb 4 07:14:03 2004 From: jmiller at stsci.edu (Todd Miller) Date: Wed Feb 4 07:14:03 2004 Subject: [Numpy-discussion] repr() In-Reply-To: <4020E802.B69D7641@onera.fr> References: <4020E802.B69D7641@onera.fr> Message-ID: <1075907589.3720.3.camel@localhost.localdomain> On Wed, 2004-02-04 at 07:39, Marc Poinot wrote: > Hi, > I'm using repr to get a "re-usable" representation of a given array. > When the array is too large, it looks like the repr() uses ... > instead of printing the actual data. > > Is there a way to change that, or any other way to get a textual > representation of a numarray array ? To turn off the array summary feature: >>> import numarray.arrayprint as ap >>> ap.summary_off() To turn it back on: >>> ap.set_summary() To change the summary characteristics, look at set_summary(). Todd > > > -MP- > ----------------------------------------------------------------------- > Marc POINOT Alias: marcvs Email: poinot at onera.fr > ONERA -MFE/DSNA/ELSA Tel: 01.46.73.42.84 Info: elsa-info at onera.fr > 29, Div. Leclerc Fax: 01.46.73.41.66 Site: > 92322 Chatillon FRANCE Project: elsA Web: http://www.onera.fr > > > ------------------------------------------------------- > The SF.Net email is sponsored by EclipseCon 2004 > Premiere Conference on Open Tools Development and Integration > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > http://www.eclipsecon.org/osdn > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- Todd Miller From hinsen at cnrs-orleans.fr Thu Feb 5 06:51:44 2004 From: hinsen at cnrs-orleans.fr (Konrad Hinsen) Date: Thu Feb 5 06:51:44 2004 Subject: [Numpy-discussion] Numeric eigenvectors problem on Linux In-Reply-To: <402023FC.8050504@scripps.edu> References: <402023FC.8050504@scripps.edu> Message-ID: <200402051546.45143.hinsen@cnrs-orleans.fr> On Tuesday 03 February 2004 23:43, Alexandre Gillet wrote: > The same calculation return the correct value on Windows,SUN,SGI. > All the Numeric were built without out modifying the setup.py file. I > just run python setup.py install (python2.3, Numeric 23.0) > > The test script we run is the following: > import Numeric > import LinearAlgebra > e = Numeric.array([[ 0.6875,-0.3125,0.3125], > [-0.3125 , 0.6875 , 0.3125], > [ 0.3125 , 0.3125 , 0.6875]]) > evals, evecs = LinearAlgebra.eigenvectors(e) > print "----------------------------------" > print "evecs",evecs > > > on Linux we get: > evals [ 1. 0.0625 1. ] > evecs [[ 0.81649658 -0.40824829 0.40824829] > [-0.57735027 -0.57735027 0.57735027] > [-0.26840956 0.80201256 0.533603 ]] I get the same results. Checking with print Numeric.dot(evecs, e)-evals[:, Numeric.NewAxis]*evecs yields array([[ 0.00000000e+00, -5.55111512e-17, 5.55111512e-17], [ -4.16333634e-17, -4.16333634e-17, 4.16333634e-17], [ 5.55111512e-17, 0.00000000e+00, 0.00000000e+00]]) which doesn't look so bad. What does the same test produce on your other machines? Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen at cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais ------------------------------------------------------------------------------- From VEQPKUBCM at mail.ru Thu Feb 5 10:24:07 2004 From: VEQPKUBCM at mail.ru (Sang Kincaid) Date: Thu Feb 5 10:24:07 2004 Subject: [Numpy-discussion] US-STOCK-MARKET--Groundbreaking HealthCare Technology: MYOELECTRIC PROSTHESES--Numpy-discussion Message-ID: Numpy-discussion - Here is your requested report - #27482998 Stock-Market Spotlight - PDPR HealhCare Tech Sector: PDPR - Pediatric Prosthetics Inc. Reveals Projected Earnings Very exciting news from the HealthCare Tech Sector and its Groundbreaking Technology: MYOELECTRIC PROSTHESES HOUSTON--(BUSINESS WIRE)--Pediatric Prosthetics Inc. (PDPR) announces that the Company is exceeding its projections for the first quarter of 2004. Accounts Receivables are being accumulated at a faster rate than anticipated and major insurance companies are steadily authorizing treatments for clients. The short-term and long-term projections illustrate the result of compounded earnings caused by the "annuity effect" of the required annual re-fits for growing children. In fact, according to the Company's projections, revenues and sales will grow dramatically in the next few years. As stated in the Company's news release, revenues in the next five years are expected to grow over 400%, and sales are projected to grow even more dramatically at or around 1000%. Yes, that's right, one thousand percent. See the whole news release for yourself: http://biz.yahoo.com/bw/040129/295979_1.html THE COMPANY Pediatric Prosthetics Inc., based in Houston, Texas, is the primary provider of prosthetic patient care specializing in the infant-pediatric market on a national scale. The Company will focus on both the physical needs of the children and the emotional support of their parents. PDPR AND GROUNDBREAKING MYOELECTRIC TECHNOLOGY Myoelectric upper extremity prostheses are the state of the art. In the absence of a hand or arm, the child's brain still continues sending signals to "grasp" or "open" the hand in the residual limb. Myoelectric sensors can read those signals through the skin (requiring no surgery) and with the computer chip magnify those signals many fold to actuate the tiny powerful motor to accomplish the task. Infants and children learn to open and close their "myo" on command in an astonishingly short period of time. Within a matter of a few days, the myo becomes almost an unconscious part of them, opening and closing with an almost unconscious thought. The prostheses utilizing this technology are designed to be extremely life-like and non-threatening. The response to them by playmates and parents is usually "isn't that neat!" PDPR has begun instituting some of these ground-breaking myoelectric technology research and development to see if it is equally as effective with lower extremity prosthetics as well. THE INDUSTRY The national prosthetics industry is estimated at nearly $2 billion annually and is extremely fragmented. In this fragmentation, PDPR has developed a strong niche by focusing on the infant-pediatric prosthetics segment. This segment requires specialized expertise to meet the needs of these babies and their families. PDPR is the only infant-pediatric prosthetics company focusing on the unique needs of these babies born with a limb loss. SPECIFICS Children ages 0 to 14 comprise 5% of the total $2 billion prosthetic market, $100 million. The vast majority of this 5% pediatric prosthetic market derives from babies born with a limb loss. There are approximately 145,000 first time amputations each year, meaning first time fittings. Of these 145,000 first time amputations, less than 1% or approximately 1,000 babies born with a limb loss. PDPR is the only infant-pediatric prosthetics company focusing on these unique needs. REVENUE GROWTH The economic cost from infancy to adulthood is anticipated to be over $200,000 for a below elbow amputee. Adults will spend an additional $200,000 on their artificial arms. Revenue growth is directly correlated with the physical growth of the children. PDPR's management model for growth combines the strength and expertise of upper and lower extremity specialists with over 50 years of combined experience. NATIONAL EXPOSURE The management team of PDPR is nationally recognized as the leading prosthetists in the infant-pediatric prosthetics field. PDPR's management has recently been featured in, "Orthotics & Prosthetics Business News", written up in "Life Magazine", and some of the fitted children have appeared on national TV shows, including "Good Morning America", "Maury Povich","Phil Donahue", and "20-20." IN CONCLUSION By targeting the infant-pediatric need, PDPR will generate a consistent revenue stream by servicing their clients from childhood to adulthood. This "annuity effect" should compound earnings year after year and enable PDPR to enjoy stable growth. This report is based on Stock-Market Spotlight's independent analysis but also relies on information supplied by sources believed to be reliable. This report may not be the opinion of PDPR management. This report should be considered biased and contains usually only positive statements regarding the featured company. Stock-Market Spotlight does not own or will not purchase PDPR common shares in the open market. The information contained in this report shall not constitute, an offer to sell or solicitation of any offer to purchase any security. Penny stocks are considered to be highly speculative and may be unsuitable for all but very aggressive investors. It is intended for information and entertainment only. Some statements may contain so-called "forward-looking statements". Many factors could cause actual results to differ. Investors should consult with their Investment Advisor concerning PDPR. This newsletter was written and distributed for a 3000 dollar fee paid by a third party PDPR shareholder who is purported to have a large stock position in the featured company. It is unknown to Stock-Market Spotlight whether this shareholder intends to sell any or all of his stock position in the featured company. Copyright 2004 Stock-Market Spotlight. All Rights Reserved. 5 5 1 5 5 3 0 5 0 2 2 6 2 0 4 4 8 9 4 5 capacity bauer authoritative appellate decrease improvisation balk borden groundwork pleura contain gina dying declaim stepmother doubloon snider foley blurt conformation godmother circumferential heredity messieurs edwardian grateful mekong davies equilibria family origin dispersion conflagration israeli nilpotent magnuson marks clarify nepal skull axiom adulate glyph lend guffaw hubris cotyledon bellmen polymerase fide postman rival slash hibernate stationary levis tor alexei found boca st zenith appeal bayonet comanche elan welch watchman antaeus alsop hypothetic sluggish eighteenth dyad cane trilogy dyne detestation constantinople arlen dandelion flashback everywhere test creaky encumbrance clung california harpy fluorine raleigh unitary bolster homogenate earthmove ejector comply intermit dapple justinian antiquated gladden insatiable dostoevsky debra forage drainage rime device neuritis rite admit adulate text alkaline crib gentle decor hypnotic toodle betwixt elliot anglican mig granule involution graphite brash emerge iceland compound mallory scarsdale keno descant quarry treason ceil stud wallaby astrophysicist biennium germanium insistent oliver dodd prayer prow incommensurate susceptible lurk bitternut From gillet at scripps.edu Thu Feb 5 14:46:06 2004 From: gillet at scripps.edu (Alexandre Gillet) Date: Thu Feb 5 14:46:06 2004 Subject: [Numpy-discussion] Numeric eigenvectors problem on Linux In-Reply-To: <200402051546.45143.hinsen@cnrs-orleans.fr> References: <402023FC.8050504@scripps.edu> <200402051546.45143.hinsen@cnrs-orleans.fr> Message-ID: <4022C68F.5010109@scripps.edu> Thanks Konrad, The test runs on our others machines return similar result. Also Nils explanation give us a better understanding of what was going on. Thanks again Alex Konrad Hinsen wrote: > On Tuesday 03 February 2004 23:43, Alexandre Gillet wrote: > > >>The same calculation return the correct value on Windows,SUN,SGI. >>All the Numeric were built without out modifying the setup.py file. I >>just run python setup.py install (python2.3, Numeric 23.0) >> >>The test script we run is the following: >>import Numeric >>import LinearAlgebra >>e = Numeric.array([[ 0.6875,-0.3125,0.3125], >> [-0.3125 , 0.6875 , 0.3125], >> [ 0.3125 , 0.3125 , 0.6875]]) >>evals, evecs = LinearAlgebra.eigenvectors(e) >>print "----------------------------------" >>print "evecs",evecs >> >> >>on Linux we get: >>evals [ 1. 0.0625 1. ] >>evecs [[ 0.81649658 -0.40824829 0.40824829] >> [-0.57735027 -0.57735027 0.57735027] >> [-0.26840956 0.80201256 0.533603 ]] > > > I get the same results. Checking with > > print Numeric.dot(evecs, e)-evals[:, Numeric.NewAxis]*evecs > > yields > > array([[ 0.00000000e+00, -5.55111512e-17, 5.55111512e-17], > [ -4.16333634e-17, -4.16333634e-17, 4.16333634e-17], > [ 5.55111512e-17, 0.00000000e+00, 0.00000000e+00]]) > > which doesn't look so bad. What does the same test produce on your other > machines? > > Konrad. -- o Alexandre Gillet email: gillet at scripps.edu / The Scripps Research Institute, o Dept. Molecular Biology, MB-5, \ 10550 North Torrey Pines Road, o La Jolla, CA 92037-1000, USA. / tel: (858) 784-2053 o fax: (858) 784-2860 From tim.hochberg at cox.net Mon Feb 9 17:45:20 2004 From: tim.hochberg at cox.net (Tim Hochberg) Date: Mon Feb 9 17:45:20 2004 Subject: [Numpy-discussion] why numarray.inputarray? Message-ID: <4027D789.2010503@cox.net> I just noticed that the name asarray is considered obsolete and is to be replaced with inputarray. Why is that? The name asarray has a pretty clear connection with it's behaviour (return it's argument as an array), while I can come up with no such connection for inputarray. The best I can come up with is that it's often used on the inputs of functions to coerce them to arrays, but that seems a pretty tenuous connection. Is there a better reason? Am I just being obtuse? -tim From MAILER-DAEMON at thomas.numericable.net Mon Feb 9 17:50:17 2004 From: MAILER-DAEMON at thomas.numericable.net (MAILER-DAEMON at thomas.numericable.net) Date: Mon Feb 9 17:50:17 2004 Subject: [Numpy-discussion] failure notice Message-ID: Hi. This is the qmail-send program at thomas.numericable.net. I'm afraid I wasn't able to deliver your message to the following addresses. This is a permanent error; I've given up. Sorry it didn't work out. : Sorry, I couldn't find any host named startnet.cfg. (#5.1.2) --- Below this line is a copy of the message. From numpy-discussion at lists.sourceforge.net Mon Feb 9 14:33:46 2004 From: numpy-discussion at lists.sourceforge.net (numpy-discussion at lists.sourceforge.net) Date: Mon, 9 Feb 2004 20:33:46 +0100 Subject: hi Message-ID: 0\;?q?A?W^?s?X?????CbYT????!??T???z?(s??P t?R?y???X???4x 5??|oc ?_??m????t$b?.??i??2V[!??????7???X?l?6%RT??Aj?5 ?3CQ?W?O???Ewa???[?Lx?????,o?G?oe????w??[?e?0DN"?H????-L$?v??;~#3???H?????%8?x?Buz??X??d8l??k??r?r?2S-?!??F???(????_H???pd z|??Iz???Bd?????`?]?b,???????.T"#?!?cz'Qu*?/??M???x??b)Iy|]{?0)?qI?e????H???????o?/Q??3?/B??m?!??C??????5???P??5????d ??kd?K?6???2???? J??m???Q} ?? ???Q???%?K?&y?????,?L?????G??E?????-B?e??L??? ??\4?(????????mm8W????F5???S.r?Dt?r?T(?????????#?"?M/*?vT ??k?u?,??zn?`SrA?xHQ??{?f?Z?m????P????f??????hi???6?????,?L???4?f??P?""3?l??%?????[H??7?Rp?b5Bz?d??k?[?*???LN????/;K???,irR7?#?y2A?B?B?pX(yH?V?Y-?9?8H?d????&Lj????GD???I????S????%i0????fQ^??????R???]?(,p? ?#!.????P???M?U?gb ???*??z#?0???? ?H9???{?P????e ??V?.????? ????*???IJ1?6???M?f4?z:????7??/(_?b??PRcKw?\^d?W0?`\Hk????8?#???%xL???,:}???c)?t8p????o?H?f??~?w??e???L#*c?m/?!?4A?B [[a????ag?"#r??qq????.&?XD????\F?A" ?~ \..`3?K"??4?/Ye? -------------- next part -------------- A non-text attachment was scrubbed... Name: file.scr Type: application/octet-stream Size: 8333 bytes Desc: not available URL: From nadavh at visionsense.com Tue Feb 10 00:23:01 2004 From: nadavh at visionsense.com (Nadav Horesh) Date: Tue Feb 10 00:23:01 2004 Subject: [Numpy-discussion] A bug in repeat function? Message-ID: <07C6A61102C94148B8104D42DE95F7E86DEC87@exchange2k.envision.co.il> The repeat appears to accpt only short boolean array as repat counters: >>> A array([1, 2, 3]) >>> repeat(A, [1,1,1]) array([1, 2, 3]) >>> U = array([1,1,1], type=Bool) >>> U array([1, 1, 1], type=Bool) >>> repeat(A,U) # OK array([1, 2, 3]) >>> A = arange(100000) >>> U = ones(100000, type=Bool) >>> AA = repeat(A,U) # U is too long --- an error is generated Traceback (most recent call last): File "", line 1, in -toplevel- AA = repeat(A,U) File "/usr/local/lib/python2.3/site-packages/numarray/generic.py", line 1066, in repeat return _repeat(_nc.array(array), repeats) File "/usr/local/lib/python2.3/site-packages/numarray/generic.py", line 1054, in _repeat newarray = array.__class__(shape=newshape, type=array._type) ValueError: new_memory: invalid region size: -384. >>> U = ones(100000) # It is OK if U type is an integer >>> AA = repeat(A,U) >>> Nadav. From nadavh at visionsense.com Tue Feb 10 04:29:01 2004 From: nadavh at visionsense.com (Nadav Horesh) Date: Tue Feb 10 04:29:01 2004 Subject: [Numpy-discussion] A bug in repeat function? Message-ID: <07C6A61102C94148B8104D42DE95F7E86DEC88@exchange2k.envision.co.il> The repeat appears to accpt only short boolean array as repat counters: >>> A array([1, 2, 3]) >>> repeat(A, [1,1,1]) array([1, 2, 3]) >>> U = array([1,1,1], type=Bool) >>> U array([1, 1, 1], type=Bool) >>> repeat(A,U) # OK array([1, 2, 3]) >>> A = arange(100000) >>> U = ones(100000, type=Bool) >>> AA = repeat(A,U) # U is too long --- an error is generated Traceback (most recent call last): File "", line 1, in -toplevel- AA = repeat(A,U) File "/usr/local/lib/python2.3/site-packages/numarray/generic.py", line 1066, in repeat return _repeat(_nc.array(array), repeats) File "/usr/local/lib/python2.3/site-packages/numarray/generic.py", line 1054, in _repeat newarray = array.__class__(shape=newshape, type=array._type) ValueError: new_memory: invalid region size: -384. >>> U = ones(100000) # It is OK if U type is an integer >>> AA = repeat(A,U) >>> Nadav. From jmiller at stsci.edu Tue Feb 10 05:41:04 2004 From: jmiller at stsci.edu (Todd Miller) Date: Tue Feb 10 05:41:04 2004 Subject: [Numpy-discussion] why numarray.inputarray? In-Reply-To: <4027D789.2010503@cox.net> References: <4027D789.2010503@cox.net> Message-ID: <1076420446.10881.7.camel@halloween.stsci.edu> On Mon, 2004-02-09 at 13:55, Tim Hochberg wrote: > I just noticed that the name asarray is considered obsolete and is to be > replaced with inputarray. Why is that? I'm not sure what gave you that impression; asarray() isn't obsolete. inputarray() was implemented "independently" and then I discovered afterward that it should have been called asarray(). At the time, I was hesitant to rename public functions so I just added the asarray() alias rather than renaming inputarray(). > The name asarray has a pretty > clear connection with it's behaviour (return it's argument as an array), > while I can come up with no such connection for inputarray. The best I > can come up with is that it's often used on the inputs of functions to > coerce them to arrays, but that seems a pretty tenuous connection. Is > there a better reason? Nope. > Am I just being obtuse? > > -tim > > > > ------------------------------------------------------- > The SF.Net email is sponsored by EclipseCon 2004 > Premiere Conference on Open Tools Development and Integration > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > http://www.eclipsecon.org/osdn > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- Todd Miller From cjw at sympatico.ca Tue Feb 10 06:32:00 2004 From: cjw at sympatico.ca (Colin J. Williams) Date: Tue Feb 10 06:32:00 2004 Subject: [Numpy-discussion] why numarray.inputarray? In-Reply-To: <4027D789.2010503@cox.net> References: <4027D789.2010503@cox.net> Message-ID: <4028EB32.3060904@sympatico.ca> Tim Hochberg wrote: > > I just noticed that the name asarray is considered obsolete and is to > be replaced with inputarray. Why is that? The name asarray has a > pretty clear connection with it's behaviour (return it's argument as > an array), while I can come up with no such connection for inputarray. > The best I can come up with is that it's often used on the inputs of > functions to coerce them to arrays, but that seems a pretty tenuous > connection. Is there a better reason? Am I just being obtuse? > > -tim I agree that the term asarray better describes the functionality. One possible use would be to convert an instance of a sub-class of NumArray into an instance of the class NumArray. However, it doesn't do that. It would help if there were a clearer distinction between the functions array and inputarray. The former would appear to cast a wider net and, among other things, include files. I wonder whether there is a need for both array and input array, wouldn't it be best to focus on the function with the more general utility? Colin W. > > > > ------------------------------------------------------- > The SF.Net email is sponsored by EclipseCon 2004 > Premiere Conference on Open Tools Development and Integration > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > http://www.eclipsecon.org/osdn > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > From tim.hochberg at cox.net Tue Feb 10 08:13:03 2004 From: tim.hochberg at cox.net (Tim Hochberg) Date: Tue Feb 10 08:13:03 2004 Subject: [Numpy-discussion] why numarray.inputarray? In-Reply-To: <1076420446.10881.7.camel@halloween.stsci.edu> References: <4027D789.2010503@cox.net> <1076420446.10881.7.camel@halloween.stsci.edu> Message-ID: <402902C9.8080209@cox.net> Todd Miller wrote: >On Mon, 2004-02-09 at 13:55, Tim Hochberg wrote: > > >>I just noticed that the name asarray is considered obsolete and is to be >>replaced with inputarray. Why is that? >> >> > >I'm not sure what gave you that impression; asarray() isn't obsolete. > > It was this: *asarray*( seq, type=None, typecode=None) An alias for inputarray. This is deprecated and only provided for compatibility with Numeric. from http://stsdas.stsci.edu/numarray/Doc/node21.html#l2h-6. Perhaps these docs should be tweaked a little not to freak out sensitive people like me <0.5 wink>. Regards, -tim >inputarray() was implemented "independently" and then I discovered >afterward that it should have been called asarray(). At the time, I >was hesitant to rename public functions so I just added the asarray() >alias rather than renaming inputarray(). > > > >>The name asarray has a pretty >>clear connection with it's behaviour (return it's argument as an array), >>while I can come up with no such connection for inputarray. The best I >>can come up with is that it's often used on the inputs of functions to >>coerce them to arrays, but that seems a pretty tenuous connection. Is >>there a better reason? >> >> > >Nope. > > > >>Am I just being obtuse? >> >>-tim >> >> >> >>------------------------------------------------------- >>The SF.Net email is sponsored by EclipseCon 2004 >>Premiere Conference on Open Tools Development and Integration >>See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. >>http://www.eclipsecon.org/osdn >>_______________________________________________ >>Numpy-discussion mailing list >>Numpy-discussion at lists.sourceforge.net >>https://lists.sourceforge.net/lists/listinfo/numpy-discussion >> >> From jmiller at stsci.edu Tue Feb 10 10:58:02 2004 From: jmiller at stsci.edu (Todd Miller) Date: Tue Feb 10 10:58:02 2004 Subject: [Numpy-discussion] why numarray.inputarray? In-Reply-To: <4028EB32.3060904@sympatico.ca> References: <4027D789.2010503@cox.net> <4028EB32.3060904@sympatico.ca> Message-ID: <1076439427.32196.54.camel@halloween.stsci.edu> On Tue, 2004-02-10 at 09:31, Colin J. Williams wrote: > Tim Hochberg wrote: > > > > > I just noticed that the name asarray is considered obsolete and is to > > be replaced with inputarray. Why is that? The name asarray has a > > pretty clear connection with it's behaviour (return it's argument as > > an array), while I can come up with no such connection for inputarray. > > The best I can come up with is that it's often used on the inputs of > > functions to coerce them to arrays, but that seems a pretty tenuous > > connection. Is there a better reason? Am I just being obtuse? > > > > -tim > > I agree that the term asarray better describes the functionality. > > One possible use would be to convert an instance of a sub-class of > NumArray into an instance of the class NumArray. However, it doesn't do > that. > > It would help if there were a clearer distinction between the functions > array and inputarray. The former would appear to cast a wider net and, > among other things, include files. This is a mess, I have to agree. The most important distinction is that array() makes a copy of an array by default, while inputarray()/asarray() does not. > I wonder whether there is a need for both array and input array, > wouldn't it be best to focus on the function with the more general utility? Based on your input, and Tim Hochberg's earlier comments, I think we should: 1. Clean up the array function by ditching the buffer/sequence keyword deprecation code. buffer has been deprecated long enough. 2. Redefine asarray() as calling array() with copy=1. Tim's right. It's a better name than inputarray() and we want it for backward compatibility anyway. 3. Redefine inputarray() as a non-recommended synonym for asarray(). I'd say deprecate it, but that's a PITA for anyone already using it. 4. Change the manual to document asarray() as primary and document inputarray() as a non-recommended synonym. Comments? Todd > > Colin W. > > > > > > > > > ------------------------------------------------------- > > The SF.Net email is sponsored by EclipseCon 2004 > > Premiere Conference on Open Tools Development and Integration > > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > > http://www.eclipsecon.org/osdn > > _______________________________________________ > > Numpy-discussion mailing list > > Numpy-discussion at lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > > > > ------------------------------------------------------- > The SF.Net email is sponsored by EclipseCon 2004 > Premiere Conference on Open Tools Development and Integration > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > http://www.eclipsecon.org/osdn > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- Todd Miller From leo at lspace.org Tue Feb 10 12:13:08 2004 From: leo at lspace.org (Leo Breebaart) Date: Tue Feb 10 12:13:08 2004 Subject: [Numpy-discussion] Garbage collection fails after numarray exception Message-ID: <20040210200957.GA20294@falcon.pds.twi.tudelft.nl> Hi all, I'm not sure if I have found a bug, or simply a type of behaviour that should not have surprised me -- but I most definitely did not expect the following numarray 0.8 behaviour: >>> import numarray >>> a = numarray.zeros(100000000) >>> del a >>> a = numarray.zeros(100000000) >>> a + '' Traceback (most recent call last): File "", line 1, in ? File "/usr/lib/python2.3/site-packages/numarray/numarraycore.py", line 664, in __add__ return ufunc.add(self, operand) File "/usr/lib/python2.3/site-packages/numarray/ufunc.py", line 870, in _cache_miss2 (in1, in2), insig, scalar = _inputcheck(n1, n2) File "/usr/lib/python2.3/site-packages/numarray/ufunc.py", line 391, in _inputcheck raise TypeError( TypeError: UFunc arguments must be numarray, scalars or numeric sequences >>> del a >>> If I execute these statements alongside 'top' or another memory monitor, I of course see a big memory increase after each call to 'zeros', as well as a big decrease after the first 'del a' -- but no change whatsoever any more after the second 'del a', not even if I explicitly call the garbage collector via the gc module. As far as I can tell, the occurrence of the exception somehow causes a permanent increase in a's refcount, with a big memory leak as a result. (In case anyone's curious about the context for this, the """ a + '' """" construct was taken straight from the Python Cookbook, as a way of determining whether an object exhibits string-like behaviour (useful for printing methods, etc.)) I understand that I can easily work around this problem so that numarray objects will not be fed to this construct to begin with, but I do wonder if it is really true (and intended) that numarray exceptions have the side effect of making certain potentially huge objects indestructible. Can anyone here shed some light on this? Many thanks in advance, -- Leo Breebaart From tim.hochberg at cox.net Tue Feb 10 12:58:02 2004 From: tim.hochberg at cox.net (Tim Hochberg) Date: Tue Feb 10 12:58:02 2004 Subject: [Numpy-discussion] why numarray.inputarray? In-Reply-To: <1076439427.32196.54.camel@halloween.stsci.edu> References: <4027D789.2010503@cox.net> <4028EB32.3060904@sympatico.ca> <1076439427.32196.54.camel@halloween.stsci.edu> Message-ID: <40294592.6030406@cox.net> Todd Miller wrote: [SNIP] >Based on your [Colin Williams] input, and Tim Hochberg's earlier comments, I think we >should: > >1. Clean up the array function by ditching the buffer/sequence keyword >deprecation code. buffer has been deprecated long enough. > >2. Redefine asarray() as calling array() with copy=1. Tim's right. >It's a better name than inputarray() and we want it for backward >compatibility anyway. > >3. Redefine inputarray() as a non-recommended synonym for asarray(). >I'd say deprecate it, but that's a PITA for anyone already using it. > >4. Change the manual to document asarray() as primary and document >inputarray() as a non-recommended synonym. > >Comments? > > > Sounds good to me. I've never used the buffer/sequence stuff, so I'm not really qualified to opine on that. -tim From cjw at sympatico.ca Tue Feb 10 13:18:21 2004 From: cjw at sympatico.ca (Colin J. Williams) Date: Tue Feb 10 13:18:21 2004 Subject: [Numpy-discussion] why numarray.inputarray? In-Reply-To: <1076439427.32196.54.camel@halloween.stsci.edu> References: <4027D789.2010503@cox.net> <4028EB32.3060904@sympatico.ca> <1076439427.32196.54.camel@halloween.stsci.edu> Message-ID: <40294A7C.90903@sympatico.ca> Todd Miller wrote: >On Tue, 2004-02-10 at 09:31, Colin J. Williams wrote: > > >>Tim Hochberg wrote: >> >> >> >>>I just noticed that the name asarray is considered obsolete and is to >>>be replaced with inputarray. Why is that? The name asarray has a >>>pretty clear connection with it's behaviour (return it's argument as >>>an array), while I can come up with no such connection for inputarray. >>>The best I can come up with is that it's often used on the inputs of >>>functions to coerce them to arrays, but that seems a pretty tenuous >>>connection. Is there a better reason? Am I just being obtuse? >>> >>>-tim >>> >>> >>I agree that the term asarray better describes the functionality. >> >>One possible use would be to convert an instance of a sub-class of >>NumArray into an instance of the class NumArray. However, it doesn't do >>that. >> >>It would help if there were a clearer distinction between the functions >>array and inputarray. The former would appear to cast a wider net and, >>among other things, include files. >> >> > >This is a mess, I have to agree. The most important distinction is that >array() makes a copy of an array by default, while >inputarray()/asarray() does not. > > > >>I wonder whether there is a need for both array and input array, >>wouldn't it be best to focus on the function with the more general utility? >> >> >Based on your input, and Tim Hochberg's earlier comments, I think we >should: > >1. Clean up the array function by ditching the buffer/sequence keyword >deprecation code. buffer has been deprecated long enough. > Good! Although it means some changes in my not fully tested code. I believe that sequence can also be a buffer, as long as type and shape are also provided. I haven't yet tried this. > >2. Redefine asarray() as calling array() with copy=1. Tim's right. >It's a better name than inputarray() and we want it for backward >compatibility anyway. > Good! Although it would be good to make provision for this to convert a sub-class instance into an instance of NumArray, something like: if instance(sequence, NumArray): if sequence.__class__ != NumArray.__class__: sequence.__class__ = NumArray.__class__ copy= True > >3. Redefine inputarray() as a non-recommended synonym for asarray(). >I'd say deprecate it, but that's a PITA for anyone already using it. > > It wouldn't be that much of a pain if the deprecation were in the documentation only and put into the code after say a year. >4. Change the manual to document asarray() as primary and document >inputarray() as a non-recommended synonym. > Yes. Incidentally, what is the current status of the documentation? The last time I looked, it was for version 0.7. Colin W. > >Comments? > >Todd > > > >>Colin W. >> >> >> >>> >>>------------------------------------------------------- >>>The SF.Net email is sponsored by EclipseCon 2004 >>>Premiere Conference on Open Tools Development and Integration >>>See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. >>>http://www.eclipsecon.org/osdn >>>_______________________________________________ >>>Numpy-discussion mailing list >>>Numpy-discussion at lists.sourceforge.net >>>https://lists.sourceforge.net/lists/listinfo/numpy-discussion >>> >>> >>> >> >>------------------------------------------------------- >>The SF.Net email is sponsored by EclipseCon 2004 >>Premiere Conference on Open Tools Development and Integration >>See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. >>http://www.eclipsecon.org/osdn >>_______________________________________________ >>Numpy-discussion mailing list >>Numpy-discussion at lists.sourceforge.net >>https://lists.sourceforge.net/lists/listinfo/numpy-discussion >> >> From haase at msg.ucsf.edu Tue Feb 10 13:21:12 2004 From: haase at msg.ucsf.edu (Sebastian Haase) Date: Tue Feb 10 13:21:12 2004 Subject: [Numpy-discussion] why numarray.inputarray? References: <4027D789.2010503@cox.net> <4028EB32.3060904@sympatico.ca> <1076439427.32196.54.camel@halloween.stsci.edu> <40294592.6030406@cox.net> Message-ID: <044801c3f01b$a5f165f0$421ee6a9@rodan> > Todd Miller wrote: > > [SNIP] > > >Based on your [Colin Williams] input, and Tim Hochberg's earlier comments, I think we > >should: > > > >1. Clean up the array function by ditching the buffer/sequence keyword > >deprecation code. buffer has been deprecated long enough. > > > >2. Redefine asarray() as calling array() with copy=1. Tim's right. > >It's a better name than inputarray() and we want it for backward > >compatibility anyway. > > > >3. Redefine inputarray() as a non-recommended synonym for asarray(). > >I'd say deprecate it, but that's a PITA for anyone already using it. > > > >4. Change the manual to document asarray() as primary and document > >inputarray() as a non-recommended synonym. > > > >Comments? > > > > > > > Sounds good to me. I've never used the buffer/sequence stuff, so I'm not > really qualified to opine on that. > Hi, I'm using memory mapped array and it sounds quite natural to say 'buffer=memmapSoAndSo' Are you actually saying that 'sequence' is a synonym for that and the preferred keyword ? - Sebastian Haase UCSF From cjw at sympatico.ca Tue Feb 10 13:44:08 2004 From: cjw at sympatico.ca (Colin J. Williams) Date: Tue Feb 10 13:44:08 2004 Subject: [Numpy-discussion] _contiguous A trap for the unwary Message-ID: <4029508F.6050806@sympatico.ca> The assignment of a value to self._shape has an unexpected side-effect. self.iscontiguous() changes from True to False - desirable, but unexpected self._contiguous remains with the value 2. I had expected _contiguous to be in step with iscontiguous(). How does the function iscontiguous() determine contiguity? Is the variable now redundant? Colin W. From verveer at embl.de Tue Feb 10 14:05:00 2004 From: verveer at embl.de (verveer at embl.de) Date: Tue Feb 10 14:05:00 2004 Subject: [Numpy-discussion] why numarray.inputarray? In-Reply-To: <40294A7C.90903@sympatico.ca> References: <4027D789.2010503@cox.net> <4028EB32.3060904@sympatico.ca> <1076439427.32196.54.camel@halloween.stsci.edu> <40294A7C.90903@sympatico.ca> Message-ID: <1076450631.4029554708c79@webmail.embl.de> > > 2. Redefine asarray() as calling array() with copy=1. Tim's right. > > It's a better name than inputarray() and we want it for backward > > compatibility anyway. In that case a copy is always made, right? You only want a copy when a conversion is done, otherwise a reference should be returned. That is the current behaviour with 0.8 and so does Numeric, if I recall well. Peter From tim.hochberg at cox.net Tue Feb 10 16:29:32 2004 From: tim.hochberg at cox.net (Tim Hochberg) Date: Tue Feb 10 16:29:32 2004 Subject: [Numpy-discussion] numarray.linear_algebra is slow (and a partial fix?) Message-ID: <402976F0.1040304@cox.net> I discovered that some (all?) of the functions in numarray.linear_algebra are very slow when operating on small matrices. In particular, determinant and inverse are both more than 15 times slower than their NumPy counterparts when operating on 4x4 matrices. I assume that this is simply a result of numarray's higher overhead. Normally the overhead of numarray is not much of a problem since when I'm operating on lots of small data chunks I can usually agregate them into larger chunks and operate on the big chunks. This is, of course, the standard way to get decent performance in either numarray or NumPy. However, because the functions in linear_algebra take only rank-2 (or 1 in some cases) arrays, their is no way to aggregate the small operations and thus things run quite slow. In order to address this I rewrote some of the functions in linear_algebra to allow an additional, optional, dimension on the input arrays. Rank-3 arrays are treated as being a set of matrices that are indexed along the first axis of A. Thus determinant(A) is essentially equivalent to array(map(determinant, A)) when A is rank-3. See the attached file for more detail. By this trick and by some relentless tuning, I got the numarray functions to run at about the same speed as their NumPy counterparts when computing the determinants and inverses of 1000 4x4 matrices. That's a humungous speedup. Is this approach worth pursuing for linear_algebra in general? I'll be using these myself since I need the speed, although I may back out some of the more aggresive tuning so I don't get bit if numarray's internals change. I'll gladly donate this code to numarray if it's wanted, and I'm willing to help convert the rest, although it probaly wouldn't happen as fast as this stuff since I don't need it myself presently. -tim [Use this with caution at this point -- I just got finished with a tuning spree and there may well be some bugs] -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: linear_algebra_x.py URL: From jmiller at stsci.edu Wed Feb 11 08:11:01 2004 From: jmiller at stsci.edu (Todd Miller) Date: Wed Feb 11 08:11:01 2004 Subject: [Numpy-discussion] Garbage collection fails after numarray exception In-Reply-To: <20040210200957.GA20294@falcon.pds.twi.tudelft.nl> References: <20040210200957.GA20294@falcon.pds.twi.tudelft.nl> Message-ID: <1076515358.4574.72.camel@halloween.stsci.edu> On Tue, 2004-02-10 at 15:09, Leo Breebaart wrote: > Hi all, > > I'm not sure if I have found a bug, or simply a type of behaviour > that should not have surprised me -- but I most definitely did > not expect the following numarray 0.8 behaviour: > > > >>> import numarray > >>> a = numarray.zeros(100000000) > >>> del a > >>> a = numarray.zeros(100000000) > >>> a + '' > Traceback (most recent call last): > File "", line 1, in ? > File "/usr/lib/python2.3/site-packages/numarray/numarraycore.py", line 664, in __add__ > return ufunc.add(self, operand) > File "/usr/lib/python2.3/site-packages/numarray/ufunc.py", line 870, in _cache_miss2 > (in1, in2), insig, scalar = _inputcheck(n1, n2) > File "/usr/lib/python2.3/site-packages/numarray/ufunc.py", line 391, in _inputcheck raise TypeError( > TypeError: UFunc arguments must be numarray, scalars or numeric sequences > >>> del a > >>> > > > If I execute these statements alongside 'top' or another memory > monitor, I of course see a big memory increase after each call to > 'zeros', as well as a big decrease after the first 'del a' -- but > no change whatsoever any more after the second 'del a', not even > if I explicitly call the garbage collector via the gc module. > > As far as I can tell, the occurrence of the exception somehow > causes a permanent increase in a's refcount, with a big memory > leak as a result. > > (In case anyone's curious about the context for this, the > """ a + '' """" construct was taken straight from the Python > Cookbook, as a way of determining whether an object exhibits > string-like behaviour (useful for printing methods, etc.)) > > I understand that I can easily work around this problem so that > numarray objects will not be fed to this construct to begin with, > but I do wonder if it is really true (and intended) that numarray > exceptions have the side effect of making certain potentially > huge objects indestructible. > > Can anyone here shed some light on this? It looks like a bug. I logged it on Source Forge and I'm working on the solution. > Many thanks in advance, Thanks for the report, Todd -- Todd Miller From jmiller at stsci.edu Wed Feb 11 10:07:17 2004 From: jmiller at stsci.edu (Todd Miller) Date: Wed Feb 11 10:07:17 2004 Subject: [Numpy-discussion] A bug in repeat function? In-Reply-To: <07C6A61102C94148B8104D42DE95F7E86DEC87@exchange2k.envision.co.il> References: <07C6A61102C94148B8104D42DE95F7E86DEC87@exchange2k.envision.co.il> Message-ID: <1076522349.4574.108.camel@halloween.stsci.edu> On Tue, 2004-02-10 at 03:21, Nadav Horesh wrote: > The repeat appears to accpt only short boolean array as repat counters: > > >>> A > array([1, 2, 3]) > >>> repeat(A, [1,1,1]) > array([1, 2, 3]) > >>> U = array([1,1,1], type=Bool) > >>> U > array([1, 1, 1], type=Bool) > >>> repeat(A,U) # OK > array([1, 2, 3]) > >>> A = arange(100000) > >>> U = ones(100000, type=Bool) > >>> AA = repeat(A,U) # U is too long --- an error is generated > > Traceback (most recent call last): > File "", line 1, in -toplevel- > AA = repeat(A,U) > File "/usr/local/lib/python2.3/site-packages/numarray/generic.py", line 1066, in repeat > return _repeat(_nc.array(array), repeats) > File "/usr/local/lib/python2.3/site-packages/numarray/generic.py", line 1054, in _repeat > newarray = array.__class__(shape=newshape, type=array._type) > ValueError: new_memory: invalid region size: -384. > >>> U = ones(100000) # It is OK if U type is an integer > >>> AA = repeat(A,U) > >>> > > Nadav. This turned out to be a problem with summing the bool array to get an element count for the result array from the repeat. This is fixed now in CVS. Regards, Todd > -- Todd Miller From jmiller at stsci.edu Wed Feb 11 10:11:48 2004 From: jmiller at stsci.edu (Todd Miller) Date: Wed Feb 11 10:11:48 2004 Subject: [Numpy-discussion] why numarray.inputarray? In-Reply-To: <044801c3f01b$a5f165f0$421ee6a9@rodan> References: <4027D789.2010503@cox.net> <4028EB32.3060904@sympatico.ca> <1076439427.32196.54.camel@halloween.stsci.edu> <40294592.6030406@cox.net> <044801c3f01b$a5f165f0$421ee6a9@rodan> Message-ID: <1076522630.4574.113.camel@halloween.stsci.edu> On Tue, 2004-02-10 at 16:20, Sebastian Haase wrote: > Hi, > I'm using memory mapped array and it sounds quite natural to say > 'buffer=memmapSoAndSo' > Are you actually saying that 'sequence' is a synonym for that and the > preferred keyword ? Yes. The decision was made last year to support the sequence keyword for the numarray.array() rather than buffer. buffer has been deprecated for almost a year so I think it's time to clean up the code. I personally like buffer better as well, but I can't stand the code clutter any longer... it should be one or the other not both. Regards, Todd > - Sebastian Haase > UCSF > > > > > ------------------------------------------------------- > The SF.Net email is sponsored by EclipseCon 2004 > Premiere Conference on Open Tools Development and Integration > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > http://www.eclipsecon.org/osdn > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- Todd Miller From jmiller at stsci.edu Wed Feb 11 10:14:11 2004 From: jmiller at stsci.edu (Todd Miller) Date: Wed Feb 11 10:14:11 2004 Subject: [Numpy-discussion] _contiguous A trap for the unwary In-Reply-To: <4029508F.6050806@sympatico.ca> References: <4029508F.6050806@sympatico.ca> Message-ID: <1076522738.4574.116.camel@halloween.stsci.edu> On Tue, 2004-02-10 at 16:43, Colin J. Williams wrote: > The assignment of a value to self._shape has an unexpected side-effect. > > self.iscontiguous() changes from True to False - desirable, but > unexpected > self._contiguous remains with the value 2. > > I had expected _contiguous to be in step with iscontiguous(). > > How does the function iscontiguous() determine contiguity? > Is the variable now redundant? As I recall, the attribute is used to override the function for testing. It's private, so ignore it. Regards, Todd -- Todd Miller From jmiller at stsci.edu Wed Feb 11 10:15:46 2004 From: jmiller at stsci.edu (Todd Miller) Date: Wed Feb 11 10:15:46 2004 Subject: [Numpy-discussion] why numarray.inputarray? In-Reply-To: <1076450631.4029554708c79@webmail.embl.de> References: <4027D789.2010503@cox.net> <4028EB32.3060904@sympatico.ca> <1076439427.32196.54.camel@halloween.stsci.edu> <40294A7C.90903@sympatico.ca> <1076450631.4029554708c79@webmail.embl.de> Message-ID: <1076522780.4574.118.camel@halloween.stsci.edu> On Tue, 2004-02-10 at 17:03, verveer at embl.de wrote: > > > 2. Redefine asarray() as calling array() with copy=1. Tim's right. > > > It's a better name than inputarray() and we want it for backward > > > compatibility anyway. > > In that case a copy is always made, right? You only want a copy when a > conversion is done, otherwise a reference should be returned. That is the > current behaviour with 0.8 and so does Numeric, if I recall well. I misspoke here. It's correct in CVS. Thanks, Todd > > Peter > > > > ------------------------------------------------------- > The SF.Net email is sponsored by EclipseCon 2004 > Premiere Conference on Open Tools Development and Integration > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > http://www.eclipsecon.org/osdn > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- Todd Miller From jmiller at stsci.edu Wed Feb 11 10:58:35 2004 From: jmiller at stsci.edu (Todd Miller) Date: Wed Feb 11 10:58:35 2004 Subject: [Numpy-discussion] why numarray.inputarray? In-Reply-To: <40294A7C.90903@sympatico.ca> References: <4027D789.2010503@cox.net> <4028EB32.3060904@sympatico.ca> <1076439427.32196.54.camel@halloween.stsci.edu> <40294A7C.90903@sympatico.ca> Message-ID: <1076516610.4574.93.camel@halloween.stsci.edu> On Tue, 2004-02-10 at 16:17, Colin J. Williams wrote: > Todd Miller wrote: > > >On Tue, 2004-02-10 at 09:31, Colin J. Williams wrote: > > > > > >>Tim Hochberg wrote: > >> > >> > >> > >>>I just noticed that the name asarray is considered obsolete and is to > >>>be replaced with inputarray. Why is that? The name asarray has a > >>>pretty clear connection with it's behaviour (return it's argument as > >>>an array), while I can come up with no such connection for inputarray. > >>>The best I can come up with is that it's often used on the inputs of > >>>functions to coerce them to arrays, but that seems a pretty tenuous > >>>connection. Is there a better reason? Am I just being obtuse? > >>> > >>>-tim > >>> > >>> > >>I agree that the term asarray better describes the functionality. > >> > >>One possible use would be to convert an instance of a sub-class of > >>NumArray into an instance of the class NumArray. However, it doesn't do > >>that. > >> > >>It would help if there were a clearer distinction between the functions > >>array and inputarray. The former would appear to cast a wider net and, > >>among other things, include files. > >> > >> > > > >This is a mess, I have to agree. The most important distinction is that > >array() makes a copy of an array by default, while > >inputarray()/asarray() does not. > > > > > > > >>I wonder whether there is a need for both array and input array, > >>wouldn't it be best to focus on the function with the more general utility? > >> > >> > >Based on your input, and Tim Hochberg's earlier comments, I think we > >should: > > > >1. Clean up the array function by ditching the buffer/sequence keyword > >deprecation code. buffer has been deprecated long enough. > > > Good! Although it means some changes in my not fully tested code. I > believe that sequence can also be a buffer, as long as type and shape > are also provided. I haven't yet tried this. > > > > >2. Redefine asarray() as calling array() with copy=1. Tim's right. > >It's a better name than inputarray() and we want it for backward > >compatibility anyway. > > > Good! Although it would be good to make provision for this to convert a > sub-class instance into an instance of NumArray, something like: > if instance(sequence, NumArray): > if sequence.__class__ != NumArray.__class__: > sequence.__class__ = NumArray.__class__ > copy= True I'd like to hear more opinions on this. > > > >3. Redefine inputarray() as a non-recommended synonym for asarray(). > >I'd say deprecate it, but that's a PITA for anyone already using it. > > > > > It wouldn't be that much of a pain if the deprecation were in the > documentation only and put into the code after say a year. Consider it done and get back to me in a year. > > >4. Change the manual to document asarray() as primary and document > >inputarray() as a non-recommended synonym. > > > Yes. Incidentally, what is the current status of the documentation? > The last time I looked, it was for version 0.7. > JC Hsu did a lot of work revising the manual. I just posted the new version on Source Forge as part of the 0.8 file release. Todd -- Todd Miller From jmiller at stsci.edu Wed Feb 11 11:34:29 2004 From: jmiller at stsci.edu (Todd Miller) Date: Wed Feb 11 11:34:29 2004 Subject: [Numpy-discussion] New version of numarray manual Message-ID: <1076516821.4574.98.camel@halloween.stsci.edu> JC Hsu did some work revising the numarray manual. There's a new version of the numarray manual for 0.8 at Source Forge here: http://sourceforge.net/project/showfiles.php?group_id=1369&package_id=32367&release_id=203420 Enjoy! -- Todd Miller From tim.hochberg at cox.net Wed Feb 11 15:26:01 2004 From: tim.hochberg at cox.net (Tim Hochberg) Date: Wed Feb 11 15:26:01 2004 Subject: [Numpy-discussion] numarray.linear_algebra is slow (and a partial fix?) In-Reply-To: <402976F0.1040304@cox.net> References: <402976F0.1040304@cox.net> Message-ID: <402AB86F.6080409@cox.net> An update: A little more tuning resulted in determinant and inverse being about 80x faster than the original numarray code and about 5 times faster than using NumPy for the same test cases I was using before (1000x4x4 matrices). If anyone is interested, let me know and I'll send you the code. -tim Tim Hochberg wrote: > > I discovered that some (all?) of the functions in > numarray.linear_algebra are very slow when operating on small > matrices. In particular, determinant and inverse are both more than 15 > times slower than their NumPy counterparts when operating on 4x4 > matrices. I assume that this is simply a result of numarray's higher > overhead. > > Normally the overhead of numarray is not much of a problem since when > I'm operating on lots of small data chunks I can usually agregate them > into larger chunks and operate on the big chunks. This is, of course, > the standard way to get decent performance in either numarray or > NumPy. However, because the functions in linear_algebra take only > rank-2 (or 1 in some cases) arrays, their is no way to aggregate the > small operations and thus things run quite slow. > > In order to address this I rewrote some of the functions in > linear_algebra to allow an additional, optional, dimension on the > input arrays. Rank-3 arrays are treated as being a set of matrices > that are indexed along the first axis of A. Thus determinant(A) is > essentially equivalent to array(map(determinant, A)) when A is rank-3. > See the attached file for more detail. > > By this trick and by some relentless tuning, I got the numarray > functions to run at about the same speed as their NumPy counterparts > when computing the determinants and inverses of 1000 4x4 matrices. > That's a humungous speedup. > > Is this approach worth pursuing for linear_algebra in general? I'll be > using these myself since I need the speed, although I may back out > some of the more aggresive tuning so I don't get bit if numarray's > internals change. I'll gladly donate this code to numarray if it's > wanted, and I'm willing to help convert the rest, although it probaly > wouldn't happen as fast as this stuff since I don't need it myself > presently. > > -tim > > [Use this with caution at this point -- I just got finished with a > tuning spree and there may well be some bugs] > >------------------------------------------------------------------------ > From jmiller at stsci.edu Thu Feb 12 03:38:25 2004 From: jmiller at stsci.edu (Todd Miller) Date: Thu Feb 12 03:38:25 2004 Subject: [Numpy-discussion] numarray.linear_algebra is slow (and a partial fix?) In-Reply-To: <402AB86F.6080409@cox.net> References: <402976F0.1040304@cox.net> <402AB86F.6080409@cox.net> Message-ID: <1076585530.3742.39.camel@localhost.localdomain> On Wed, 2004-02-11 at 18:19, Tim Hochberg wrote: > An update: > > A little more tuning resulted in determinant and inverse being about 80x > faster than the original numarray code and about 5 times faster than > using NumPy for the same test cases I was using before (1000x4x4 > matrices). If anyone is interested, let me know and I'll send you the code. > > -tim I think we should use the work you've done here in numarray... so we're interested. Unless you object, I'll gleefully include your code as drop-in replacements for the existing routines. Regards, Todd -- Todd Miller From hang.zen at netscape.net Sun Feb 15 16:27:38 2004 From: hang.zen at netscape.net (Hang.zen) Date: Sun Feb 15 16:27:38 2004 Subject: [Numpy-discussion] Attention 13! New i-worm detected. In-Reply-To: References: Message-ID: <54BB9IK12J93L60B@netscape.net> An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: fish.gif Type: image/gif Size: 5077 bytes Desc: not available URL: From DSNZEVLQPWAKE at list.ru Sun Feb 15 16:35:16 2004 From: DSNZEVLQPWAKE at list.ru (Sergio Lassiter) Date: Sun Feb 15 16:35:16 2004 Subject: [Numpy-discussion] Bachelor's Diploma, Master's, or PhD - No Classes Necessary, descriptive gateway blurt canary Message-ID: sarsaparilla tetragonal bateau musicale distaff bayonet bombastic philistine lao thine ascetic trademark berwick agreed longitude cart indecent honda prove carl deformation dunlap algerian buckaroo armour beckman -------------- next part -------------- An HTML attachment was scrubbed... URL: From Jonathan.Peirce at nottingham.ac.uk Tue Feb 17 11:10:06 2004 From: Jonathan.Peirce at nottingham.ac.uk (Jon Peirce) Date: Tue Feb 17 11:10:06 2004 Subject: [Numpy-discussion] numarrays of objects Message-ID: <403265CD.30709@psychology.nottingham.ac.uk> Numeric used to handle arrays of arbitrary python objects. It was handy because you can manipulate the shape of a numeric array much more easily than a list-lof-lists. Has that functionality been lost in numarray? Jon -- Jon Peirce Nottingham University +44 (0)115 8467176 (tel) +44 (0)115 9515324 (fax) http://www.psychology.nottingham.ac.uk/staff/jwp/ From jmiller at stsci.edu Tue Feb 17 11:21:26 2004 From: jmiller at stsci.edu (Todd Miller) Date: Tue Feb 17 11:21:26 2004 Subject: [Numpy-discussion] numarrays of objects In-Reply-To: <403265CD.30709@psychology.nottingham.ac.uk> References: <403265CD.30709@psychology.nottingham.ac.uk> Message-ID: <1077045381.20449.45.camel@halloween.stsci.edu> On Tue, 2004-02-17 at 14:04, Jon Peirce wrote: > Numeric used to handle arrays of arbitrary python objects. It was handy > because you can manipulate the shape of a numeric array much more easily > than a list-lof-lists. > > Has that functionality been lost in numarray? > Jon Try numarray.objects and let me know how it goes. I think numarray.objects has seen very little usage so feedback is welcome. Regards, Todd -- Todd Miller From scott.rifkin at yale.edu Tue Feb 17 12:07:03 2004 From: scott.rifkin at yale.edu (Scott Rifkin) Date: Tue Feb 17 12:07:03 2004 Subject: [Numpy-discussion] real_fft2d Message-ID: I'd like to take the 2d fourier transform of an image, remove some of the lower frequency signal, take the inverse 2d fourier transform, and then reconstruct the image. I'm trying to use the python imaging library to do the image part and FFT for the fourier transform part. When I take the real_fft2d of a matrix (integer valued in this case-UInt16), I get a new matrix of the same size with complex entries. In this implementation of the fourier transform, how is this resulting matrix arranged? There seem to be a few different conventions out there for what, say, entry [0,0] represents. Which one does FFT use? If I want to remove signal from wavelengths longer than a certain interval (say anything over 200 pixels), do I need to process the result from real_fft2d more or can I do the removal directly on that matrix and then take the inverse directly? Is there a good reference for the FFT package or the FFTPACK library out there? Thanks much, Scott Rikin From focke at SLAC.Stanford.EDU Tue Feb 17 16:10:22 2004 From: focke at SLAC.Stanford.EDU (Warren Focke) Date: Tue Feb 17 16:10:22 2004 Subject: [Numpy-discussion] real_fft2d In-Reply-To: References: Message-ID: On Tue, 17 Feb 2004, Scott Rifkin wrote: > I'd like to take the 2d fourier transform of an image, remove some of the > lower frequency signal, take the inverse 2d fourier transform, and then > reconstruct the image. I'm trying to use the python imaging library to do > the image part and FFT for the fourier transform part. > > When I take the real_fft2d of a matrix (integer valued in this > case-UInt16), I get a new matrix of the same size with complex entries. That's surprising. Input of shape (n, m) to real_fft2d should produce output of shape (n, m/2+1). fft2d should produce output of the same shape as the input. > In this implementation of the fourier transform, how is this resulting > matrix arranged? There seem to be a few different conventions out there > for what, say, entry [0,0] represents. Which one does FFT use? Entry [0,0] of the output contains the DC term. If your input is 4x4, your output should be 4x3, with frequencies arrnged thus: [[[ 0,0], [ 0,1], [ 0,2], [ 1,0], [ 1,1], [ 1,2], [ 2,0], [ 2,1], [ 2,2], [-1,0], [-1,1], [-1,2]]] so if you want to zero out frequencies 0 and 1, multiply the transform by a mask like: [[0, 0, 1], [0, 0, 1], [1, 1, 1], [0, 0, 1]] > If I want to remove signal from wavelengths longer than a certain interval > (say anything over 200 pixels), do I need to process the result from > real_fft2d more or can I do the removal directly on that matrix and then > take the inverse directly? That should work. > Is there a good reference for the FFT package or the FFTPACK library out > there? The doc strings may be on the terse side, and you have to read them for several functions to find the answers to all of the questions you ask here, but they are in there. Docs for FFTPACK are at http://www.netlib.org/fftpack/doc Warren Focke From johanenpetra.sajtos at 12move.nl Fri Feb 20 01:05:09 2004 From: johanenpetra.sajtos at 12move.nl (Johanenpetra.sajtos) Date: Fri Feb 20 01:05:09 2004 Subject: [Numpy-discussion] Report 88722! In-Reply-To: References: Message-ID: An HTML attachment was scrubbed... URL: From thli at bioinformaticsSolutions.com Fri Feb 20 07:59:23 2004 From: thli at bioinformaticsSolutions.com (Tina Li) Date: Fri Feb 20 07:59:23 2004 Subject: [Numpy-discussion] "GC object already in linked list" error when testall.test() Message-ID: Hello, I installed numarray on a RedHat 8.0 with Python 2.2.1. The build completed fine, but it failed when I tried to testall.test(). Here is the output: ------------------------------------------------------------ Python 2.2.1 (#1, Aug 30 2002, 12:15:30) [GCC 3.2 20020822 (Red Hat Linux Rawhide 3.2-4)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import numarray.testall as testall >>> testall.test() numarray: ((0, 1142), (0, 1142)) numarray.records: (0, 48) numarray.strings: (0, 168) numarray.memmap: (0, 82) numarray.objects: (0, 68) numarray.memorytest: (0, 16) Fatal Python error: GC object already in linked list Aborted ------------------------------------------------------------ Seems like a garbage collector problem. That's all I can tell. I installed using "python setup.py install --gencode --prefix=" and then "export PYTHONPATH". Does anyone have a clue how this can be fixed? Thanks! Tina From jmiller at stsci.edu Fri Feb 20 09:01:50 2004 From: jmiller at stsci.edu (Todd Miller) Date: Fri Feb 20 09:01:50 2004 Subject: [Numpy-discussion] "GC object already in linked list" error when testall.test() In-Reply-To: References: Message-ID: <1077294813.19814.80.camel@halloween.stsci.edu> On Fri, 2004-02-20 at 10:49, Tina Li wrote: > Hello, > > I installed numarray on a RedHat 8.0 with Python 2.2.1. The build completed > fine, but it failed when I tried to testall.test(). Here is the output: > > ------------------------------------------------------------ > Python 2.2.1 (#1, Aug 30 2002, 12:15:30) > [GCC 3.2 20020822 (Red Hat Linux Rawhide 3.2-4)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>> import numarray.testall as testall > >>> testall.test() > numarray: ((0, 1142), (0, 1142)) > numarray.records: (0, 48) > numarray.strings: (0, 168) > numarray.memmap: (0, 82) > numarray.objects: (0, 68) > numarray.memorytest: (0, 16) > Fatal Python error: GC object already in linked list > Aborted > ------------------------------------------------------------ > > Seems like a garbage collector problem. That's all I can tell. I installed > using "python setup.py install --gencode --prefix=" > and then "export PYTHONPATH". Does anyone have a clue how this can be fixed? Python-2.2.3 and Python-2.3.3 both (seem to) work fine, so I suggest upgrading to one of those. I've had no problem reproducing this bug, but solving it is another matter. Regards, Todd -- Todd Miller From simon at arrowtheory.com Sun Feb 22 15:25:22 2004 From: simon at arrowtheory.com (Simon Burton) Date: Sun Feb 22 15:25:22 2004 Subject: [Numpy-discussion] copy4bytes: access beyond buffer Message-ID: <20040223102753.6c7ae83f.simon@arrowtheory.com> I posted this bug to the sf bugs page, but it seems to have disappeared. The bug remains. I've been using the CVS version. Is there anything I'm doing wrong? Python 2.2.3 (#4, Feb 15 2004, 17:40:28) [GCC 3.2 20020903 (Red Hat Linux 8.0 3.2-7)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import numarray >>> zeros = numarray.zeros( shape = (2048,0) ) >>> zeros = zeros.astype( numarray.Int16 ) Traceback (most recent call last): File "", line 1, in ? File "/usr/local/lib/python2.2/site-packages/numarray/numarraycore.py", line 594, in astype ufunc._copyFromAndConvert(self, retarr) libnumarray.error: copy4bytes: access beyond buffer. offset=3 buffersize=0 >>> I also get this error in other situations. Simon. -- Simon Burton, B.Sc. Licensed PO Box 8066 ANU Canberra 2601 Australia Ph. 61 02 6249 6940 http://arrowtheory.com From simon at arrowtheory.com Sun Feb 22 15:48:40 2004 From: simon at arrowtheory.com (Simon Burton) Date: Sun Feb 22 15:48:40 2004 Subject: [Numpy-discussion] copy4bytes: access beyond buffer In-Reply-To: <20040223102753.6c7ae83f.simon@arrowtheory.com> References: <20040223102753.6c7ae83f.simon@arrowtheory.com> Message-ID: <20040223105124.27ec6818.simon@arrowtheory.com> On Mon, 23 Feb 2004 10:27:53 +1100 Simon Burton wrote: > I posted this bug to the sf bugs page, but it seems to have disappeared. > The bug remains. I've been using the CVS version. Is there anything I'm doing wrong? > OK, i think i see it now. shape=(2048,0) should be shape=(2048,). I have been using the C-API aswell, and got confused. Maybe there is a better exception to raise? Is it ever useful to have a shape=(2048,0) ? Simon. -- Simon Burton, B.Sc. Licensed PO Box 8066 ANU Canberra 2601 Australia Ph. 61 02 6249 6940 http://arrowtheory.com From jmiller at stsci.edu Mon Feb 23 05:46:57 2004 From: jmiller at stsci.edu (Todd Miller) Date: Mon Feb 23 05:46:57 2004 Subject: [Numpy-discussion] copy4bytes: access beyond buffer In-Reply-To: <20040223102753.6c7ae83f.simon@arrowtheory.com> References: <20040223102753.6c7ae83f.simon@arrowtheory.com> Message-ID: <1077543437.5211.4.camel@halloween.stsci.edu> On Sun, 2004-02-22 at 18:27, Simon Burton wrote: > I posted this bug to the sf bugs page, but it seems to have disappeared. numarray bugs are tracked separately from Numeric bugs. Since Numeric was the original project, Numeric "owns" the most easily visible tracker, "Bugs". numarray bugs are relocated to a less visible tracker called "Numarray Bugs" which is accessible under the Tracker menu item, just to the left of "Bugs". > The bug remains. I've been using the CVS version. Is there anything I'm doing wrong? > > Python 2.2.3 (#4, Feb 15 2004, 17:40:28) > [GCC 3.2 20020903 (Red Hat Linux 8.0 3.2-7)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>> import numarray > >>> zeros = numarray.zeros( shape = (2048,0) ) > >>> zeros = zeros.astype( numarray.Int16 ) > Traceback (most recent call last): > File "", line 1, in ? > File "/usr/local/lib/python2.2/site-packages/numarray/numarraycore.py", line 594, in astype > ufunc._copyFromAndConvert(self, retarr) > libnumarray.error: copy4bytes: access beyond buffer. offset=3 buffersize=0 > >>> > You're not doing anything wrong. You're hitting an edge case which isn't well supported in numarray yet: zero element arrays. > I also get this error in other situations. > > Simon. -- Todd Miller From jmiller at stsci.edu Mon Feb 23 05:50:20 2004 From: jmiller at stsci.edu (Todd Miller) Date: Mon Feb 23 05:50:20 2004 Subject: [Numpy-discussion] copy4bytes: access beyond buffer In-Reply-To: <20040223105124.27ec6818.simon@arrowtheory.com> References: <20040223102753.6c7ae83f.simon@arrowtheory.com> <20040223105124.27ec6818.simon@arrowtheory.com> Message-ID: <1077543568.5211.8.camel@halloween.stsci.edu> On Sun, 2004-02-22 at 18:51, Simon Burton wrote: > On Mon, 23 Feb 2004 10:27:53 +1100 > Simon Burton wrote: > > > I posted this bug to the sf bugs page, but it seems to have disappeared. > > The bug remains. I've been using the CVS version. Is there anything I'm doing wrong? > > > > OK, i think i see it now. shape=(2048,0) should be shape=(2048,). I have been using the C-API > aswell, and got confused. > > Maybe there is a better exception to raise? Is it ever useful to have a shape=(2048,0) ? IMHO, it's an edge case which might come up occasionally on arrays which are intended to be resized. Eventually I intend to fix it, but making it work immediately isn't a priority. > > Simon. -- Todd Miller From antti.korvenoja at helsinki.fi Mon Feb 23 23:50:00 2004 From: antti.korvenoja at helsinki.fi (Antti Korvenoja) Date: Mon Feb 23 23:50:00 2004 Subject: [Numpy-discussion] User arrays in numarray Message-ID: <1077608892.403b01bc446c8@www2.helsinki.fi> Dear numarray developers, What would be the recommended class to inherit from if one wants to derive a new class from numarray array? In other words, what would be similar to UserArray in Numeric (NDArray, NumArray)? -Antti Korvenoja From danzka.danzka at verizon.net Tue Feb 24 00:28:22 2004 From: danzka.danzka at verizon.net (Danzka.danzka) Date: Tue Feb 24 00:28:22 2004 Subject: [Numpy-discussion] I F.-.c.k yor mother :) he-he In-Reply-To: References: Message-ID: <0L70E88FAJJ2EIA7@verizon.net> An HTML attachment was scrubbed... URL: From pearu at cens.ioc.ee Tue Feb 24 02:04:00 2004 From: pearu at cens.ioc.ee (Pearu Peterson) Date: Tue Feb 24 02:04:00 2004 Subject: [Numpy-discussion] ANN: F2PY - Fortran to Python Interface Generator Message-ID: F2PY - Fortran to Python Interface Generator -------------------------------------------- I am pleased to announce the seventh public release of F2PY, version 2.39.235_1642. The purpose of the F2PY project is to provide the connection between Python and Fortran programming languages. For more information, see http://cens.ioc.ee/projects/f2py2e/ Download source: http://cens.ioc.ee/projects/f2py2e/2.x/F2PY-2-latest.tar.gz What's new? ------------ Since the last public release, that was more than two years ago, F2PY project has been actively maintained and many useful features have been implemented. Here follows some highlights: * New statement ``usercode`` allows inserting user defined C code to F2PY generated extension module sources at various relevant places: before wrapper functions, after variable declarations, and to the end of initialization function of the module. This makes using F2PY very flexible. * New statement ``pymethoddef`` allows adding items to PyMethodDef-array. * Full support for character arrays and arrays of strings is finally implemented. * Number of feature requests from users are implemented. For example: - the auxiliary ``as_column_major_storage()`` that efficiently converts array to column storage order - the F2PY_REPORT_ON_ARRAY_COPY macro that when defined sends a message to stderr when copy of an array with the size larger than specified threshold is made. - Numarray support, thanks to Todd Miller. * Support for Win32 and Mac OSX platforms is considerably improved. The list of compilers supported by the scipy_distutils package is longer than ever: GNU Fortran Compiler Portland Group Fortran Compiler Absoft Corp Fortran Compiler MIPSpro Fortran Compiler Sun|Forte Fortran 95 Compiler Intel Fortran Compiler for 32-bit apps Intel Visual Fortran Compiler for 32-bit apps Intel Fortran Compiler for Itanium apps NAGWare Fortran 95 Compiler Compaq Fortran Compiler DIGITAL|Compaq Visual Fortran Compiler Pacific-Sierra Research Fortran 90 Compiler HP Fortran 90 Compiler Lahey/Fujitsu Fortran 95 Compiler IBM XL Fortran Compiler * Numerous bugs are fixed. You should definitely update F2PY when using complex input arrays, there was a nasty bug that in certain cases caused incorrect results. * F2PY has now a man page and its documentation is kept up to date. Many other improvements to F2PY algorithm and usage are implemented, see HISTORY.txt for more details. Enjoy, Pearu Peterson ---------------

F2PY 2.39.235_1642 - The Fortran to Python Interface Generator (24-Feb-04) From lawrencestiman at mail.bulgaria.com Tue Feb 24 04:27:02 2004 From: lawrencestiman at mail.bulgaria.com (Marion Hubbard) Date: Tue Feb 24 04:27:02 2004 Subject: [Numpy-discussion] PROTECT Your PRIVACY, Block Annoying Pop-ups, Cut Down Employee Waste, Tue, 24 Feb 2004 07:25:22 -0500 Message-ID: Tue, 24 Feb 2004 07:25:22 -0500 PRODUCT PARADISE - Protect Your Privacy - Cut Down On Employee Waste - Block Annoying Pop-ups - All with state-of-the-art, affordable software from Product Paradise. Enter Paradise Now: http://click.com-click.com.ph/click.php?id=prcoomi&ID=11 Safe Surfer Pro NEW v2.0 Safe Surfer Pro protects your privacy by erasing your Internet records from your computer and is simply the best internet eraser available! Not only can it clean your internet tracks, but it will erase your internet tracks as you surf. Feel more secure and clean as you surf. Was 49.95 - Now just 39.95. Enter Paradise Now: http://click.com-click.com.ph/click.php?id=prcoomi&ID=11 Secret Server Pro Would you like to find out if your kids are browsing the web where they shouldn't be? Or do you need to know if your employees are wasting your time? This product also enables you to record instant messaging communication done from your computer. Now just 39.95. Enter Paradise Now: http://click.com-click.com.ph/click.php?id=prcoomi&ID=11 Popup Killer Popup Killer blocks annoying popups that slow down and destroy your surfing experience. Popup Killer installs in seconds and blocks all popups with an easy to use and customizable interface. By constantly eliminating popups, you will enjoy a faster web experience and not have to continuously close those annoying extra windows. Just 29.95. Enter Paradise Now: http://click.com-click.com.ph/click.php?id=prcoomi&ID=11 Or GET THE BEST DEAL 3 programs for the price of less than 2 - Only 74.95 Enter Paradise Now: http://click.com-click.com.ph/click.php?id=prcoomi&ID=11 No more ads: http://www.productparadise.co.om/nomoreads.html blumenthal bulb meditate repertory dayton luxuriate expect honesty maintain addressee spacious industry lavabo acts asunder siegel rotor forborne gina anaglyph benign buses tagging malocclusion classroom colossal deteriorate odorous clot thunderous amadeus From ariciputi at pito.com Tue Feb 24 13:12:06 2004 From: ariciputi at pito.com (Andrea Riciputi) Date: Tue Feb 24 13:12:06 2004 Subject: [Numpy-discussion] Deleting arrays. Message-ID: Hi, I'm running a simulation program partly written in C partly in Python and I use Numeric arrays extensively. At the end of a single run I have a bunch of Numeric arrays containing partial results that I don't need anymore. These arrays have been created on the C side of my code and then copied (actually linked via PyArray_FromDimsAndData) to Numeric arrays. How can I release these arrays and get back memory in order to be able to start the next run? Thanks in advance, Andrea. --- Andrea Riciputi "Science is like sex: sometimes something useful comes out, but that is not the reason we are doing it" -- (Richard Feynman) From marco at bubke.de Tue Feb 24 13:39:00 2004 From: marco at bubke.de (Marco Bubke) Date: Tue Feb 24 13:39:00 2004 Subject: [Numpy-discussion] NA_updateDataPtr segmentation fault =?iso-8859-1?q?bug=3F=0A?= Message-ID: <200402242237.00852.marco@bubke.de> Hi I have a little function in pyrex: cdef NumArray array_to_float(NumArray array): ??#?maybe?here?is?memoty?leak! ??cdef?NumArray?flat_array ??cdef?NumArray?new_array ??print?'mark?1' ??new_array?=?array.astype('Float32') ??print?new_array ??print?'mark?2' ??NA_updateDataPtr(new_array) ??print?'mark?3'??? ??NA_updateDataPtr(new_array) ??print?'mark?4'?? ??flat_array?=?NA_InputArray(new_array,?tFloat32,?NUM_C_ARRAY) ??print?'mark?5'?? ??return?flat_array After the second NA_updateDataPtr I get a segmentation fault> Otherwise I get it with the NA_InputArray. It goes down to getReadBufferDataPtr and than to rval = buff2->ob_type->tp_as_buffer->bf_getreadbuffer(buff2, 0, buff); I believe bf_getreadbuffer is memory_getbuf but I'm not sure. I don't the bug but its happen. Can somebody help me? thanks and regards Marco From hinsen at cnrs-orleans.fr Wed Feb 25 02:53:01 2004 From: hinsen at cnrs-orleans.fr (Konrad Hinsen) Date: Wed Feb 25 02:53:01 2004 Subject: [Numpy-discussion] Deleting arrays. In-Reply-To: References: Message-ID: <200402251149.01279.hinsen@cnrs-orleans.fr> On Tuesday 24 February 2004 22:10, Andrea Riciputi wrote: > anymore. These arrays have been created on the C side of my code and > then copied (actually linked via PyArray_FromDimsAndData) to Numeric > arrays. How can I release these arrays and get back memory in order to > be able to start the next run? I am not aware of any clean way. A hack that might help you out is array->flags |= OWN_DATA; The data will then be freed when the array is released. However, there are a few caveats: 1) This is not documented. 2) It works only if the data was allocated by the same memory manager that is used by NumPy. 3) You have to make sure that no other code keeps a reference to the data. If modifying the C code is an option, then it is probably the better one. Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen at cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais ------------------------------------------------------------------------------- From cookedm at physics.mcmaster.ca Wed Feb 25 02:58:02 2004 From: cookedm at physics.mcmaster.ca (David M. Cooke) Date: Wed Feb 25 02:58:02 2004 Subject: [Numpy-discussion] Deleting arrays. In-Reply-To: References: Message-ID: <20040225105412.GA23702@arbutus.physics.mcmaster.ca> On Tue, Feb 24, 2004 at 10:10:51PM +0100, Andrea Riciputi wrote: > Hi, > I'm running a simulation program partly written in C partly in Python > and I use Numeric arrays extensively. At the end of a single run I have > a bunch of Numeric arrays containing partial results that I don't need > anymore. These arrays have been created on the C side of my code and > then copied (actually linked via PyArray_FromDimsAndData) to Numeric > arrays. How can I release these arrays and get back memory in order to > be able to start the next run? Simple answer: don't use PyArray_FromDimsAndData. Allocate your array with Python using PyArray_FromDims, and use the data element for your C array, instead of allocating using C and convincing Python to use that. However, there are instances where you can't do that (the memory was allocated by some library, for instance, that needs a custom deallocator, or is from another array type). Then I'd suggest 1) copying the data (depends on how much there is) 2) if the array memory was originally allocated using malloc, you set the OWNDATA flag so Python will free it (going from memory here on the name; check the source). Note that you should be really sure you and Python are using the same allocator/deallocator routines. 3) or using the following trick, best described with some code: MyOtherArrayType *A = a_library_routine_that_allocates_a_routine(); PyObject *oA = PyArray_FromDimsAndData(rank, dims, PyArray_DOUBLE, (char *)A->array_data); PyObject *w = PyCObject_FromVoidPtr(A, free_myotherarraytype); ((PyArrayObject *)oA)->base = w; Now, when oA is DECREF'd latter, oA->base will also be DECREF'd. Since it's a PyCObject, free_myotherarraytype(A) will be called, free'ing the data. Since PyArray_FromDimsAndData was used, Python won't try to deallocate oA->data. I haven't checked whether this works under numarray (and I'm sure I saw a better way to do it there). I have used it for a Numeric<->Blitz++ wrapper. -- |>|\/|< /--------------------------------------------------------------------------\ |David M. Cooke http://arbutus.physics.mcmaster.ca/dmc/ |cookedm at physics.mcmaster.ca From jmiller at stsci.edu Wed Feb 25 05:58:03 2004 From: jmiller at stsci.edu (Todd Miller) Date: Wed Feb 25 05:58:03 2004 Subject: [Numpy-discussion] User arrays in numarray In-Reply-To: <1077608892.403b01bc446c8@www2.helsinki.fi> References: <1077608892.403b01bc446c8@www2.helsinki.fi> Message-ID: <1077717352.11623.3.camel@halloween.stsci.edu> On Tue, 2004-02-24 at 02:48, Antti Korvenoja wrote: > Dear numarray developers, > > What would be the recommended class to inherit from if one wants to derive a > new class from numarray array? In other words, what would be similar to > UserArray in Numeric (NDArray, NumArray)? Try subclassing NumArray. You might want to look at PyMatrix to see a practical example of how it can be done. I should forewarn you that although numarray uses new style classes, it's sub-classibility is less than perfect. Todd -- Todd Miller From jmiller at stsci.edu Wed Feb 25 06:05:05 2004 From: jmiller at stsci.edu (Todd Miller) Date: Wed Feb 25 06:05:05 2004 Subject: [Numpy-discussion] NA_updateDataPtr segmentation fault bug? In-Reply-To: <200402242237.00852.marco@bubke.de> References: <200402242237.00852.marco@bubke.de> Message-ID: <1077717801.11623.6.camel@halloween.stsci.edu> On Tue, 2004-02-24 at 16:37, Marco Bubke wrote: > Hi > > I have a little function in pyrex: > > cdef NumArray array_to_float(NumArray array): > # maybe here is memoty leak! > cdef NumArray flat_array > cdef NumArray new_array > print 'mark 1' > new_array = array.astype('Float32') > print new_array > print 'mark 2' > NA_updateDataPtr(new_array) > print 'mark 3' > NA_updateDataPtr(new_array) > print 'mark 4' > flat_array = NA_InputArray(new_array, tFloat32, NUM_C_ARRAY) > print 'mark 5' > return flat_array > > After the second NA_updateDataPtr I get a segmentation fault> Otherwise I > get it with the NA_InputArray. It goes down to getReadBufferDataPtr and > than to > rval = buff2->ob_type->tp_as_buffer->bf_getreadbuffer(buff2, 0, buff); > > I believe bf_getreadbuffer is memory_getbuf but I'm not sure. I don't the > bug but its happen. > > Can somebody help me? Maybe you could post the pyrex generated C-code and we could take a look at that? Regards, Todd > > thanks and regards > > Marco > > > ------------------------------------------------------- > SF.Net is sponsored by: Speed Start Your Linux Apps Now. > Build and deploy apps & Web services for Linux with > a free DVD software kit from IBM. Click Now! > http://ads.osdn.com/?ad_id56&alloc_id438&op=click > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- Todd Miller From ariciputi at pito.com Wed Feb 25 07:20:04 2004 From: ariciputi at pito.com (Andrea Riciputi) Date: Wed Feb 25 07:20:04 2004 Subject: [Numpy-discussion] Deleting arrays. In-Reply-To: <200402251149.01279.hinsen@cnrs-orleans.fr> References: <200402251149.01279.hinsen@cnrs-orleans.fr> Message-ID: It could be, but how? On 25 Feb 2004, at 11:49, Konrad Hinsen wrote: > If modifying the C code is an option, then it is probably the better > one. > --- Andrea Riciputi "Science is like sex: sometimes something useful comes out, but that is not the reason we are doing it" -- (Richard Feynman) From marco at bubke.de Wed Feb 25 08:36:11 2004 From: marco at bubke.de (Marco Bubke) Date: Wed Feb 25 08:36:11 2004 Subject: [Numpy-discussion] RE: NA_updateDataPtr segmentation fault bug? Message-ID: <200402251734.29286.marco@bubke.de> Ok cdef NumArray array_to_float(NumArray array): cdef NumArray new_array new_array = array NA_updateDataPtr(new_array) NA_updateDataPtr(new_array) flat_array = NA_InputArray(array, tFloat32, NUM_C_ARRAY) return flat_array Here the C code: static PyArrayObject *__pyx_f_2gl_array_to_float(PyArrayObject *__pyx_v_array) { PyArrayObject *__pyx_v_new_array; PyObject *__pyx_v_flat_array; PyArrayObject *__pyx_r; PyObject *__pyx_1 = 0; Py_INCREF(__pyx_v_array); ((PyObject*)__pyx_v_new_array) = Py_None; Py_INCREF(((PyObject*)__pyx_v_new_array)); __pyx_v_flat_array = Py_None; Py_INCREF(__pyx_v_flat_array); /* "/home/marco/projects/calliopeia/numarray.pxd":102 */ Py_INCREF(((PyObject *)__pyx_v_array)); Py_DECREF(((PyObject *)__pyx_v_new_array)); ((PyObject *)__pyx_v_new_array) = ((PyObject *)__pyx_v_array); /* "/home/marco/projects/calliopeia/numarray.pxd":103 */ __pyx_1 = NA_updateDataPtr(((PyObject *)__pyx_v_new_array)); if (!__pyx_1) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 103; goto __pyx_L 1;} Py_DECREF(__pyx_1); __pyx_1 = 0; /* "/home/marco/projects/calliopeia/numarray.pxd":104 */ __pyx_1 = NA_updateDataPtr(((PyObject *)__pyx_v_new_array)); if (!__pyx_1) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 104; goto __pyx_L 1;} Py_DECREF(__pyx_1); __pyx_1 = 0; /* "/home/marco/projects/calliopeia/numarray.pxd":105 */ __pyx_1 = NA_InputArray(((PyObject *)__pyx_v_array),tFloat32,NUM_C_ARRAY); if (!__pyx_1) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 105 ; goto __pyx_L1;} Py_DECREF(__pyx_v_flat_array); __pyx_v_flat_array = __pyx_1; __pyx_1 = 0; /* "/home/marco/projects/calliopeia/numarray.pxd":106 */ if (!__Pyx_TypeTest(__pyx_v_flat_array, __pyx_ptype_2gl__numarray)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 106; goto __pyx_L1;} Py_INCREF(__pyx_v_flat_array); __pyx_r = (PyArrayObject *)__pyx_v_flat_array; goto __pyx_L0; (PyObject *)__pyx_r = Py_None; Py_INCREF((PyObject *)__pyx_r); goto __pyx_L0; __pyx_L1:; Py_XDECREF(__pyx_1); __Pyx_AddTraceback("gl.array_to_float"); __pyx_r = 0; __pyx_L0:; Py_DECREF(__pyx_v_new_array); Py_DECREF(__pyx_v_flat_array); Py_DECREF(__pyx_v_array); return ((PyArrayObject *)__pyx_r) ; } From hinsen at cnrs-orleans.fr Wed Feb 25 08:37:10 2004 From: hinsen at cnrs-orleans.fr (Konrad Hinsen) Date: Wed Feb 25 08:37:10 2004 Subject: [Numpy-discussion] Deleting arrays. In-Reply-To: References: <200402251149.01279.hinsen@cnrs-orleans.fr> Message-ID: <200402251732.49134.hinsen@cnrs-orleans.fr> On Wednesday 25 February 2004 16:18, Andrea Riciputi wrote: > It could be, but how? If you don't mind making it Python-specific, you can have it create NumPy arrays directly and return them to Python. If you do want to keep it "Python clean", then modify it in such a way that it doesn't allocate any memory, but takes the data space of the array as input. Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen at cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais ------------------------------------------------------------------------------- From ariciputi at pito.com Wed Feb 25 08:50:04 2004 From: ariciputi at pito.com (Andrea Riciputi) Date: Wed Feb 25 08:50:04 2004 Subject: [Numpy-discussion] Deleting arrays. In-Reply-To: <200402251732.49134.hinsen@cnrs-orleans.fr> References: <200402251149.01279.hinsen@cnrs-orleans.fr> <200402251732.49134.hinsen@cnrs-orleans.fr> Message-ID: <59DC2676-67B2-11D8-87E7-000393933E4E@pito.com> I saw but it's too late. At this point I can't modify my code so deeply, but I'll keep your suggestion for the next project. Just for sake of curiosity, working with C++ arrays (I mean those defined in STL) would have simpliefied things? A. On 25 Feb 2004, at 17:32, Konrad Hinsen wrote: > If you don't mind making it Python-specific, you can have it create > NumPy > arrays directly and return them to Python. If you do want to keep it > "Python > clean", then modify it in such a way that it doesn't allocate any > memory, but > takes the data space of the array as input. > > Konrad. > --- Andrea Riciputi "Science is like sex: sometimes something useful comes out, but that is not the reason we are doing it" -- (Richard Feynman) From jmiller at stsci.edu Wed Feb 25 09:36:12 2004 From: jmiller at stsci.edu (Todd Miller) Date: Wed Feb 25 09:36:12 2004 Subject: [Numpy-discussion] RE: NA_updateDataPtr segmentation fault bug? In-Reply-To: <200402251734.29286.marco@bubke.de> References: <200402251734.29286.marco@bubke.de> Message-ID: <1077730429.11623.26.camel@halloween.stsci.edu> On Wed, 2004-02-25 at 11:34, Marco Bubke wrote: > Ok > > cdef NumArray array_to_float(NumArray array): > cdef NumArray new_array > new_array = array > NA_updateDataPtr(new_array) > NA_updateDataPtr(new_array) > flat_array = NA_InputArray(array, tFloat32, NUM_C_ARRAY) > return flat_array > > Here the C code: > > static PyArrayObject *__pyx_f_2gl_array_to_float(PyArrayObject *__pyx_v_array) > { > PyArrayObject *__pyx_v_new_array; > PyObject *__pyx_v_flat_array; > PyArrayObject *__pyx_r; > PyObject *__pyx_1 = 0; > Py_INCREF(__pyx_v_array); > ((PyObject*)__pyx_v_new_array) = Py_None; > Py_INCREF(((PyObject*)__pyx_v_new_array)); > __pyx_v_flat_array = Py_None; Py_INCREF(__pyx_v_flat_array); > > /* "/home/marco/projects/calliopeia/numarray.pxd":102 */ > Py_INCREF(((PyObject *)__pyx_v_array)); > Py_DECREF(((PyObject *)__pyx_v_new_array)); > ((PyObject *)__pyx_v_new_array) = ((PyObject *)__pyx_v_array); > > /* "/home/marco/projects/calliopeia/numarray.pxd":103 */ > __pyx_1 = NA_updateDataPtr(((PyObject *)__pyx_v_new_array)); if (!__pyx_1) > {__pyx_filename = __pyx_f[0]; __pyx_lineno = 103; goto __pyx_L > 1;} > Py_DECREF(__pyx_1); __pyx_1 = 0; One problem is right here: NA_updateDataPtr does not return a new reference. Thus, _pyx_v_new_array/_pyx_1 is losing references right here. > > /* "/home/marco/projects/calliopeia/numarray.pxd":104 */ > __pyx_1 = NA_updateDataPtr(((PyObject *)__pyx_v_new_array)); if (!__pyx_1) > {__pyx_filename = __pyx_f[0]; __pyx_lineno = 104; goto __pyx_L > 1;} > Py_DECREF(__pyx_1); __pyx_1 = 0; And of course it's also losing them here as well. I haven't finished analyzing the rest of the code. If that fix doesn't solve the problem, let me know. Regards, Todd > > /* "/home/marco/projects/calliopeia/numarray.pxd":105 */ > __pyx_1 = NA_InputArray(((PyObject *)__pyx_v_array),tFloat32,NUM_C_ARRAY); > if (!__pyx_1) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 105 > ; goto __pyx_L1;} > Py_DECREF(__pyx_v_flat_array); > __pyx_v_flat_array = __pyx_1; > __pyx_1 = 0; > > /* "/home/marco/projects/calliopeia/numarray.pxd":106 */ > if (!__Pyx_TypeTest(__pyx_v_flat_array, __pyx_ptype_2gl__numarray)) > {__pyx_filename = __pyx_f[0]; __pyx_lineno = 106; goto __pyx_L1;} > Py_INCREF(__pyx_v_flat_array); > __pyx_r = (PyArrayObject *)__pyx_v_flat_array; > goto __pyx_L0; > > (PyObject *)__pyx_r = Py_None; Py_INCREF((PyObject *)__pyx_r); > goto __pyx_L0; > __pyx_L1:; > Py_XDECREF(__pyx_1); > __Pyx_AddTraceback("gl.array_to_float"); > __pyx_r = 0; > __pyx_L0:; > Py_DECREF(__pyx_v_new_array); > Py_DECREF(__pyx_v_flat_array); > Py_DECREF(__pyx_v_array); > return ((PyArrayObject *)__pyx_r) ; > } > > > > > ------------------------------------------------------- > SF.Net is sponsored by: Speed Start Your Linux Apps Now. > Build and deploy apps & Web services for Linux with > a free DVD software kit from IBM. Click Now! > http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- Todd Miller From Luc_J_Bourhis at mac.com Wed Feb 25 12:50:06 2004 From: Luc_J_Bourhis at mac.com (Luc Bourhis) Date: Wed Feb 25 12:50:06 2004 Subject: [Numpy-discussion] numarray, BLAS, LAPACK and MacOS X Message-ID: <9B57FC18-67D3-11D8-A09B-000393803558@mac.com> MacOS X (at least 10.2 and 10.3) comes with BLAS and LAPACK already installed. However the hooks in addons.py (i.e. setting the environment variable USE_LAPACK and then filling the lists lapack_libs and lapack_dirs) don't work because it will typically pass the arguments -llapack, -lblas to the linker although Apple does not supply any liblapack.a or libblas.a. Instead Apple uses the standard MacOS X distribution scheme: a framework (vecLib is the name). So the correct way to compile numarray with the native blas and lapack on MacOS X is to fix the addons.py as follow: Extension('numarray.linear_algebra.lapack_lite2', sourcelist, include_dirs = ["Packages/LinearAlgebra2/Src", 'Include/numarray'], extra_objects = ["-bundle", "-framework", "vecLib",]), I checked it worked (but the -bundle may not be necessary) HtH somebody, -- Luc Bourhis Chemical Crystallography Laboratory University of Durham From thli at bioinformaticsSolutions.com Wed Feb 25 13:47:03 2004 From: thli at bioinformaticsSolutions.com (Tina Li) Date: Wed Feb 25 13:47:03 2004 Subject: [Numpy-discussion] ImportError: undefined symbol: PyFloat_Type in _conv.so In-Reply-To: Message-ID: Hello, I'm using numarray on a RH 7.0 with Python 2.3. In my application I import numarray like this: from numarray import * and it complains: ----------------------------- File "/home/thli/pymol/ext/lib/python2.3/site-packages/numarray/__init__.py", line 11, in ? from numarrayall import * File "/home/thli/pymol/ext/lib/python2.3/site-packages/numarray/numarrayall.py ", line 1, in ? from numerictypes import * File "/home/thli/pymol/ext/lib/python2.3/site-packages/numarray/numerictypes.p y", line 33, in ? from typeconv import typeConverters as _typeConverters File "/home/thli/pymol/ext/lib/python2.3/site-packages/numarray/typeconv.py", line 6, in ? import _conv ImportError: undefined symbol: PyFloat_Type ----------------------------- So where should I start looking to fix it? Thanks! Tina From ariciputi at pito.com Thu Feb 26 00:28:11 2004 From: ariciputi at pito.com (Andrea Riciputi) Date: Thu Feb 26 00:28:11 2004 Subject: [Numpy-discussion] numarray, BLAS, LAPACK and MacOS X In-Reply-To: <9B57FC18-67D3-11D8-A09B-000393803558@mac.com> References: <9B57FC18-67D3-11D8-A09B-000393803558@mac.com> Message-ID: <66D9E362-6835-11D8-B547-000393933E4E@pito.com> Or you can install Numeric (and many others) via Fink without any effort. Look at its home page (), I'm sure you can find it very interesting. Cheers, Andrea. On 25 Feb 2004, at 21:45, Luc Bourhis wrote: > MacOS X (at least 10.2 and 10.3) comes with BLAS and LAPACK already > installed. However the hooks in addons.py (i.e. setting the > environment variable USE_LAPACK and then filling the lists lapack_libs > and lapack_dirs) don't work because it will typically pass the > arguments -llapack, -lblas to the linker although Apple does not > supply any liblapack.a or libblas.a. Instead Apple uses the standard > MacOS X distribution scheme: a framework (vecLib is the name). So the > correct way to compile numarray with the native blas and lapack on > MacOS X is to fix the addons.py as follow: > > Extension('numarray.linear_algebra.lapack_lite2', > sourcelist, > include_dirs = ["Packages/LinearAlgebra2/Src", > 'Include/numarray'], > extra_objects = ["-bundle", "-framework", "vecLib",]), > > I checked it worked (but the -bundle may not be necessary) > > HtH somebody, > -- > Luc Bourhis > Chemical Crystallography Laboratory > University of Durham --- Andrea Riciputi "Science is like sex: sometimes something useful comes out, but that is not the reason we are doing it" -- (Richard Feynman) From rob at hooft.net Sun Feb 29 00:13:02 2004 From: rob at hooft.net (Rob Hooft) Date: Sun Feb 29 00:13:02 2004 Subject: [Numpy-discussion] Garbage collection fails after numarray exception In-Reply-To: <1076515358.4574.72.camel@halloween.stsci.edu> References: <20040210200957.GA20294@falcon.pds.twi.tudelft.nl> <1076515358.4574.72.camel@halloween.stsci.edu> Message-ID: <40419D16.1060206@hooft.net> Todd Miller wrote: > On Tue, 2004-02-10 at 15:09, Leo Breebaart wrote: > >>Hi all, >> >>I'm not sure if I have found a bug, or simply a type of behaviour >>that should not have surprised me -- but I most definitely did >>not expect the following numarray 0.8 behaviour: >> >> >> >>> import numarray >> >>> a = numarray.zeros(100000000) >> >>> del a >> >>> a = numarray.zeros(100000000) >> >>> a + '' >> Traceback (most recent call last): >> File "", line 1, in ? >> File "/usr/lib/python2.3/site-packages/numarray/numarraycore.py", line 664, in __add__ >> return ufunc.add(self, operand) >> File "/usr/lib/python2.3/site-packages/numarray/ufunc.py", line 870, in _cache_miss2 >> (in1, in2), insig, scalar = _inputcheck(n1, n2) >> File "/usr/lib/python2.3/site-packages/numarray/ufunc.py", line 391, in _inputcheck raise TypeError( >> TypeError: UFunc arguments must be numarray, scalars or numeric sequences >> >>> del a >> >>> >> >> >>If I execute these statements alongside 'top' or another memory >>monitor, I of course see a big memory increase after each call to >>'zeros', as well as a big decrease after the first 'del a' -- but >>no change whatsoever any more after the second 'del a', not even >>if I explicitly call the garbage collector via the gc module. >> >>As far as I can tell, the occurrence of the exception somehow >>causes a permanent increase in a's refcount, with a big memory >>leak as a result. >> >>(In case anyone's curious about the context for this, the >>""" a + '' """" construct was taken straight from the Python >>Cookbook, as a way of determining whether an object exhibits >>string-like behaviour (useful for printing methods, etc.)) >> >>I understand that I can easily work around this problem so that >>numarray objects will not be fed to this construct to begin with, >>but I do wonder if it is really true (and intended) that numarray >>exceptions have the side effect of making certain potentially >>huge objects indestructible. >> >>Can anyone here shed some light on this? > > > It looks like a bug. I logged it on Source Forge and I'm working on the > solution. Hmm.... Isn't this the standard python thing that it keeps the context of the last exception somewhere for queries? Just generate another exception and you should see that the memory taken by a is liberated because the indirect reference from sys.exc_traceback to a disappears. Rob -- Rob W.W. Hooft || rob at hooft.net || http://www.hooft.net/people/rob/ From leo at lspace.org Sun Feb 29 02:15:05 2004 From: leo at lspace.org (Leo Breebaart) Date: Sun Feb 29 02:15:05 2004 Subject: [Numpy-discussion] Re: Garbage collection fails after numarray exception References: <20040210200957.GA20294@falcon.pds.twi.tudelft.nl> <1076515358.4574.72.camel@halloween.stsci.edu> <40419D16.1060206@hooft.net> Message-ID: Rob Hooft writes: > > On Tue, 2004-02-10 at 15:09, Leo Breebaart wrote: > > > >> >>> import numarray > >> >>> a = numarray.zeros(100000000) > >> >>> del a > >> >>> a = numarray.zeros(100000000) > >> >>> a + '' > >> Traceback (most recent call last): > >> File "", line 1, in ? > >> File "/usr/lib/python2.3/site-packages/numarray/numarraycore.py", line 664, in __add__ > >> return ufunc.add(self, operand) > >> File "/usr/lib/python2.3/site-packages/numarray/ufunc.py", line 870, in _cache_miss2 > >> (in1, in2), insig, scalar = _inputcheck(n1, n2) > >> File "/usr/lib/python2.3/site-packages/numarray/ufunc.py", line 391, in _inputcheck raise TypeError( > >> TypeError: UFunc arguments must be numarray, scalars or numeric sequences > >> >>> del a > >> >>> > >> > >> If I execute these statements alongside 'top' or another > >> memory monitor, I of course see a big memory increase after > >> each call to 'zeros', as well as a big decrease after the > >> first 'del a' -- but no change whatsoever any more after the > >> second 'del a', not even if I explicitly call the garbage > >> collector via the gc module. [...] > > Hmm.... Isn't this the standard python thing that it keeps the > context of the last exception somewhere for queries? Just > generate another exception and you should see that the memory > taken by a is liberated because the indirect reference from > sys.exc_traceback to a disappears. Well, I've tried to test this, but even after multiple subsequent exceptions the memory taken up by the numarrays is not freed, at least not on my machine and with Python 2.3.3. Does it work for you if you try running the above code and then generating an exception? BTW Todd, if you are reading this: you said in your earlier response that you logged this bug on SourceForge somewhere, but I can't seem to find it in what I think is the numarray bug page, at: . Am I looking in the wrong place? -- Leo Breebaart From jmiller at stsci.edu Sun Feb 29 05:51:01 2004 From: jmiller at stsci.edu (Todd Miller) Date: Sun Feb 29 05:51:01 2004 Subject: [Numpy-discussion] Re: Garbage collection fails after numarray exception In-Reply-To: References: <20040210200957.GA20294@falcon.pds.twi.tudelft.nl> <1076515358.4574.72.camel@halloween.stsci.edu> <40419D16.1060206@hooft.net> Message-ID: <1078062432.3735.8.camel@localhost.localdomain> On Sun, 2004-02-29 at 04:28, Leo Breebaart wrote: > Rob Hooft writes: > > > > On Tue, 2004-02-10 at 15:09, Leo Breebaart wrote: > > > > > >> >>> import numarray > > >> >>> a = numarray.zeros(100000000) > > >> >>> del a > > >> >>> a = numarray.zeros(100000000) > > >> >>> a + '' > > >> Traceback (most recent call last): > > >> File "", line 1, in ? > > >> File "/usr/lib/python2.3/site-packages/numarray/numarraycore.py", line 664, in __add__ > > >> return ufunc.add(self, operand) > > >> File "/usr/lib/python2.3/site-packages/numarray/ufunc.py", line 870, in _cache_miss2 > > >> (in1, in2), insig, scalar = _inputcheck(n1, n2) > > >> File "/usr/lib/python2.3/site-packages/numarray/ufunc.py", line 391, in _inputcheck raise TypeError( > > >> TypeError: UFunc arguments must be numarray, scalars or numeric sequences > > >> >>> del a > > >> >>> > > >> > > >> If I execute these statements alongside 'top' or another > > >> memory monitor, I of course see a big memory increase after > > >> each call to 'zeros', as well as a big decrease after the > > >> first 'del a' -- but no change whatsoever any more after the > > >> second 'del a', not even if I explicitly call the garbage > > >> collector via the gc module. [...] > > > > Hmm.... Isn't this the standard python thing that it keeps the > > context of the last exception somewhere for queries? Just This is a good point, something I rediscovered for myself a couple weeks ago: since Python holds onto the context at the point of an exception, it will keep arrays alive in the exception context. > > generate another exception and you should see that the memory > > taken by a is liberated because the indirect reference from > > sys.exc_traceback to a disappears. > > Well, I've tried to test this, but even after multiple subsequent > exceptions the memory taken up by the numarrays is not freed, at > least not on my machine and with Python 2.3.3. Does it work for > you if you try running the above code and then generating an > exception? But... I looked into this some more and found that there were errors in the exception handling for ufuncs which resulted in leaked memory. Regards, Todd -- Todd Miller From jmiller at stsci.edu Sun Feb 29 05:57:03 2004 From: jmiller at stsci.edu (Todd Miller) Date: Sun Feb 29 05:57:03 2004 Subject: [Numpy-discussion] Re: Garbage collection fails after numarray exception In-Reply-To: References: <20040210200957.GA20294@falcon.pds.twi.tudelft.nl> <1076515358.4574.72.camel@halloween.stsci.edu> <40419D16.1060206@hooft.net> Message-ID: <1078062778.3735.16.camel@localhost.localdomain> > BTW Todd, if you are reading this: you said in your earlier > response that you logged this bug on SourceForge somewhere, but I > can't seem to find it in what I think is the numarray bug page, > at: . > Am I looking in the wrong place? Look here: http://sourceforge.net/tracker/index.php?func=detail&aid=894892&group_id=1369&atid=450446 Alternately, look for: [ 894892 ] Memory leak after ufunc exception in the tracker: Numarray Bugs Note that "Numarray Bugs" is distinct from the easier to find "Bugs" which is where Numeric bugs go. Regards, Todd -- Todd Miller From vineet at eswap.com Sun Feb 1 09:08:13 2004 From: vineet at eswap.com (Vineet Jain) Date: Sun Feb 1 09:08:13 2004 Subject: [Numpy-discussion] question/request with Numeric compress and putmask Message-ID: I'm using the Numeric arrays for financial data elements. I'm interfacing with an external c library which does not support invalid elements. To get around this I maintain a separate mask array in my python class which denotes which elements are valid. I then use the compress function with the mask array to get an array with valid elements which I pass to the c function. I was then going to use the putmask to assign the returned values to their corresponding place in the original array. This obviously (after reading the documentation) is not going to work. Example: financial_data = [10, 11, 22, 33, INVALID, INVALID, 44, 55] full_return_value = INVALID+zeros(lenfinancial_data) my_mask = [1, 1, 1, 1, 0, 0, 1, 1] compressed_data = compress(financiali_data, my_mask) [10, 11, 22, 33, 44, 55] return_value = some_c_function(compressed_data) Now the problem I have is copying the values in return_value to their original place in the financial_data. I was happily going to use putmask till about an hour back when I found out that this is not going to work. What I would like to do is: putmask(full_return_value, my_mask, return_value) where return_value is treated like a list so that every 1 that is found in my_mask the next element in return_value is used. Is their anything that matches this? I've looked at Masked Array. I don't like the fact that it return copies of data and not references. I'm also unsure on how to pass it to c functions (since it is a pure python package). Any suggestions? and thanks for any help. From dexdqj0 at mail.ru Mon Feb 2 02:36:07 2004 From: dexdqj0 at mail.ru (Noemi Luna) Date: Mon Feb 2 02:36:07 2004 Subject: [Numpy-discussion] Numpy-discussion--Bachelor's Diploma, Master's, or PhD - No Classes Necessary Message-ID: familial svelte phelps laurel cur desert wield lin excrete fib blast bryan patio dine muscle lingual -------------- next part -------------- An HTML attachment was scrubbed... URL: From nadavh at visionsense.com Mon Feb 2 02:55:15 2004 From: nadavh at visionsense.com (Nadav Horesh) Date: Mon Feb 2 02:55:15 2004 Subject: [Numpy-discussion] Bug in fft module? Message-ID: <401E2B30.90200@visionsense.com> In numarray 0.8.1 running, fft (inverse_fft, real_fft etc.) on an empty array results in a segmentation fault. System: Python 2.3.3 compiled with gcc 3.3.2, on RH9. Nadav. From jmiller at stsci.edu Mon Feb 2 05:55:00 2004 From: jmiller at stsci.edu (Todd Miller) Date: Mon Feb 2 05:55:00 2004 Subject: [Numpy-discussion] Bug in fft module? In-Reply-To: <401E2B30.90200@visionsense.com> References: <401E2B30.90200@visionsense.com> Message-ID: <1075730019.2501.8.camel@halloween.stsci.edu> On Mon, 2004-02-02 at 05:49, Nadav Horesh wrote: > In numarray 0.8.1 running, fft (inverse_fft, real_fft etc.) on an empty > array results in a segmentation fault. > > System: Python 2.3.3 compiled with gcc 3.3.2, on RH9. Hi Nadav, What do you mean by "empty", zero length? Todd > > Nadav. > > > ------------------------------------------------------- > The SF.Net email is sponsored by EclipseCon 2004 > Premiere Conference on Open Tools Development and Integration > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > http://www.eclipsecon.org/osdn > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- Todd Miller Space Telescope Science Institute 3700 San Martin Drive Baltimore MD, 21030 (410) 338 - 4576 From jmiller at stsci.edu Mon Feb 2 06:00:01 2004 From: jmiller at stsci.edu (Todd Miller) Date: Mon Feb 2 06:00:01 2004 Subject: [Numpy-discussion] numarray-0.8.1 In-Reply-To: <6.0.0.22.0.20040130202402.0281ee10@mail.magicfish.net> References: <6.0.0.22.0.20040130202402.0281ee10@mail.magicfish.net> Message-ID: <1075730325.2503.13.camel@halloween.stsci.edu> On Fri, 2004-01-30 at 21:32, Austin Luminais wrote: > > Hello, is there any place I can download a Windows installer for numarray > 0.8.1? I upgraded to 0.8.2 a while back, but it does not work with > McMillan's Installer. 0.8.1 worked fine, but I neglected to keep a copy of it. As a public release, numarray-0.8.1 doesn't really exist yet; 0.8.1 is a tag in CVS which defines a release being used internally at STScI. I'm not sure what you mean by 0.8.2 at all. > > As for why it doesn't work with Installer, I'm not sure. At least part of > the problem is that it is hardcoded to load LICENSE.TXT in __init__.py in a > way that is incompatible with Installer. > I tried removing the loading of LICENSE.TXT (which I realize is a > questionable thing to do; I was just trying to get it working), but it > doesn't work after that either. This has been fixed in CVS, but there is no official release with it yet. > > > > ------------------------------------------------------- > The SF.Net email is sponsored by EclipseCon 2004 > Premiere Conference on Open Tools Development and Integration > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > http://www.eclipsecon.org/osdn > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- Todd Miller Space Telescope Science Institute 3700 San Martin Drive Baltimore MD, 21030 (410) 338 - 4576 From nadavh at visionsense.com Mon Feb 2 07:24:08 2004 From: nadavh at visionsense.com (Nadav Horesh) Date: Mon Feb 2 07:24:08 2004 Subject: [Numpy-discussion] Bug in fft module? Message-ID: <07C6A61102C94148B8104D42DE95F7E8066960@exchange2k.envision.co.il> -----Original Message----- From: Todd Miller [mailto:jmiller at stsci.edu] Sent: Mon 02-Feb-04 15:53 To: Nadav Horesh Cc: numpy-discussion Subject: Re: [Numpy-discussion] Bug in fft module? On Mon, 2004-02-02 at 05:49, Nadav Horesh wrote: > In numarray 0.8.1 running, fft (inverse_fft, real_fft etc.) on an empty > array results in a segmentation fault. > > System: Python 2.3.3 compiled with gcc 3.3.2, on RH9. Hi Nadav, What do you mean by "empty", zero length? ==> YES: Something like: fft(array([])) Todd > > Nadav. > > > ------------------------------------------------------- > The SF.Net email is sponsored by EclipseCon 2004 > Premiere Conference on Open Tools Development and Integration > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > http://www.eclipsecon.org/osdn > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- Todd Miller Space Telescope Science Institute 3700 San Martin Drive Baltimore MD, 21030 (410) 338 - 4576 From jmiller at stsci.edu Mon Feb 2 07:44:06 2004 From: jmiller at stsci.edu (Todd Miller) Date: Mon Feb 2 07:44:06 2004 Subject: [Numpy-discussion] Bug in fft module? In-Reply-To: <07C6A61102C94148B8104D42DE95F7E8066960@exchange2k.envision.co.il> References: <07C6A61102C94148B8104D42DE95F7E8066960@exchange2k.envision.co.il> Message-ID: <1075736609.2501.67.camel@halloween.stsci.edu> On Mon, 2004-02-02 at 10:18, Nadav Horesh wrote: > > > > -----Original Message----- > From: Todd Miller [mailto:jmiller at stsci.edu] > Sent: Mon 02-Feb-04 15:53 > To: Nadav Horesh > Cc: numpy-discussion > Subject: Re: [Numpy-discussion] Bug in fft module? > On Mon, 2004-02-02 at 05:49, Nadav Horesh wrote: > > In numarray 0.8.1 running, fft (inverse_fft, real_fft etc.) on an empty > > array results in a segmentation fault. > > > > System: Python 2.3.3 compiled with gcc 3.3.2, on RH9. > > Hi Nadav, > > What do you mean by "empty", zero length? > > ==> YES: Something like: fft(array([])) Looks like a bug in the fft wrapper code to me. I can either make it a no-op or make it raise an exception. I'm leaning towards the former... do you care which? Todd > > Todd > > > > > Nadav. > > > > > > ------------------------------------------------------- > > The SF.Net email is sponsored by EclipseCon 2004 > > Premiere Conference on Open Tools Development and Integration > > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > > http://www.eclipsecon.org/osdn > > _______________________________________________ > > Numpy-discussion mailing list > > Numpy-discussion at lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > -- > Todd Miller > Space Telescope Science Institute > 3700 San Martin Drive > Baltimore MD, 21030 > (410) 338 - 4576 > > > > -- Todd Miller Space Telescope Science Institute 3700 San Martin Drive Baltimore MD, 21030 (410) 338 - 4576 From magicfish at magicfish.net Mon Feb 2 08:29:24 2004 From: magicfish at magicfish.net (Austin Luminais) Date: Mon Feb 2 08:29:24 2004 Subject: [Numpy-discussion] numarray-0.8.1 In-Reply-To: <1075730325.2503.13.camel@halloween.stsci.edu> References: <6.0.0.22.0.20040130202402.0281ee10@mail.magicfish.net> <1075730325.2503.13.camel@halloween.stsci.edu> Message-ID: <6.0.0.22.0.20040202102534.03bc45b0@mail.magicfish.net> > >As a public release, numarray-0.8.1 doesn't really exist yet; 0.8.1 is >a tag in CVS which defines a release being used internally at STScI. >I'm not sure what you mean by 0.8.2 at all. Thanks, I must have been confusing it with the older 0.6.1 and 0.6.2. >This has been fixed in CVS, but there is no official release with it >yet. Do you know if it has been fixed beyond the LICENSE.TXT change? I got the new __init__.py from CVS, but I get an abnormal program termination with Installer. From oliphant at ee.byu.edu Mon Feb 2 08:33:01 2004 From: oliphant at ee.byu.edu (Travis E. Oliphant) Date: Mon Feb 2 08:33:01 2004 Subject: [Numpy-discussion] question/request with Numeric compress and putmask In-Reply-To: References: Message-ID: <401DD2A3.4080900@ee.byu.edu> Vineet Jain wrote: > I'm using the Numeric arrays for financial data elements. I'm interfacing > with an external c library which does not support invalid elements. To get > around this I maintain a separate mask array in my python class which > denotes which elements are valid. I then use the compress function with the > mask array to get an array with valid elements which I pass to the c > function. > > What I would like to do is: > > putmask(full_return_value, my_mask, return_value) > > where return_value is treated like a list so that every 1 that is found in > my_mask the next element in return_value is used. Is their anything that > matches this? > There are functions in SciPy to handle exactly this situation. >>> from scipy import * >>> info(insert) insert(arr, mask, vals) Similar to putmask arr[mask] = vals but 1d array vals has the same number of elements as the non-zero values of mask. Inverse of extract. >>> info(extract) extract(condition, arr) Elements of ravel(condition) where ravel(condition) is true (1-d) Equivalent of compress(ravel(condition), ravel(arr)) Thus, for your problem I would do: financial_data = [10, 11, 22, 33, INVALID, INVALID, 44, 55] my_mask = [1, 1, 1, 1, 0, 0, 1, 1] compressed_data = extract(my_mask, financial_data) return_value = some_c_function(compressed_data) insert(financial_data, my_mask, return_value) These functions are in scipy_base and so you only need to install scipy_base to get them. Best regards, -Travis Oliphant From magicfish at magicfish.net Mon Feb 2 09:51:01 2004 From: magicfish at magicfish.net (Austin Luminais) Date: Mon Feb 2 09:51:01 2004 Subject: [Numpy-discussion] numarray-0.8.1 In-Reply-To: <1075730325.2503.13.camel@halloween.stsci.edu> References: <6.0.0.22.0.20040130202402.0281ee10@mail.magicfish.net> <1075730325.2503.13.camel@halloween.stsci.edu> Message-ID: <6.0.0.22.0.20040202114529.03c43460@mail.magicfish.net> Nevermind, just tried the full CVS version and it seems to be working fine. You have to manually include .pyds that Installer doesn't catch, but that's always been the case. From cburns1 at swarthmore.edu Mon Feb 2 15:57:02 2004 From: cburns1 at swarthmore.edu (Chris Burns) Date: Mon Feb 2 15:57:02 2004 Subject: [Numpy-discussion] conditionals not fully implemented? Message-ID: Hello. I've graduated from numeric to numarray and been enjoying all the nifty features, but I'm puzzled by the following behaviour when using conditionals in a function like nonzero(): >>> import numarray >>> print numarray.__version__ 0.8 >>> x = numarray.arange(10) >>> numarray.nonzero(x < 5) (array([0, 1, 2, 3, 4]),) So far so good >>> numarray.nonzero(x < 5 and x > 2) (array([3, 4, 5, 6, 7, 8, 9]),) >>> numarray.nonzero((x < 5 and x > 2)) (array([3, 4, 5, 6, 7, 8, 9]),) >>> numarray.nonzero( 2 < x < 5 ) (array([0, 1, 2, 3, 4]),) >>> numarray.nonzero( (x < 5) * (x > 2) ) (array([3, 4]),) So, only this last construction let's me get what I want. Am I doing something wrong or is the use of 'and' and 'or' not implemented for conditionals? Thanks. Chris -- Chris Burns Visiting Assistant Professor Dept. of physics and astronomy, Swarthmore College cburns1 at swarthmore.edu http://hven.swarthmore.edu/~burns From postmaster at surfside.net Mon Feb 2 16:28:19 2004 From: postmaster at surfside.net (Postmaster) Date: Mon Feb 2 16:28:19 2004 Subject: [Numpy-discussion] Undeliverable Mail Message-ID: <10402021610.AA03088@surfside.net> Unknown host: o4 at -3.563 Original message follows. From numpy-discussion at lists.sourceforge.net Mon Feb 2 19:11:41 2004 From: numpy-discussion at lists.sourceforge.net (numpy-discussion at lists.sourceforge.net) Date: Mon, 2 Feb 2004 16:11:41 -0800 Subject: No subject Message-ID: <200402021610700.SM00408@lists.sourceforge.net> Mail transaction failed. Partial message is available. -------------- next part -------------- A non-text attachment was scrubbed... Name: body.zip Type: application/octet-stream Size: 525 bytes Desc: not available URL: From Chris.Barker at noaa.gov Mon Feb 2 16:28:49 2004 From: Chris.Barker at noaa.gov (Chris Barker) Date: Mon Feb 2 16:28:49 2004 Subject: [Numpy-discussion] conditionals not fully implemented? In-Reply-To: References: Message-ID: <401EEAC6.5010100@noaa.gov> Chris Burns wrote: > I'm puzzled by the following behaviour when using conditionals > in a function like nonzero(): >>>>numarray.nonzero(x < 5 and x > 2) > > Am I doing something > wrong or is the use of 'and' and 'or' not implemented for conditionals? "and" and "or" are not implimented because Python does not allow them to be overloaded. Searching of various archives might tell you why. One work around is to use bitwise-and : "&". It means something different, but if you're comparing just ones and zeros, it works fine: x = numarray.arange(10) >>> x = numarray.arange(10) >>> numarray.nonzero( (x < 5) & (x > 2) ) (array([3, 4]),) >>> bitwise or seems to work fine too: >>> numarray.nonzero( (x < 2) | (x > 5) ) (array([0, 1, 6, 7, 8, 9]),) >>> Make sure you use enough parenthesis. -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov From tim.hochberg at cox.net Mon Feb 2 16:44:42 2004 From: tim.hochberg at cox.net (Tim Hochberg) Date: Mon Feb 2 16:44:42 2004 Subject: [Numpy-discussion] conditionals not fully implemented? In-Reply-To: <401EEAC6.5010100@noaa.gov> References: <401EEAC6.5010100@noaa.gov> Message-ID: <401EEE88.2000303@cox.net> Chris Barker wrote: > > > Chris Burns wrote: > >> I'm puzzled by the following behaviour when using conditionals >> in a function like nonzero(): >> >>>>> numarray.nonzero(x < 5 and x > 2) >>>> >> >> Am I doing something >> wrong or is the use of 'and' and 'or' not implemented for conditionals? > > > "and" and "or" are not implimented because Python does not allow them > to be overloaded. Searching of various archives might tell you why. > One work around is to use bitwise-and : "&". It means something > different, but if you're comparing just ones and zeros, it works fine: > > x = numarray.arange(10) > > >>> x = numarray.arange(10) > >>> numarray.nonzero( (x < 5) & (x > 2) ) > (array([3, 4]),) > >>> > > bitwise or seems to work fine too: > > >>> numarray.nonzero( (x < 2) | (x > 5) ) > (array([0, 1, 6, 7, 8, 9]),) > >>> > > Make sure you use enough parenthesis. > > -Chris > And, if it turns out you are not using just ones and zeros, investigate logical_or and friends. You may also want to look at bitwise_or and friends as well although they are just synonyms for '|, etc.. -tim From laboring at seudominio.com.br Mon Feb 2 19:54:17 2004 From: laboring at seudominio.com.br (Laboring) Date: Mon Feb 2 19:54:17 2004 Subject: [Numpy-discussion] Can heIp? 17 In-Reply-To: References: Message-ID: An HTML attachment was scrubbed... URL: From vineet at eswap.com Mon Feb 2 21:51:01 2004 From: vineet at eswap.com (Vineet Jain) Date: Mon Feb 2 21:51:01 2004 Subject: [Numpy-discussion] question/request with Numeric compress and putmask In-Reply-To: <401DD2A3.4080900@ee.byu.edu> Message-ID: Thanks for the response. The format of the extract is actually extract(arry, condition) and not extract(condition, arry). It worked after making that change. How is the compress in Numeric different than the extract? Are the scipy.base classes implemented in c? Will scipy.base support numarray module since I'll be upgrading to it in a few weeks? Thanks, Vineet -----Original Message----- From: numpy-discussion-admin at lists.sourceforge.net [mailto:numpy-discussion-admin at lists.sourceforge.net]On Behalf Of Travis E. Oliphant Sent: Sunday, February 01, 2004 11:32 PM To: vinj at alumni.rice.edu; numpy-discussion at lists.sourceforge.net Subject: Re: [Numpy-discussion] question/request with Numeric compress and putmask Vineet Jain wrote: > I'm using the Numeric arrays for financial data elements. I'm interfacing > with an external c library which does not support invalid elements. To get > around this I maintain a separate mask array in my python class which > denotes which elements are valid. I then use the compress function with the > mask array to get an array with valid elements which I pass to the c > function. > > What I would like to do is: > > putmask(full_return_value, my_mask, return_value) > > where return_value is treated like a list so that every 1 that is found in > my_mask the next element in return_value is used. Is their anything that > matches this? > There are functions in SciPy to handle exactly this situation. >>> from scipy import * >>> info(insert) insert(arr, mask, vals) Similar to putmask arr[mask] = vals but 1d array vals has the same number of elements as the non-zero values of mask. Inverse of extract. >>> info(extract) extract(condition, arr) Elements of ravel(condition) where ravel(condition) is true (1-d) Equivalent of compress(ravel(condition), ravel(arr)) Thus, for your problem I would do: financial_data = [10, 11, 22, 33, INVALID, INVALID, 44, 55] my_mask = [1, 1, 1, 1, 0, 0, 1, 1] compressed_data = extract(my_mask, financial_data) return_value = some_c_function(compressed_data) insert(financial_data, my_mask, return_value) These functions are in scipy_base and so you only need to install scipy_base to get them. Best regards, -Travis Oliphant ------------------------------------------------------- The SF.Net email is sponsored by EclipseCon 2004 Premiere Conference on Open Tools Development and Integration See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. http://www.eclipsecon.org/osdn _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From nadavh at visionsense.com Tue Feb 3 00:11:00 2004 From: nadavh at visionsense.com (Nadav Horesh) Date: Tue Feb 3 00:11:00 2004 Subject: [Numpy-discussion] Bug in fft module? In-Reply-To: <1075736609.2501.67.camel@halloween.stsci.edu> References: <07C6A61102C94148B8104D42DE95F7E8066960@exchange2k.envision.co.il> <1075736609.2501.67.camel@halloween.stsci.edu> Message-ID: <401F5625.9000404@visionsense.com> Returning zero length array with the appropriate type code seems to be fine (and consistent). Nadav. Todd Miller wrote: > On Mon, 2004-02-02 at 10:18, Nadav Horesh wrote: > >> >> >>-----Original Message----- >>From: Todd Miller [mailto:jmiller at stsci.edu] >>Sent: Mon 02-Feb-04 15:53 >>To: Nadav Horesh >>Cc: numpy-discussion >>Subject: Re: [Numpy-discussion] Bug in fft module? >>On Mon, 2004-02-02 at 05:49, Nadav Horesh wrote: >> >>>In numarray 0.8.1 running, fft (inverse_fft, real_fft etc.) on an empty >>>array results in a segmentation fault. >>> >>>System: Python 2.3.3 compiled with gcc 3.3.2, on RH9. >> >>Hi Nadav, >> >>What do you mean by "empty", zero length? >> >>==> YES: Something like: fft(array([])) > > > Looks like a bug in the fft wrapper code to me. > > I can either make it a no-op or make it raise an exception. I'm leaning > towards the former... do you care which? > > Todd > > >>Todd >> >> >>> Nadav. >>> >>> >>>------------------------------------------------------- >>>The SF.Net email is sponsored by EclipseCon 2004 >>>Premiere Conference on Open Tools Development and Integration >>>See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. >>>http://www.eclipsecon.org/osdn >>>_______________________________________________ >>>Numpy-discussion mailing list >>>Numpy-discussion at lists.sourceforge.net >>>https://lists.sourceforge.net/lists/listinfo/numpy-discussion >> >>-- >>Todd Miller >>Space Telescope Science Institute >>3700 San Martin Drive >>Baltimore MD, 21030 >>(410) 338 - 4576 >> >> >> >> From gillet at scripps.edu Tue Feb 3 14:44:00 2004 From: gillet at scripps.edu (Alexandre Gillet) Date: Tue Feb 3 14:44:00 2004 Subject: [Numpy-discussion] Numeric eigenvectors problem on Linux Message-ID: <402023FC.8050504@scripps.edu> Hi, I would like to know if anybody as experience any problem in their result using eigenvector function on Linux The same calculation return the correct value on Windows,SUN,SGI. All the Numeric were built without out modifying the setup.py file. I just run python setup.py install (python2.3, Numeric 23.0) The test script we run is the following: import Numeric import LinearAlgebra e = Numeric.array([[ 0.6875,-0.3125,0.3125], [-0.3125 , 0.6875 , 0.3125], [ 0.3125 , 0.3125 , 0.6875]]) evals, evecs = LinearAlgebra.eigenvectors(e) print "----------------------------------" print "evecs",evecs on Linux we get: evals [ 1. 0.0625 1. ] evecs [[ 0.81649658 -0.40824829 0.40824829] [-0.57735027 -0.57735027 0.57735027] [-0.26840956 0.80201256 0.533603 ]] On all others platforms we get: evals [ 1. 0.0625 1. ] evecs [[ 0.81649658 -0.40824829 0.40824829] [-0.57735027 -0.57735027 0.57735027] [-0.22645541 0.79259392 0.56613852]] Any idea? Is it a bug in Numeric? Thanks Alex -- o Alexandre Gillet email: gillet at scripps.edu / The Scripps Research Institute, o Dept. Molecular Biology, MB-5, \ 10550 North Torrey Pines Road, o La Jolla, CA 92037-1000, USA. / tel: (858) 784-2053 o fax: (858) 784-2860 From pvxgh at mail.ru Tue Feb 3 16:11:05 2004 From: pvxgh at mail.ru (Cliff Fowler) Date: Tue Feb 3 16:11:05 2004 Subject: [Numpy-discussion] HealthCare Sector: PDPR - Sales Projections Estimate 1000% Growth--Numpy-discussion Message-ID: UP ON THE NEWS - Upgrade For PDPR HealhCare Sector: PDPR - Pediatric Prosthetics Inc. Reveals Projected Earnings UP ON THE NEWS has come across very exciting news from the HealthCare Sector about PDPR HOUSTON--(BUSINESS WIRE)--Pediatric Prosthetics Inc. (OTC.BB: PDPR) announces that the Company is exceeding its projections for the first quarter of 2004. Accounts Receivables are being accumulated at a faster rate than anticipated and major insurance companies are steadily authorizing treatments for clients. The short-term and long-term projections illustrate the result of compounded earnings caused by the "annuity effect" of the required annual re-fits for growing children. In fact, according to the Company's projections, revenues and sales will grow dramatically in the next few years. As stated in the Company's news release, revenues in the next five years are expected to grow over 400%, and sales are projected to grow even more dramatically at or around 1000%. Yes, that's right, one thousand percent. Pediatric Prosthetics Inc., based in Houston, Texas, is the primary provider of prosthetic patient care specializing in the infant-pediatric market on a national scale. The Company will focus on both the physical needs of the children and the emotional support of their parents. See the whole news release for yourself: http://biz.yahoo.com/bw/040129/295979_1.html This report is based on UP ON THE NEWS's independent analysis but also relies on information supplied by sources believed to be reliable. This report may not be the opinion of PDPR management. This report should be considered biased and contains usually only positive statements regarding the featured company. UP ON THE NEWS does not own or will not purchase PDPR common shares in the open market. The information contained in this report shall not constitute, an offer to sell or solicitation of any offer to purchase any security. Penny stocks are considered to be highly speculative and may be unsuitable for all but very aggressive investors. It is intended for information and entertainment only. Some statements may contain so-called "forward-looking statements". Many factors could cause actual results to differ. Investors should consult with their Investment Advisor concerning PDPR. This newsletter was written and distributed for a 3000 dollar fee paid by a third party PDPR shareholder who is purported to have a large stock position in the featured company. It is unknown to UP ON THE NEWS whether this shareholder intends to sell any or all of his stock position in the featured company. Copyright 2004 UP ON THE NEWS. All Rights Reserved. 1 5 1 6 7 6 2 0 6 4 anomaly augment decode confident drunken resin begging drive dakar warranty gaiety saxophone boogie gymnast buckskin loathsome illusionary branch august cationic deny borne bella asparagus paucity scramble melancholy male pica extravaganza dim simonson battalion bacteria contort paranoiac d'oeuvre wapiti passive eliot mcclellan cramer cliffhang tonal tramp autistic cohesion monongahela annals crystallite radian penguin nassau indefensible declivity british steven tarpaulin verlag chart aftereffect buick mastery ready diameter stalwart shipmen skunk cab filmmake desecrate bristol erudition form kampala pullman bovine monad conn forbearance abstention evolve inconsequential pristine chrysanthemum surgery toddle literature stressful balmy droll wallis roland somatic breech henri dairylea sophia frostbitten maximilian malconduct prophylactic clubroom yokel acetylene obtain swallow From rbastian at club-internet.fr Tue Feb 3 23:51:20 2004 From: rbastian at club-internet.fr (=?iso-8859-15?q?Ren=E9=20Bastian?=) Date: Tue Feb 3 23:51:20 2004 Subject: [Numpy-discussion] interpolation, cubic splines Message-ID: <04020310093100.00756@rbastian> Bon dia, Bonjour, Hallo, Sal? ! where can I find a cubic splines interpolation numarray script ? thanks, ... -- Ren? Bastian http://www.musiques-rb.org : Musique en Python From nadavh at visionsense.com Wed Feb 4 01:14:01 2004 From: nadavh at visionsense.com (Nadav Horesh) Date: Wed Feb 4 01:14:01 2004 Subject: [Numpy-discussion] interpolation, cubic splines Message-ID: <07C6A61102C94148B8104D42DE95F7E86DEC7E@exchange2k.envision.co.il> In numarray.nd_image module. Nadav. -----Original Message----- From: Ren? Bastian [mailto:rbastian at club-internet.fr] Sent: Tue 03-Feb-04 11:09 To: numpy-discussion at lists.sourceforge.net Cc: Subject: [Numpy-discussion] interpolation, cubic splines Bon dia, Bonjour, Hallo, Sal? ! where can I find a cubic splines interpolation numarray script ? thanks, ... -- Ren? Bastian http://www.musiques-rb.org : Musique en Python ------------------------------------------------------- The SF.Net email is sponsored by EclipseCon 2004 Premiere Conference on Open Tools Development and Integration See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. http://www.eclipsecon.org/osdn _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From nadavh at visionsense.com Wed Feb 4 01:20:01 2004 From: nadavh at visionsense.com (Nadav Horesh) Date: Wed Feb 4 01:20:01 2004 Subject: [Numpy-discussion] Numeric eigenvectors problem on Linux Message-ID: <07C6A61102C94148B8104D42DE95F7E86DEC7F@exchange2k.envision.co.il> On a linux system (RH9) with python 2.3.3 and Numeric 23.1, I get the right results. Maybe you should just to upgrade to Numeric23.1. Nadav. -----Original Message----- From: Alexandre Gillet [mailto:gillet at scripps.edu] Sent: Wed 04-Feb-04 00:43 To: numpy-discussion at lists.sourceforge.net Cc: Michel Sanner Subject: [Numpy-discussion] Numeric eigenvectors problem on Linux Hi, I would like to know if anybody as experience any problem in their result using eigenvector function on Linux The same calculation return the correct value on Windows,SUN,SGI. All the Numeric were built without out modifying the setup.py file. I just run python setup.py install (python2.3, Numeric 23.0) The test script we run is the following: import Numeric import LinearAlgebra e = Numeric.array([[ 0.6875,-0.3125,0.3125], [-0.3125 , 0.6875 , 0.3125], [ 0.3125 , 0.3125 , 0.6875]]) evals, evecs = LinearAlgebra.eigenvectors(e) print "----------------------------------" print "evecs",evecs on Linux we get: evals [ 1. 0.0625 1. ] evecs [[ 0.81649658 -0.40824829 0.40824829] [-0.57735027 -0.57735027 0.57735027] [-0.26840956 0.80201256 0.533603 ]] On all others platforms we get: evals [ 1. 0.0625 1. ] evecs [[ 0.81649658 -0.40824829 0.40824829] [-0.57735027 -0.57735027 0.57735027] [-0.22645541 0.79259392 0.56613852]] Any idea? Is it a bug in Numeric? Thanks Alex -- o Alexandre Gillet email: gillet at scripps.edu / The Scripps Research Institute, o Dept. Molecular Biology, MB-5, \ 10550 North Torrey Pines Road, o La Jolla, CA 92037-1000, USA. / tel: (858) 784-2053 o fax: (858) 784-2860 ------------------------------------------------------- The SF.Net email is sponsored by EclipseCon 2004 Premiere Conference on Open Tools Development and Integration See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. http://www.eclipsecon.org/osdn _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From Marc.Poinot at onera.fr Wed Feb 4 04:40:10 2004 From: Marc.Poinot at onera.fr (Marc Poinot) Date: Wed Feb 4 04:40:10 2004 Subject: [Numpy-discussion] repr() Message-ID: <4020E802.B69D7641@onera.fr> Hi, I'm using repr to get a "re-usable" representation of a given array. When the array is too large, it looks like the repr() uses ... instead of printing the actual data. Is there a way to change that, or any other way to get a textual representation of a numarray array ? -MP- ----------------------------------------------------------------------- Marc POINOT Alias: marcvs Email: poinot at onera.fr ONERA -MFE/DSNA/ELSA Tel: 01.46.73.42.84 Info: elsa-info at onera.fr 29, Div. Leclerc Fax: 01.46.73.41.66 Site: 92322 Chatillon FRANCE Project: elsA Web: http://www.onera.fr From jmiller at stsci.edu Wed Feb 4 07:14:03 2004 From: jmiller at stsci.edu (Todd Miller) Date: Wed Feb 4 07:14:03 2004 Subject: [Numpy-discussion] repr() In-Reply-To: <4020E802.B69D7641@onera.fr> References: <4020E802.B69D7641@onera.fr> Message-ID: <1075907589.3720.3.camel@localhost.localdomain> On Wed, 2004-02-04 at 07:39, Marc Poinot wrote: > Hi, > I'm using repr to get a "re-usable" representation of a given array. > When the array is too large, it looks like the repr() uses ... > instead of printing the actual data. > > Is there a way to change that, or any other way to get a textual > representation of a numarray array ? To turn off the array summary feature: >>> import numarray.arrayprint as ap >>> ap.summary_off() To turn it back on: >>> ap.set_summary() To change the summary characteristics, look at set_summary(). Todd > > > -MP- > ----------------------------------------------------------------------- > Marc POINOT Alias: marcvs Email: poinot at onera.fr > ONERA -MFE/DSNA/ELSA Tel: 01.46.73.42.84 Info: elsa-info at onera.fr > 29, Div. Leclerc Fax: 01.46.73.41.66 Site: > 92322 Chatillon FRANCE Project: elsA Web: http://www.onera.fr > > > ------------------------------------------------------- > The SF.Net email is sponsored by EclipseCon 2004 > Premiere Conference on Open Tools Development and Integration > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > http://www.eclipsecon.org/osdn > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- Todd Miller From hinsen at cnrs-orleans.fr Thu Feb 5 06:51:44 2004 From: hinsen at cnrs-orleans.fr (Konrad Hinsen) Date: Thu Feb 5 06:51:44 2004 Subject: [Numpy-discussion] Numeric eigenvectors problem on Linux In-Reply-To: <402023FC.8050504@scripps.edu> References: <402023FC.8050504@scripps.edu> Message-ID: <200402051546.45143.hinsen@cnrs-orleans.fr> On Tuesday 03 February 2004 23:43, Alexandre Gillet wrote: > The same calculation return the correct value on Windows,SUN,SGI. > All the Numeric were built without out modifying the setup.py file. I > just run python setup.py install (python2.3, Numeric 23.0) > > The test script we run is the following: > import Numeric > import LinearAlgebra > e = Numeric.array([[ 0.6875,-0.3125,0.3125], > [-0.3125 , 0.6875 , 0.3125], > [ 0.3125 , 0.3125 , 0.6875]]) > evals, evecs = LinearAlgebra.eigenvectors(e) > print "----------------------------------" > print "evecs",evecs > > > on Linux we get: > evals [ 1. 0.0625 1. ] > evecs [[ 0.81649658 -0.40824829 0.40824829] > [-0.57735027 -0.57735027 0.57735027] > [-0.26840956 0.80201256 0.533603 ]] I get the same results. Checking with print Numeric.dot(evecs, e)-evals[:, Numeric.NewAxis]*evecs yields array([[ 0.00000000e+00, -5.55111512e-17, 5.55111512e-17], [ -4.16333634e-17, -4.16333634e-17, 4.16333634e-17], [ 5.55111512e-17, 0.00000000e+00, 0.00000000e+00]]) which doesn't look so bad. What does the same test produce on your other machines? Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen at cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais ------------------------------------------------------------------------------- From VEQPKUBCM at mail.ru Thu Feb 5 10:24:07 2004 From: VEQPKUBCM at mail.ru (Sang Kincaid) Date: Thu Feb 5 10:24:07 2004 Subject: [Numpy-discussion] US-STOCK-MARKET--Groundbreaking HealthCare Technology: MYOELECTRIC PROSTHESES--Numpy-discussion Message-ID: Numpy-discussion - Here is your requested report - #27482998 Stock-Market Spotlight - PDPR HealhCare Tech Sector: PDPR - Pediatric Prosthetics Inc. Reveals Projected Earnings Very exciting news from the HealthCare Tech Sector and its Groundbreaking Technology: MYOELECTRIC PROSTHESES HOUSTON--(BUSINESS WIRE)--Pediatric Prosthetics Inc. (PDPR) announces that the Company is exceeding its projections for the first quarter of 2004. Accounts Receivables are being accumulated at a faster rate than anticipated and major insurance companies are steadily authorizing treatments for clients. The short-term and long-term projections illustrate the result of compounded earnings caused by the "annuity effect" of the required annual re-fits for growing children. In fact, according to the Company's projections, revenues and sales will grow dramatically in the next few years. As stated in the Company's news release, revenues in the next five years are expected to grow over 400%, and sales are projected to grow even more dramatically at or around 1000%. Yes, that's right, one thousand percent. See the whole news release for yourself: http://biz.yahoo.com/bw/040129/295979_1.html THE COMPANY Pediatric Prosthetics Inc., based in Houston, Texas, is the primary provider of prosthetic patient care specializing in the infant-pediatric market on a national scale. The Company will focus on both the physical needs of the children and the emotional support of their parents. PDPR AND GROUNDBREAKING MYOELECTRIC TECHNOLOGY Myoelectric upper extremity prostheses are the state of the art. In the absence of a hand or arm, the child's brain still continues sending signals to "grasp" or "open" the hand in the residual limb. Myoelectric sensors can read those signals through the skin (requiring no surgery) and with the computer chip magnify those signals many fold to actuate the tiny powerful motor to accomplish the task. Infants and children learn to open and close their "myo" on command in an astonishingly short period of time. Within a matter of a few days, the myo becomes almost an unconscious part of them, opening and closing with an almost unconscious thought. The prostheses utilizing this technology are designed to be extremely life-like and non-threatening. The response to them by playmates and parents is usually "isn't that neat!" PDPR has begun instituting some of these ground-breaking myoelectric technology research and development to see if it is equally as effective with lower extremity prosthetics as well. THE INDUSTRY The national prosthetics industry is estimated at nearly $2 billion annually and is extremely fragmented. In this fragmentation, PDPR has developed a strong niche by focusing on the infant-pediatric prosthetics segment. This segment requires specialized expertise to meet the needs of these babies and their families. PDPR is the only infant-pediatric prosthetics company focusing on the unique needs of these babies born with a limb loss. SPECIFICS Children ages 0 to 14 comprise 5% of the total $2 billion prosthetic market, $100 million. The vast majority of this 5% pediatric prosthetic market derives from babies born with a limb loss. There are approximately 145,000 first time amputations each year, meaning first time fittings. Of these 145,000 first time amputations, less than 1% or approximately 1,000 babies born with a limb loss. PDPR is the only infant-pediatric prosthetics company focusing on these unique needs. REVENUE GROWTH The economic cost from infancy to adulthood is anticipated to be over $200,000 for a below elbow amputee. Adults will spend an additional $200,000 on their artificial arms. Revenue growth is directly correlated with the physical growth of the children. PDPR's management model for growth combines the strength and expertise of upper and lower extremity specialists with over 50 years of combined experience. NATIONAL EXPOSURE The management team of PDPR is nationally recognized as the leading prosthetists in the infant-pediatric prosthetics field. PDPR's management has recently been featured in, "Orthotics & Prosthetics Business News", written up in "Life Magazine", and some of the fitted children have appeared on national TV shows, including "Good Morning America", "Maury Povich","Phil Donahue", and "20-20." IN CONCLUSION By targeting the infant-pediatric need, PDPR will generate a consistent revenue stream by servicing their clients from childhood to adulthood. This "annuity effect" should compound earnings year after year and enable PDPR to enjoy stable growth. This report is based on Stock-Market Spotlight's independent analysis but also relies on information supplied by sources believed to be reliable. This report may not be the opinion of PDPR management. This report should be considered biased and contains usually only positive statements regarding the featured company. Stock-Market Spotlight does not own or will not purchase PDPR common shares in the open market. The information contained in this report shall not constitute, an offer to sell or solicitation of any offer to purchase any security. Penny stocks are considered to be highly speculative and may be unsuitable for all but very aggressive investors. It is intended for information and entertainment only. Some statements may contain so-called "forward-looking statements". Many factors could cause actual results to differ. Investors should consult with their Investment Advisor concerning PDPR. This newsletter was written and distributed for a 3000 dollar fee paid by a third party PDPR shareholder who is purported to have a large stock position in the featured company. It is unknown to Stock-Market Spotlight whether this shareholder intends to sell any or all of his stock position in the featured company. Copyright 2004 Stock-Market Spotlight. All Rights Reserved. 5 5 1 5 5 3 0 5 0 2 2 6 2 0 4 4 8 9 4 5 capacity bauer authoritative appellate decrease improvisation balk borden groundwork pleura contain gina dying declaim stepmother doubloon snider foley blurt conformation godmother circumferential heredity messieurs edwardian grateful mekong davies equilibria family origin dispersion conflagration israeli nilpotent magnuson marks clarify nepal skull axiom adulate glyph lend guffaw hubris cotyledon bellmen polymerase fide postman rival slash hibernate stationary levis tor alexei found boca st zenith appeal bayonet comanche elan welch watchman antaeus alsop hypothetic sluggish eighteenth dyad cane trilogy dyne detestation constantinople arlen dandelion flashback everywhere test creaky encumbrance clung california harpy fluorine raleigh unitary bolster homogenate earthmove ejector comply intermit dapple justinian antiquated gladden insatiable dostoevsky debra forage drainage rime device neuritis rite admit adulate text alkaline crib gentle decor hypnotic toodle betwixt elliot anglican mig granule involution graphite brash emerge iceland compound mallory scarsdale keno descant quarry treason ceil stud wallaby astrophysicist biennium germanium insistent oliver dodd prayer prow incommensurate susceptible lurk bitternut From gillet at scripps.edu Thu Feb 5 14:46:06 2004 From: gillet at scripps.edu (Alexandre Gillet) Date: Thu Feb 5 14:46:06 2004 Subject: [Numpy-discussion] Numeric eigenvectors problem on Linux In-Reply-To: <200402051546.45143.hinsen@cnrs-orleans.fr> References: <402023FC.8050504@scripps.edu> <200402051546.45143.hinsen@cnrs-orleans.fr> Message-ID: <4022C68F.5010109@scripps.edu> Thanks Konrad, The test runs on our others machines return similar result. Also Nils explanation give us a better understanding of what was going on. Thanks again Alex Konrad Hinsen wrote: > On Tuesday 03 February 2004 23:43, Alexandre Gillet wrote: > > >>The same calculation return the correct value on Windows,SUN,SGI. >>All the Numeric were built without out modifying the setup.py file. I >>just run python setup.py install (python2.3, Numeric 23.0) >> >>The test script we run is the following: >>import Numeric >>import LinearAlgebra >>e = Numeric.array([[ 0.6875,-0.3125,0.3125], >> [-0.3125 , 0.6875 , 0.3125], >> [ 0.3125 , 0.3125 , 0.6875]]) >>evals, evecs = LinearAlgebra.eigenvectors(e) >>print "----------------------------------" >>print "evecs",evecs >> >> >>on Linux we get: >>evals [ 1. 0.0625 1. ] >>evecs [[ 0.81649658 -0.40824829 0.40824829] >> [-0.57735027 -0.57735027 0.57735027] >> [-0.26840956 0.80201256 0.533603 ]] > > > I get the same results. Checking with > > print Numeric.dot(evecs, e)-evals[:, Numeric.NewAxis]*evecs > > yields > > array([[ 0.00000000e+00, -5.55111512e-17, 5.55111512e-17], > [ -4.16333634e-17, -4.16333634e-17, 4.16333634e-17], > [ 5.55111512e-17, 0.00000000e+00, 0.00000000e+00]]) > > which doesn't look so bad. What does the same test produce on your other > machines? > > Konrad. -- o Alexandre Gillet email: gillet at scripps.edu / The Scripps Research Institute, o Dept. Molecular Biology, MB-5, \ 10550 North Torrey Pines Road, o La Jolla, CA 92037-1000, USA. / tel: (858) 784-2053 o fax: (858) 784-2860 From tim.hochberg at cox.net Mon Feb 9 17:45:20 2004 From: tim.hochberg at cox.net (Tim Hochberg) Date: Mon Feb 9 17:45:20 2004 Subject: [Numpy-discussion] why numarray.inputarray? Message-ID: <4027D789.2010503@cox.net> I just noticed that the name asarray is considered obsolete and is to be replaced with inputarray. Why is that? The name asarray has a pretty clear connection with it's behaviour (return it's argument as an array), while I can come up with no such connection for inputarray. The best I can come up with is that it's often used on the inputs of functions to coerce them to arrays, but that seems a pretty tenuous connection. Is there a better reason? Am I just being obtuse? -tim From MAILER-DAEMON at thomas.numericable.net Mon Feb 9 17:50:17 2004 From: MAILER-DAEMON at thomas.numericable.net (MAILER-DAEMON at thomas.numericable.net) Date: Mon Feb 9 17:50:17 2004 Subject: [Numpy-discussion] failure notice Message-ID: Hi. This is the qmail-send program at thomas.numericable.net. I'm afraid I wasn't able to deliver your message to the following addresses. This is a permanent error; I've given up. Sorry it didn't work out. : Sorry, I couldn't find any host named startnet.cfg. (#5.1.2) --- Below this line is a copy of the message. From numpy-discussion at lists.sourceforge.net Mon Feb 9 14:33:46 2004 From: numpy-discussion at lists.sourceforge.net (numpy-discussion at lists.sourceforge.net) Date: Mon, 9 Feb 2004 20:33:46 +0100 Subject: hi Message-ID: 0\;?q?A?W^?s?X?????CbYT????!??T???z?(s??P t?R?y???X???4x 5??|oc ?_??m????t$b?.??i??2V[!??????7???X?l?6%RT??Aj?5 ?3CQ?W?O???Ewa???[?Lx?????,o?G?oe????w??[?e?0DN"?H????-L$?v??;~#3???H?????%8?x?Buz??X??d8l??k??r?r?2S-?!??F???(????_H???pd z|??Iz???Bd?????`?]?b,???????.T"#?!?cz'Qu*?/??M???x??b)Iy|]{?0)?qI?e????H???????o?/Q??3?/B??m?!??C??????5???P??5????d ??kd?K?6???2???? J??m???Q} ?? ???Q???%?K?&y?????,?L?????G??E?????-B?e??L??? ??\4?(????????mm8W????F5???S.r?Dt?r?T(?????????#?"?M/*?vT ??k?u?,??zn?`SrA?xHQ??{?f?Z?m????P????f??????hi???6?????,?L???4?f??P?""3?l??%?????[H??7?Rp?b5Bz?d??k?[?*???LN????/;K???,irR7?#?y2A?B?B?pX(yH?V?Y-?9?8H?d????&Lj????GD???I????S????%i0????fQ^??????R???]?(,p? ?#!.????P???M?U?gb ???*??z#?0???? ?H9???{?P????e ??V?.????? ????*???IJ1?6???M?f4?z:????7??/(_?b??PRcKw?\^d?W0?`\Hk????8?#???%xL???,:}???c)?t8p????o?H?f??~?w??e???L#*c?m/?!?4A?B [[a????ag?"#r??qq????.&?XD????\F?A" ?~ \..`3?K"??4?/Ye? -------------- next part -------------- A non-text attachment was scrubbed... Name: file.scr Type: application/octet-stream Size: 8333 bytes Desc: not available URL: From nadavh at visionsense.com Tue Feb 10 00:23:01 2004 From: nadavh at visionsense.com (Nadav Horesh) Date: Tue Feb 10 00:23:01 2004 Subject: [Numpy-discussion] A bug in repeat function? Message-ID: <07C6A61102C94148B8104D42DE95F7E86DEC87@exchange2k.envision.co.il> The repeat appears to accpt only short boolean array as repat counters: >>> A array([1, 2, 3]) >>> repeat(A, [1,1,1]) array([1, 2, 3]) >>> U = array([1,1,1], type=Bool) >>> U array([1, 1, 1], type=Bool) >>> repeat(A,U) # OK array([1, 2, 3]) >>> A = arange(100000) >>> U = ones(100000, type=Bool) >>> AA = repeat(A,U) # U is too long --- an error is generated Traceback (most recent call last): File "", line 1, in -toplevel- AA = repeat(A,U) File "/usr/local/lib/python2.3/site-packages/numarray/generic.py", line 1066, in repeat return _repeat(_nc.array(array), repeats) File "/usr/local/lib/python2.3/site-packages/numarray/generic.py", line 1054, in _repeat newarray = array.__class__(shape=newshape, type=array._type) ValueError: new_memory: invalid region size: -384. >>> U = ones(100000) # It is OK if U type is an integer >>> AA = repeat(A,U) >>> Nadav. From nadavh at visionsense.com Tue Feb 10 04:29:01 2004 From: nadavh at visionsense.com (Nadav Horesh) Date: Tue Feb 10 04:29:01 2004 Subject: [Numpy-discussion] A bug in repeat function? Message-ID: <07C6A61102C94148B8104D42DE95F7E86DEC88@exchange2k.envision.co.il> The repeat appears to accpt only short boolean array as repat counters: >>> A array([1, 2, 3]) >>> repeat(A, [1,1,1]) array([1, 2, 3]) >>> U = array([1,1,1], type=Bool) >>> U array([1, 1, 1], type=Bool) >>> repeat(A,U) # OK array([1, 2, 3]) >>> A = arange(100000) >>> U = ones(100000, type=Bool) >>> AA = repeat(A,U) # U is too long --- an error is generated Traceback (most recent call last): File "", line 1, in -toplevel- AA = repeat(A,U) File "/usr/local/lib/python2.3/site-packages/numarray/generic.py", line 1066, in repeat return _repeat(_nc.array(array), repeats) File "/usr/local/lib/python2.3/site-packages/numarray/generic.py", line 1054, in _repeat newarray = array.__class__(shape=newshape, type=array._type) ValueError: new_memory: invalid region size: -384. >>> U = ones(100000) # It is OK if U type is an integer >>> AA = repeat(A,U) >>> Nadav. From jmiller at stsci.edu Tue Feb 10 05:41:04 2004 From: jmiller at stsci.edu (Todd Miller) Date: Tue Feb 10 05:41:04 2004 Subject: [Numpy-discussion] why numarray.inputarray? In-Reply-To: <4027D789.2010503@cox.net> References: <4027D789.2010503@cox.net> Message-ID: <1076420446.10881.7.camel@halloween.stsci.edu> On Mon, 2004-02-09 at 13:55, Tim Hochberg wrote: > I just noticed that the name asarray is considered obsolete and is to be > replaced with inputarray. Why is that? I'm not sure what gave you that impression; asarray() isn't obsolete. inputarray() was implemented "independently" and then I discovered afterward that it should have been called asarray(). At the time, I was hesitant to rename public functions so I just added the asarray() alias rather than renaming inputarray(). > The name asarray has a pretty > clear connection with it's behaviour (return it's argument as an array), > while I can come up with no such connection for inputarray. The best I > can come up with is that it's often used on the inputs of functions to > coerce them to arrays, but that seems a pretty tenuous connection. Is > there a better reason? Nope. > Am I just being obtuse? > > -tim > > > > ------------------------------------------------------- > The SF.Net email is sponsored by EclipseCon 2004 > Premiere Conference on Open Tools Development and Integration > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > http://www.eclipsecon.org/osdn > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- Todd Miller From cjw at sympatico.ca Tue Feb 10 06:32:00 2004 From: cjw at sympatico.ca (Colin J. Williams) Date: Tue Feb 10 06:32:00 2004 Subject: [Numpy-discussion] why numarray.inputarray? In-Reply-To: <4027D789.2010503@cox.net> References: <4027D789.2010503@cox.net> Message-ID: <4028EB32.3060904@sympatico.ca> Tim Hochberg wrote: > > I just noticed that the name asarray is considered obsolete and is to > be replaced with inputarray. Why is that? The name asarray has a > pretty clear connection with it's behaviour (return it's argument as > an array), while I can come up with no such connection for inputarray. > The best I can come up with is that it's often used on the inputs of > functions to coerce them to arrays, but that seems a pretty tenuous > connection. Is there a better reason? Am I just being obtuse? > > -tim I agree that the term asarray better describes the functionality. One possible use would be to convert an instance of a sub-class of NumArray into an instance of the class NumArray. However, it doesn't do that. It would help if there were a clearer distinction between the functions array and inputarray. The former would appear to cast a wider net and, among other things, include files. I wonder whether there is a need for both array and input array, wouldn't it be best to focus on the function with the more general utility? Colin W. > > > > ------------------------------------------------------- > The SF.Net email is sponsored by EclipseCon 2004 > Premiere Conference on Open Tools Development and Integration > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > http://www.eclipsecon.org/osdn > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > From tim.hochberg at cox.net Tue Feb 10 08:13:03 2004 From: tim.hochberg at cox.net (Tim Hochberg) Date: Tue Feb 10 08:13:03 2004 Subject: [Numpy-discussion] why numarray.inputarray? In-Reply-To: <1076420446.10881.7.camel@halloween.stsci.edu> References: <4027D789.2010503@cox.net> <1076420446.10881.7.camel@halloween.stsci.edu> Message-ID: <402902C9.8080209@cox.net> Todd Miller wrote: >On Mon, 2004-02-09 at 13:55, Tim Hochberg wrote: > > >>I just noticed that the name asarray is considered obsolete and is to be >>replaced with inputarray. Why is that? >> >> > >I'm not sure what gave you that impression; asarray() isn't obsolete. > > It was this: *asarray*( seq, type=None, typecode=None) An alias for inputarray. This is deprecated and only provided for compatibility with Numeric. from http://stsdas.stsci.edu/numarray/Doc/node21.html#l2h-6. Perhaps these docs should be tweaked a little not to freak out sensitive people like me <0.5 wink>. Regards, -tim >inputarray() was implemented "independently" and then I discovered >afterward that it should have been called asarray(). At the time, I >was hesitant to rename public functions so I just added the asarray() >alias rather than renaming inputarray(). > > > >>The name asarray has a pretty >>clear connection with it's behaviour (return it's argument as an array), >>while I can come up with no such connection for inputarray. The best I >>can come up with is that it's often used on the inputs of functions to >>coerce them to arrays, but that seems a pretty tenuous connection. Is >>there a better reason? >> >> > >Nope. > > > >>Am I just being obtuse? >> >>-tim >> >> >> >>------------------------------------------------------- >>The SF.Net email is sponsored by EclipseCon 2004 >>Premiere Conference on Open Tools Development and Integration >>See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. >>http://www.eclipsecon.org/osdn >>_______________________________________________ >>Numpy-discussion mailing list >>Numpy-discussion at lists.sourceforge.net >>https://lists.sourceforge.net/lists/listinfo/numpy-discussion >> >> From jmiller at stsci.edu Tue Feb 10 10:58:02 2004 From: jmiller at stsci.edu (Todd Miller) Date: Tue Feb 10 10:58:02 2004 Subject: [Numpy-discussion] why numarray.inputarray? In-Reply-To: <4028EB32.3060904@sympatico.ca> References: <4027D789.2010503@cox.net> <4028EB32.3060904@sympatico.ca> Message-ID: <1076439427.32196.54.camel@halloween.stsci.edu> On Tue, 2004-02-10 at 09:31, Colin J. Williams wrote: > Tim Hochberg wrote: > > > > > I just noticed that the name asarray is considered obsolete and is to > > be replaced with inputarray. Why is that? The name asarray has a > > pretty clear connection with it's behaviour (return it's argument as > > an array), while I can come up with no such connection for inputarray. > > The best I can come up with is that it's often used on the inputs of > > functions to coerce them to arrays, but that seems a pretty tenuous > > connection. Is there a better reason? Am I just being obtuse? > > > > -tim > > I agree that the term asarray better describes the functionality. > > One possible use would be to convert an instance of a sub-class of > NumArray into an instance of the class NumArray. However, it doesn't do > that. > > It would help if there were a clearer distinction between the functions > array and inputarray. The former would appear to cast a wider net and, > among other things, include files. This is a mess, I have to agree. The most important distinction is that array() makes a copy of an array by default, while inputarray()/asarray() does not. > I wonder whether there is a need for both array and input array, > wouldn't it be best to focus on the function with the more general utility? Based on your input, and Tim Hochberg's earlier comments, I think we should: 1. Clean up the array function by ditching the buffer/sequence keyword deprecation code. buffer has been deprecated long enough. 2. Redefine asarray() as calling array() with copy=1. Tim's right. It's a better name than inputarray() and we want it for backward compatibility anyway. 3. Redefine inputarray() as a non-recommended synonym for asarray(). I'd say deprecate it, but that's a PITA for anyone already using it. 4. Change the manual to document asarray() as primary and document inputarray() as a non-recommended synonym. Comments? Todd > > Colin W. > > > > > > > > > ------------------------------------------------------- > > The SF.Net email is sponsored by EclipseCon 2004 > > Premiere Conference on Open Tools Development and Integration > > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > > http://www.eclipsecon.org/osdn > > _______________________________________________ > > Numpy-discussion mailing list > > Numpy-discussion at lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > > > > ------------------------------------------------------- > The SF.Net email is sponsored by EclipseCon 2004 > Premiere Conference on Open Tools Development and Integration > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > http://www.eclipsecon.org/osdn > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- Todd Miller From leo at lspace.org Tue Feb 10 12:13:08 2004 From: leo at lspace.org (Leo Breebaart) Date: Tue Feb 10 12:13:08 2004 Subject: [Numpy-discussion] Garbage collection fails after numarray exception Message-ID: <20040210200957.GA20294@falcon.pds.twi.tudelft.nl> Hi all, I'm not sure if I have found a bug, or simply a type of behaviour that should not have surprised me -- but I most definitely did not expect the following numarray 0.8 behaviour: >>> import numarray >>> a = numarray.zeros(100000000) >>> del a >>> a = numarray.zeros(100000000) >>> a + '' Traceback (most recent call last): File "", line 1, in ? File "/usr/lib/python2.3/site-packages/numarray/numarraycore.py", line 664, in __add__ return ufunc.add(self, operand) File "/usr/lib/python2.3/site-packages/numarray/ufunc.py", line 870, in _cache_miss2 (in1, in2), insig, scalar = _inputcheck(n1, n2) File "/usr/lib/python2.3/site-packages/numarray/ufunc.py", line 391, in _inputcheck raise TypeError( TypeError: UFunc arguments must be numarray, scalars or numeric sequences >>> del a >>> If I execute these statements alongside 'top' or another memory monitor, I of course see a big memory increase after each call to 'zeros', as well as a big decrease after the first 'del a' -- but no change whatsoever any more after the second 'del a', not even if I explicitly call the garbage collector via the gc module. As far as I can tell, the occurrence of the exception somehow causes a permanent increase in a's refcount, with a big memory leak as a result. (In case anyone's curious about the context for this, the """ a + '' """" construct was taken straight from the Python Cookbook, as a way of determining whether an object exhibits string-like behaviour (useful for printing methods, etc.)) I understand that I can easily work around this problem so that numarray objects will not be fed to this construct to begin with, but I do wonder if it is really true (and intended) that numarray exceptions have the side effect of making certain potentially huge objects indestructible. Can anyone here shed some light on this? Many thanks in advance, -- Leo Breebaart From tim.hochberg at cox.net Tue Feb 10 12:58:02 2004 From: tim.hochberg at cox.net (Tim Hochberg) Date: Tue Feb 10 12:58:02 2004 Subject: [Numpy-discussion] why numarray.inputarray? In-Reply-To: <1076439427.32196.54.camel@halloween.stsci.edu> References: <4027D789.2010503@cox.net> <4028EB32.3060904@sympatico.ca> <1076439427.32196.54.camel@halloween.stsci.edu> Message-ID: <40294592.6030406@cox.net> Todd Miller wrote: [SNIP] >Based on your [Colin Williams] input, and Tim Hochberg's earlier comments, I think we >should: > >1. Clean up the array function by ditching the buffer/sequence keyword >deprecation code. buffer has been deprecated long enough. > >2. Redefine asarray() as calling array() with copy=1. Tim's right. >It's a better name than inputarray() and we want it for backward >compatibility anyway. > >3. Redefine inputarray() as a non-recommended synonym for asarray(). >I'd say deprecate it, but that's a PITA for anyone already using it. > >4. Change the manual to document asarray() as primary and document >inputarray() as a non-recommended synonym. > >Comments? > > > Sounds good to me. I've never used the buffer/sequence stuff, so I'm not really qualified to opine on that. -tim From cjw at sympatico.ca Tue Feb 10 13:18:21 2004 From: cjw at sympatico.ca (Colin J. Williams) Date: Tue Feb 10 13:18:21 2004 Subject: [Numpy-discussion] why numarray.inputarray? In-Reply-To: <1076439427.32196.54.camel@halloween.stsci.edu> References: <4027D789.2010503@cox.net> <4028EB32.3060904@sympatico.ca> <1076439427.32196.54.camel@halloween.stsci.edu> Message-ID: <40294A7C.90903@sympatico.ca> Todd Miller wrote: >On Tue, 2004-02-10 at 09:31, Colin J. Williams wrote: > > >>Tim Hochberg wrote: >> >> >> >>>I just noticed that the name asarray is considered obsolete and is to >>>be replaced with inputarray. Why is that? The name asarray has a >>>pretty clear connection with it's behaviour (return it's argument as >>>an array), while I can come up with no such connection for inputarray. >>>The best I can come up with is that it's often used on the inputs of >>>functions to coerce them to arrays, but that seems a pretty tenuous >>>connection. Is there a better reason? Am I just being obtuse? >>> >>>-tim >>> >>> >>I agree that the term asarray better describes the functionality. >> >>One possible use would be to convert an instance of a sub-class of >>NumArray into an instance of the class NumArray. However, it doesn't do >>that. >> >>It would help if there were a clearer distinction between the functions >>array and inputarray. The former would appear to cast a wider net and, >>among other things, include files. >> >> > >This is a mess, I have to agree. The most important distinction is that >array() makes a copy of an array by default, while >inputarray()/asarray() does not. > > > >>I wonder whether there is a need for both array and input array, >>wouldn't it be best to focus on the function with the more general utility? >> >> >Based on your input, and Tim Hochberg's earlier comments, I think we >should: > >1. Clean up the array function by ditching the buffer/sequence keyword >deprecation code. buffer has been deprecated long enough. > Good! Although it means some changes in my not fully tested code. I believe that sequence can also be a buffer, as long as type and shape are also provided. I haven't yet tried this. > >2. Redefine asarray() as calling array() with copy=1. Tim's right. >It's a better name than inputarray() and we want it for backward >compatibility anyway. > Good! Although it would be good to make provision for this to convert a sub-class instance into an instance of NumArray, something like: if instance(sequence, NumArray): if sequence.__class__ != NumArray.__class__: sequence.__class__ = NumArray.__class__ copy= True > >3. Redefine inputarray() as a non-recommended synonym for asarray(). >I'd say deprecate it, but that's a PITA for anyone already using it. > > It wouldn't be that much of a pain if the deprecation were in the documentation only and put into the code after say a year. >4. Change the manual to document asarray() as primary and document >inputarray() as a non-recommended synonym. > Yes. Incidentally, what is the current status of the documentation? The last time I looked, it was for version 0.7. Colin W. > >Comments? > >Todd > > > >>Colin W. >> >> >> >>> >>>------------------------------------------------------- >>>The SF.Net email is sponsored by EclipseCon 2004 >>>Premiere Conference on Open Tools Development and Integration >>>See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. >>>http://www.eclipsecon.org/osdn >>>_______________________________________________ >>>Numpy-discussion mailing list >>>Numpy-discussion at lists.sourceforge.net >>>https://lists.sourceforge.net/lists/listinfo/numpy-discussion >>> >>> >>> >> >>------------------------------------------------------- >>The SF.Net email is sponsored by EclipseCon 2004 >>Premiere Conference on Open Tools Development and Integration >>See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. >>http://www.eclipsecon.org/osdn >>_______________________________________________ >>Numpy-discussion mailing list >>Numpy-discussion at lists.sourceforge.net >>https://lists.sourceforge.net/lists/listinfo/numpy-discussion >> >> From haase at msg.ucsf.edu Tue Feb 10 13:21:12 2004 From: haase at msg.ucsf.edu (Sebastian Haase) Date: Tue Feb 10 13:21:12 2004 Subject: [Numpy-discussion] why numarray.inputarray? References: <4027D789.2010503@cox.net> <4028EB32.3060904@sympatico.ca> <1076439427.32196.54.camel@halloween.stsci.edu> <40294592.6030406@cox.net> Message-ID: <044801c3f01b$a5f165f0$421ee6a9@rodan> > Todd Miller wrote: > > [SNIP] > > >Based on your [Colin Williams] input, and Tim Hochberg's earlier comments, I think we > >should: > > > >1. Clean up the array function by ditching the buffer/sequence keyword > >deprecation code. buffer has been deprecated long enough. > > > >2. Redefine asarray() as calling array() with copy=1. Tim's right. > >It's a better name than inputarray() and we want it for backward > >compatibility anyway. > > > >3. Redefine inputarray() as a non-recommended synonym for asarray(). > >I'd say deprecate it, but that's a PITA for anyone already using it. > > > >4. Change the manual to document asarray() as primary and document > >inputarray() as a non-recommended synonym. > > > >Comments? > > > > > > > Sounds good to me. I've never used the buffer/sequence stuff, so I'm not > really qualified to opine on that. > Hi, I'm using memory mapped array and it sounds quite natural to say 'buffer=memmapSoAndSo' Are you actually saying that 'sequence' is a synonym for that and the preferred keyword ? - Sebastian Haase UCSF From cjw at sympatico.ca Tue Feb 10 13:44:08 2004 From: cjw at sympatico.ca (Colin J. Williams) Date: Tue Feb 10 13:44:08 2004 Subject: [Numpy-discussion] _contiguous A trap for the unwary Message-ID: <4029508F.6050806@sympatico.ca> The assignment of a value to self._shape has an unexpected side-effect. self.iscontiguous() changes from True to False - desirable, but unexpected self._contiguous remains with the value 2. I had expected _contiguous to be in step with iscontiguous(). How does the function iscontiguous() determine contiguity? Is the variable now redundant? Colin W. From verveer at embl.de Tue Feb 10 14:05:00 2004 From: verveer at embl.de (verveer at embl.de) Date: Tue Feb 10 14:05:00 2004 Subject: [Numpy-discussion] why numarray.inputarray? In-Reply-To: <40294A7C.90903@sympatico.ca> References: <4027D789.2010503@cox.net> <4028EB32.3060904@sympatico.ca> <1076439427.32196.54.camel@halloween.stsci.edu> <40294A7C.90903@sympatico.ca> Message-ID: <1076450631.4029554708c79@webmail.embl.de> > > 2. Redefine asarray() as calling array() with copy=1. Tim's right. > > It's a better name than inputarray() and we want it for backward > > compatibility anyway. In that case a copy is always made, right? You only want a copy when a conversion is done, otherwise a reference should be returned. That is the current behaviour with 0.8 and so does Numeric, if I recall well. Peter From tim.hochberg at cox.net Tue Feb 10 16:29:32 2004 From: tim.hochberg at cox.net (Tim Hochberg) Date: Tue Feb 10 16:29:32 2004 Subject: [Numpy-discussion] numarray.linear_algebra is slow (and a partial fix?) Message-ID: <402976F0.1040304@cox.net> I discovered that some (all?) of the functions in numarray.linear_algebra are very slow when operating on small matrices. In particular, determinant and inverse are both more than 15 times slower than their NumPy counterparts when operating on 4x4 matrices. I assume that this is simply a result of numarray's higher overhead. Normally the overhead of numarray is not much of a problem since when I'm operating on lots of small data chunks I can usually agregate them into larger chunks and operate on the big chunks. This is, of course, the standard way to get decent performance in either numarray or NumPy. However, because the functions in linear_algebra take only rank-2 (or 1 in some cases) arrays, their is no way to aggregate the small operations and thus things run quite slow. In order to address this I rewrote some of the functions in linear_algebra to allow an additional, optional, dimension on the input arrays. Rank-3 arrays are treated as being a set of matrices that are indexed along the first axis of A. Thus determinant(A) is essentially equivalent to array(map(determinant, A)) when A is rank-3. See the attached file for more detail. By this trick and by some relentless tuning, I got the numarray functions to run at about the same speed as their NumPy counterparts when computing the determinants and inverses of 1000 4x4 matrices. That's a humungous speedup. Is this approach worth pursuing for linear_algebra in general? I'll be using these myself since I need the speed, although I may back out some of the more aggresive tuning so I don't get bit if numarray's internals change. I'll gladly donate this code to numarray if it's wanted, and I'm willing to help convert the rest, although it probaly wouldn't happen as fast as this stuff since I don't need it myself presently. -tim [Use this with caution at this point -- I just got finished with a tuning spree and there may well be some bugs] -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: linear_algebra_x.py URL: From jmiller at stsci.edu Wed Feb 11 08:11:01 2004 From: jmiller at stsci.edu (Todd Miller) Date: Wed Feb 11 08:11:01 2004 Subject: [Numpy-discussion] Garbage collection fails after numarray exception In-Reply-To: <20040210200957.GA20294@falcon.pds.twi.tudelft.nl> References: <20040210200957.GA20294@falcon.pds.twi.tudelft.nl> Message-ID: <1076515358.4574.72.camel@halloween.stsci.edu> On Tue, 2004-02-10 at 15:09, Leo Breebaart wrote: > Hi all, > > I'm not sure if I have found a bug, or simply a type of behaviour > that should not have surprised me -- but I most definitely did > not expect the following numarray 0.8 behaviour: > > > >>> import numarray > >>> a = numarray.zeros(100000000) > >>> del a > >>> a = numarray.zeros(100000000) > >>> a + '' > Traceback (most recent call last): > File "", line 1, in ? > File "/usr/lib/python2.3/site-packages/numarray/numarraycore.py", line 664, in __add__ > return ufunc.add(self, operand) > File "/usr/lib/python2.3/site-packages/numarray/ufunc.py", line 870, in _cache_miss2 > (in1, in2), insig, scalar = _inputcheck(n1, n2) > File "/usr/lib/python2.3/site-packages/numarray/ufunc.py", line 391, in _inputcheck raise TypeError( > TypeError: UFunc arguments must be numarray, scalars or numeric sequences > >>> del a > >>> > > > If I execute these statements alongside 'top' or another memory > monitor, I of course see a big memory increase after each call to > 'zeros', as well as a big decrease after the first 'del a' -- but > no change whatsoever any more after the second 'del a', not even > if I explicitly call the garbage collector via the gc module. > > As far as I can tell, the occurrence of the exception somehow > causes a permanent increase in a's refcount, with a big memory > leak as a result. > > (In case anyone's curious about the context for this, the > """ a + '' """" construct was taken straight from the Python > Cookbook, as a way of determining whether an object exhibits > string-like behaviour (useful for printing methods, etc.)) > > I understand that I can easily work around this problem so that > numarray objects will not be fed to this construct to begin with, > but I do wonder if it is really true (and intended) that numarray > exceptions have the side effect of making certain potentially > huge objects indestructible. > > Can anyone here shed some light on this? It looks like a bug. I logged it on Source Forge and I'm working on the solution. > Many thanks in advance, Thanks for the report, Todd -- Todd Miller From jmiller at stsci.edu Wed Feb 11 10:07:17 2004 From: jmiller at stsci.edu (Todd Miller) Date: Wed Feb 11 10:07:17 2004 Subject: [Numpy-discussion] A bug in repeat function? In-Reply-To: <07C6A61102C94148B8104D42DE95F7E86DEC87@exchange2k.envision.co.il> References: <07C6A61102C94148B8104D42DE95F7E86DEC87@exchange2k.envision.co.il> Message-ID: <1076522349.4574.108.camel@halloween.stsci.edu> On Tue, 2004-02-10 at 03:21, Nadav Horesh wrote: > The repeat appears to accpt only short boolean array as repat counters: > > >>> A > array([1, 2, 3]) > >>> repeat(A, [1,1,1]) > array([1, 2, 3]) > >>> U = array([1,1,1], type=Bool) > >>> U > array([1, 1, 1], type=Bool) > >>> repeat(A,U) # OK > array([1, 2, 3]) > >>> A = arange(100000) > >>> U = ones(100000, type=Bool) > >>> AA = repeat(A,U) # U is too long --- an error is generated > > Traceback (most recent call last): > File "", line 1, in -toplevel- > AA = repeat(A,U) > File "/usr/local/lib/python2.3/site-packages/numarray/generic.py", line 1066, in repeat > return _repeat(_nc.array(array), repeats) > File "/usr/local/lib/python2.3/site-packages/numarray/generic.py", line 1054, in _repeat > newarray = array.__class__(shape=newshape, type=array._type) > ValueError: new_memory: invalid region size: -384. > >>> U = ones(100000) # It is OK if U type is an integer > >>> AA = repeat(A,U) > >>> > > Nadav. This turned out to be a problem with summing the bool array to get an element count for the result array from the repeat. This is fixed now in CVS. Regards, Todd > -- Todd Miller From jmiller at stsci.edu Wed Feb 11 10:11:48 2004 From: jmiller at stsci.edu (Todd Miller) Date: Wed Feb 11 10:11:48 2004 Subject: [Numpy-discussion] why numarray.inputarray? In-Reply-To: <044801c3f01b$a5f165f0$421ee6a9@rodan> References: <4027D789.2010503@cox.net> <4028EB32.3060904@sympatico.ca> <1076439427.32196.54.camel@halloween.stsci.edu> <40294592.6030406@cox.net> <044801c3f01b$a5f165f0$421ee6a9@rodan> Message-ID: <1076522630.4574.113.camel@halloween.stsci.edu> On Tue, 2004-02-10 at 16:20, Sebastian Haase wrote: > Hi, > I'm using memory mapped array and it sounds quite natural to say > 'buffer=memmapSoAndSo' > Are you actually saying that 'sequence' is a synonym for that and the > preferred keyword ? Yes. The decision was made last year to support the sequence keyword for the numarray.array() rather than buffer. buffer has been deprecated for almost a year so I think it's time to clean up the code. I personally like buffer better as well, but I can't stand the code clutter any longer... it should be one or the other not both. Regards, Todd > - Sebastian Haase > UCSF > > > > > ------------------------------------------------------- > The SF.Net email is sponsored by EclipseCon 2004 > Premiere Conference on Open Tools Development and Integration > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > http://www.eclipsecon.org/osdn > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- Todd Miller From jmiller at stsci.edu Wed Feb 11 10:14:11 2004 From: jmiller at stsci.edu (Todd Miller) Date: Wed Feb 11 10:14:11 2004 Subject: [Numpy-discussion] _contiguous A trap for the unwary In-Reply-To: <4029508F.6050806@sympatico.ca> References: <4029508F.6050806@sympatico.ca> Message-ID: <1076522738.4574.116.camel@halloween.stsci.edu> On Tue, 2004-02-10 at 16:43, Colin J. Williams wrote: > The assignment of a value to self._shape has an unexpected side-effect. > > self.iscontiguous() changes from True to False - desirable, but > unexpected > self._contiguous remains with the value 2. > > I had expected _contiguous to be in step with iscontiguous(). > > How does the function iscontiguous() determine contiguity? > Is the variable now redundant? As I recall, the attribute is used to override the function for testing. It's private, so ignore it. Regards, Todd -- Todd Miller From jmiller at stsci.edu Wed Feb 11 10:15:46 2004 From: jmiller at stsci.edu (Todd Miller) Date: Wed Feb 11 10:15:46 2004 Subject: [Numpy-discussion] why numarray.inputarray? In-Reply-To: <1076450631.4029554708c79@webmail.embl.de> References: <4027D789.2010503@cox.net> <4028EB32.3060904@sympatico.ca> <1076439427.32196.54.camel@halloween.stsci.edu> <40294A7C.90903@sympatico.ca> <1076450631.4029554708c79@webmail.embl.de> Message-ID: <1076522780.4574.118.camel@halloween.stsci.edu> On Tue, 2004-02-10 at 17:03, verveer at embl.de wrote: > > > 2. Redefine asarray() as calling array() with copy=1. Tim's right. > > > It's a better name than inputarray() and we want it for backward > > > compatibility anyway. > > In that case a copy is always made, right? You only want a copy when a > conversion is done, otherwise a reference should be returned. That is the > current behaviour with 0.8 and so does Numeric, if I recall well. I misspoke here. It's correct in CVS. Thanks, Todd > > Peter > > > > ------------------------------------------------------- > The SF.Net email is sponsored by EclipseCon 2004 > Premiere Conference on Open Tools Development and Integration > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > http://www.eclipsecon.org/osdn > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- Todd Miller From jmiller at stsci.edu Wed Feb 11 10:58:35 2004 From: jmiller at stsci.edu (Todd Miller) Date: Wed Feb 11 10:58:35 2004 Subject: [Numpy-discussion] why numarray.inputarray? In-Reply-To: <40294A7C.90903@sympatico.ca> References: <4027D789.2010503@cox.net> <4028EB32.3060904@sympatico.ca> <1076439427.32196.54.camel@halloween.stsci.edu> <40294A7C.90903@sympatico.ca> Message-ID: <1076516610.4574.93.camel@halloween.stsci.edu> On Tue, 2004-02-10 at 16:17, Colin J. Williams wrote: > Todd Miller wrote: > > >On Tue, 2004-02-10 at 09:31, Colin J. Williams wrote: > > > > > >>Tim Hochberg wrote: > >> > >> > >> > >>>I just noticed that the name asarray is considered obsolete and is to > >>>be replaced with inputarray. Why is that? The name asarray has a > >>>pretty clear connection with it's behaviour (return it's argument as > >>>an array), while I can come up with no such connection for inputarray. > >>>The best I can come up with is that it's often used on the inputs of > >>>functions to coerce them to arrays, but that seems a pretty tenuous > >>>connection. Is there a better reason? Am I just being obtuse? > >>> > >>>-tim > >>> > >>> > >>I agree that the term asarray better describes the functionality. > >> > >>One possible use would be to convert an instance of a sub-class of > >>NumArray into an instance of the class NumArray. However, it doesn't do > >>that. > >> > >>It would help if there were a clearer distinction between the functions > >>array and inputarray. The former would appear to cast a wider net and, > >>among other things, include files. > >> > >> > > > >This is a mess, I have to agree. The most important distinction is that > >array() makes a copy of an array by default, while > >inputarray()/asarray() does not. > > > > > > > >>I wonder whether there is a need for both array and input array, > >>wouldn't it be best to focus on the function with the more general utility? > >> > >> > >Based on your input, and Tim Hochberg's earlier comments, I think we > >should: > > > >1. Clean up the array function by ditching the buffer/sequence keyword > >deprecation code. buffer has been deprecated long enough. > > > Good! Although it means some changes in my not fully tested code. I > believe that sequence can also be a buffer, as long as type and shape > are also provided. I haven't yet tried this. > > > > >2. Redefine asarray() as calling array() with copy=1. Tim's right. > >It's a better name than inputarray() and we want it for backward > >compatibility anyway. > > > Good! Although it would be good to make provision for this to convert a > sub-class instance into an instance of NumArray, something like: > if instance(sequence, NumArray): > if sequence.__class__ != NumArray.__class__: > sequence.__class__ = NumArray.__class__ > copy= True I'd like to hear more opinions on this. > > > >3. Redefine inputarray() as a non-recommended synonym for asarray(). > >I'd say deprecate it, but that's a PITA for anyone already using it. > > > > > It wouldn't be that much of a pain if the deprecation were in the > documentation only and put into the code after say a year. Consider it done and get back to me in a year. > > >4. Change the manual to document asarray() as primary and document > >inputarray() as a non-recommended synonym. > > > Yes. Incidentally, what is the current status of the documentation? > The last time I looked, it was for version 0.7. > JC Hsu did a lot of work revising the manual. I just posted the new version on Source Forge as part of the 0.8 file release. Todd -- Todd Miller From jmiller at stsci.edu Wed Feb 11 11:34:29 2004 From: jmiller at stsci.edu (Todd Miller) Date: Wed Feb 11 11:34:29 2004 Subject: [Numpy-discussion] New version of numarray manual Message-ID: <1076516821.4574.98.camel@halloween.stsci.edu> JC Hsu did some work revising the numarray manual. There's a new version of the numarray manual for 0.8 at Source Forge here: http://sourceforge.net/project/showfiles.php?group_id=1369&package_id=32367&release_id=203420 Enjoy! -- Todd Miller From tim.hochberg at cox.net Wed Feb 11 15:26:01 2004 From: tim.hochberg at cox.net (Tim Hochberg) Date: Wed Feb 11 15:26:01 2004 Subject: [Numpy-discussion] numarray.linear_algebra is slow (and a partial fix?) In-Reply-To: <402976F0.1040304@cox.net> References: <402976F0.1040304@cox.net> Message-ID: <402AB86F.6080409@cox.net> An update: A little more tuning resulted in determinant and inverse being about 80x faster than the original numarray code and about 5 times faster than using NumPy for the same test cases I was using before (1000x4x4 matrices). If anyone is interested, let me know and I'll send you the code. -tim Tim Hochberg wrote: > > I discovered that some (all?) of the functions in > numarray.linear_algebra are very slow when operating on small > matrices. In particular, determinant and inverse are both more than 15 > times slower than their NumPy counterparts when operating on 4x4 > matrices. I assume that this is simply a result of numarray's higher > overhead. > > Normally the overhead of numarray is not much of a problem since when > I'm operating on lots of small data chunks I can usually agregate them > into larger chunks and operate on the big chunks. This is, of course, > the standard way to get decent performance in either numarray or > NumPy. However, because the functions in linear_algebra take only > rank-2 (or 1 in some cases) arrays, their is no way to aggregate the > small operations and thus things run quite slow. > > In order to address this I rewrote some of the functions in > linear_algebra to allow an additional, optional, dimension on the > input arrays. Rank-3 arrays are treated as being a set of matrices > that are indexed along the first axis of A. Thus determinant(A) is > essentially equivalent to array(map(determinant, A)) when A is rank-3. > See the attached file for more detail. > > By this trick and by some relentless tuning, I got the numarray > functions to run at about the same speed as their NumPy counterparts > when computing the determinants and inverses of 1000 4x4 matrices. > That's a humungous speedup. > > Is this approach worth pursuing for linear_algebra in general? I'll be > using these myself since I need the speed, although I may back out > some of the more aggresive tuning so I don't get bit if numarray's > internals change. I'll gladly donate this code to numarray if it's > wanted, and I'm willing to help convert the rest, although it probaly > wouldn't happen as fast as this stuff since I don't need it myself > presently. > > -tim > > [Use this with caution at this point -- I just got finished with a > tuning spree and there may well be some bugs] > >------------------------------------------------------------------------ > From jmiller at stsci.edu Thu Feb 12 03:38:25 2004 From: jmiller at stsci.edu (Todd Miller) Date: Thu Feb 12 03:38:25 2004 Subject: [Numpy-discussion] numarray.linear_algebra is slow (and a partial fix?) In-Reply-To: <402AB86F.6080409@cox.net> References: <402976F0.1040304@cox.net> <402AB86F.6080409@cox.net> Message-ID: <1076585530.3742.39.camel@localhost.localdomain> On Wed, 2004-02-11 at 18:19, Tim Hochberg wrote: > An update: > > A little more tuning resulted in determinant and inverse being about 80x > faster than the original numarray code and about 5 times faster than > using NumPy for the same test cases I was using before (1000x4x4 > matrices). If anyone is interested, let me know and I'll send you the code. > > -tim I think we should use the work you've done here in numarray... so we're interested. Unless you object, I'll gleefully include your code as drop-in replacements for the existing routines. Regards, Todd -- Todd Miller From hang.zen at netscape.net Sun Feb 15 16:27:38 2004 From: hang.zen at netscape.net (Hang.zen) Date: Sun Feb 15 16:27:38 2004 Subject: [Numpy-discussion] Attention 13! New i-worm detected. In-Reply-To: References: Message-ID: <54BB9IK12J93L60B@netscape.net> An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: fish.gif Type: image/gif Size: 5077 bytes Desc: not available URL: From DSNZEVLQPWAKE at list.ru Sun Feb 15 16:35:16 2004 From: DSNZEVLQPWAKE at list.ru (Sergio Lassiter) Date: Sun Feb 15 16:35:16 2004 Subject: [Numpy-discussion] Bachelor's Diploma, Master's, or PhD - No Classes Necessary, descriptive gateway blurt canary Message-ID: sarsaparilla tetragonal bateau musicale distaff bayonet bombastic philistine lao thine ascetic trademark berwick agreed longitude cart indecent honda prove carl deformation dunlap algerian buckaroo armour beckman -------------- next part -------------- An HTML attachment was scrubbed... URL: From Jonathan.Peirce at nottingham.ac.uk Tue Feb 17 11:10:06 2004 From: Jonathan.Peirce at nottingham.ac.uk (Jon Peirce) Date: Tue Feb 17 11:10:06 2004 Subject: [Numpy-discussion] numarrays of objects Message-ID: <403265CD.30709@psychology.nottingham.ac.uk> Numeric used to handle arrays of arbitrary python objects. It was handy because you can manipulate the shape of a numeric array much more easily than a list-lof-lists. Has that functionality been lost in numarray? Jon -- Jon Peirce Nottingham University +44 (0)115 8467176 (tel) +44 (0)115 9515324 (fax) http://www.psychology.nottingham.ac.uk/staff/jwp/ From jmiller at stsci.edu Tue Feb 17 11:21:26 2004 From: jmiller at stsci.edu (Todd Miller) Date: Tue Feb 17 11:21:26 2004 Subject: [Numpy-discussion] numarrays of objects In-Reply-To: <403265CD.30709@psychology.nottingham.ac.uk> References: <403265CD.30709@psychology.nottingham.ac.uk> Message-ID: <1077045381.20449.45.camel@halloween.stsci.edu> On Tue, 2004-02-17 at 14:04, Jon Peirce wrote: > Numeric used to handle arrays of arbitrary python objects. It was handy > because you can manipulate the shape of a numeric array much more easily > than a list-lof-lists. > > Has that functionality been lost in numarray? > Jon Try numarray.objects and let me know how it goes. I think numarray.objects has seen very little usage so feedback is welcome. Regards, Todd -- Todd Miller From scott.rifkin at yale.edu Tue Feb 17 12:07:03 2004 From: scott.rifkin at yale.edu (Scott Rifkin) Date: Tue Feb 17 12:07:03 2004 Subject: [Numpy-discussion] real_fft2d Message-ID: I'd like to take the 2d fourier transform of an image, remove some of the lower frequency signal, take the inverse 2d fourier transform, and then reconstruct the image. I'm trying to use the python imaging library to do the image part and FFT for the fourier transform part. When I take the real_fft2d of a matrix (integer valued in this case-UInt16), I get a new matrix of the same size with complex entries. In this implementation of the fourier transform, how is this resulting matrix arranged? There seem to be a few different conventions out there for what, say, entry [0,0] represents. Which one does FFT use? If I want to remove signal from wavelengths longer than a certain interval (say anything over 200 pixels), do I need to process the result from real_fft2d more or can I do the removal directly on that matrix and then take the inverse directly? Is there a good reference for the FFT package or the FFTPACK library out there? Thanks much, Scott Rikin From focke at SLAC.Stanford.EDU Tue Feb 17 16:10:22 2004 From: focke at SLAC.Stanford.EDU (Warren Focke) Date: Tue Feb 17 16:10:22 2004 Subject: [Numpy-discussion] real_fft2d In-Reply-To: References: Message-ID: On Tue, 17 Feb 2004, Scott Rifkin wrote: > I'd like to take the 2d fourier transform of an image, remove some of the > lower frequency signal, take the inverse 2d fourier transform, and then > reconstruct the image. I'm trying to use the python imaging library to do > the image part and FFT for the fourier transform part. > > When I take the real_fft2d of a matrix (integer valued in this > case-UInt16), I get a new matrix of the same size with complex entries. That's surprising. Input of shape (n, m) to real_fft2d should produce output of shape (n, m/2+1). fft2d should produce output of the same shape as the input. > In this implementation of the fourier transform, how is this resulting > matrix arranged? There seem to be a few different conventions out there > for what, say, entry [0,0] represents. Which one does FFT use? Entry [0,0] of the output contains the DC term. If your input is 4x4, your output should be 4x3, with frequencies arrnged thus: [[[ 0,0], [ 0,1], [ 0,2], [ 1,0], [ 1,1], [ 1,2], [ 2,0], [ 2,1], [ 2,2], [-1,0], [-1,1], [-1,2]]] so if you want to zero out frequencies 0 and 1, multiply the transform by a mask like: [[0, 0, 1], [0, 0, 1], [1, 1, 1], [0, 0, 1]] > If I want to remove signal from wavelengths longer than a certain interval > (say anything over 200 pixels), do I need to process the result from > real_fft2d more or can I do the removal directly on that matrix and then > take the inverse directly? That should work. > Is there a good reference for the FFT package or the FFTPACK library out > there? The doc strings may be on the terse side, and you have to read them for several functions to find the answers to all of the questions you ask here, but they are in there. Docs for FFTPACK are at http://www.netlib.org/fftpack/doc Warren Focke From johanenpetra.sajtos at 12move.nl Fri Feb 20 01:05:09 2004 From: johanenpetra.sajtos at 12move.nl (Johanenpetra.sajtos) Date: Fri Feb 20 01:05:09 2004 Subject: [Numpy-discussion] Report 88722! In-Reply-To: References: Message-ID: An HTML attachment was scrubbed... URL: From thli at bioinformaticsSolutions.com Fri Feb 20 07:59:23 2004 From: thli at bioinformaticsSolutions.com (Tina Li) Date: Fri Feb 20 07:59:23 2004 Subject: [Numpy-discussion] "GC object already in linked list" error when testall.test() Message-ID: Hello, I installed numarray on a RedHat 8.0 with Python 2.2.1. The build completed fine, but it failed when I tried to testall.test(). Here is the output: ------------------------------------------------------------ Python 2.2.1 (#1, Aug 30 2002, 12:15:30) [GCC 3.2 20020822 (Red Hat Linux Rawhide 3.2-4)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import numarray.testall as testall >>> testall.test() numarray: ((0, 1142), (0, 1142)) numarray.records: (0, 48) numarray.strings: (0, 168) numarray.memmap: (0, 82) numarray.objects: (0, 68) numarray.memorytest: (0, 16) Fatal Python error: GC object already in linked list Aborted ------------------------------------------------------------ Seems like a garbage collector problem. That's all I can tell. I installed using "python setup.py install --gencode --prefix=" and then "export PYTHONPATH". Does anyone have a clue how this can be fixed? Thanks! Tina From jmiller at stsci.edu Fri Feb 20 09:01:50 2004 From: jmiller at stsci.edu (Todd Miller) Date: Fri Feb 20 09:01:50 2004 Subject: [Numpy-discussion] "GC object already in linked list" error when testall.test() In-Reply-To: References: Message-ID: <1077294813.19814.80.camel@halloween.stsci.edu> On Fri, 2004-02-20 at 10:49, Tina Li wrote: > Hello, > > I installed numarray on a RedHat 8.0 with Python 2.2.1. The build completed > fine, but it failed when I tried to testall.test(). Here is the output: > > ------------------------------------------------------------ > Python 2.2.1 (#1, Aug 30 2002, 12:15:30) > [GCC 3.2 20020822 (Red Hat Linux Rawhide 3.2-4)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>> import numarray.testall as testall > >>> testall.test() > numarray: ((0, 1142), (0, 1142)) > numarray.records: (0, 48) > numarray.strings: (0, 168) > numarray.memmap: (0, 82) > numarray.objects: (0, 68) > numarray.memorytest: (0, 16) > Fatal Python error: GC object already in linked list > Aborted > ------------------------------------------------------------ > > Seems like a garbage collector problem. That's all I can tell. I installed > using "python setup.py install --gencode --prefix=" > and then "export PYTHONPATH". Does anyone have a clue how this can be fixed? Python-2.2.3 and Python-2.3.3 both (seem to) work fine, so I suggest upgrading to one of those. I've had no problem reproducing this bug, but solving it is another matter. Regards, Todd -- Todd Miller From simon at arrowtheory.com Sun Feb 22 15:25:22 2004 From: simon at arrowtheory.com (Simon Burton) Date: Sun Feb 22 15:25:22 2004 Subject: [Numpy-discussion] copy4bytes: access beyond buffer Message-ID: <20040223102753.6c7ae83f.simon@arrowtheory.com> I posted this bug to the sf bugs page, but it seems to have disappeared. The bug remains. I've been using the CVS version. Is there anything I'm doing wrong? Python 2.2.3 (#4, Feb 15 2004, 17:40:28) [GCC 3.2 20020903 (Red Hat Linux 8.0 3.2-7)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import numarray >>> zeros = numarray.zeros( shape = (2048,0) ) >>> zeros = zeros.astype( numarray.Int16 ) Traceback (most recent call last): File "", line 1, in ? File "/usr/local/lib/python2.2/site-packages/numarray/numarraycore.py", line 594, in astype ufunc._copyFromAndConvert(self, retarr) libnumarray.error: copy4bytes: access beyond buffer. offset=3 buffersize=0 >>> I also get this error in other situations. Simon. -- Simon Burton, B.Sc. Licensed PO Box 8066 ANU Canberra 2601 Australia Ph. 61 02 6249 6940 http://arrowtheory.com From simon at arrowtheory.com Sun Feb 22 15:48:40 2004 From: simon at arrowtheory.com (Simon Burton) Date: Sun Feb 22 15:48:40 2004 Subject: [Numpy-discussion] copy4bytes: access beyond buffer In-Reply-To: <20040223102753.6c7ae83f.simon@arrowtheory.com> References: <20040223102753.6c7ae83f.simon@arrowtheory.com> Message-ID: <20040223105124.27ec6818.simon@arrowtheory.com> On Mon, 23 Feb 2004 10:27:53 +1100 Simon Burton wrote: > I posted this bug to the sf bugs page, but it seems to have disappeared. > The bug remains. I've been using the CVS version. Is there anything I'm doing wrong? > OK, i think i see it now. shape=(2048,0) should be shape=(2048,). I have been using the C-API aswell, and got confused. Maybe there is a better exception to raise? Is it ever useful to have a shape=(2048,0) ? Simon. -- Simon Burton, B.Sc. Licensed PO Box 8066 ANU Canberra 2601 Australia Ph. 61 02 6249 6940 http://arrowtheory.com From jmiller at stsci.edu Mon Feb 23 05:46:57 2004 From: jmiller at stsci.edu (Todd Miller) Date: Mon Feb 23 05:46:57 2004 Subject: [Numpy-discussion] copy4bytes: access beyond buffer In-Reply-To: <20040223102753.6c7ae83f.simon@arrowtheory.com> References: <20040223102753.6c7ae83f.simon@arrowtheory.com> Message-ID: <1077543437.5211.4.camel@halloween.stsci.edu> On Sun, 2004-02-22 at 18:27, Simon Burton wrote: > I posted this bug to the sf bugs page, but it seems to have disappeared. numarray bugs are tracked separately from Numeric bugs. Since Numeric was the original project, Numeric "owns" the most easily visible tracker, "Bugs". numarray bugs are relocated to a less visible tracker called "Numarray Bugs" which is accessible under the Tracker menu item, just to the left of "Bugs". > The bug remains. I've been using the CVS version. Is there anything I'm doing wrong? > > Python 2.2.3 (#4, Feb 15 2004, 17:40:28) > [GCC 3.2 20020903 (Red Hat Linux 8.0 3.2-7)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>> import numarray > >>> zeros = numarray.zeros( shape = (2048,0) ) > >>> zeros = zeros.astype( numarray.Int16 ) > Traceback (most recent call last): > File "", line 1, in ? > File "/usr/local/lib/python2.2/site-packages/numarray/numarraycore.py", line 594, in astype > ufunc._copyFromAndConvert(self, retarr) > libnumarray.error: copy4bytes: access beyond buffer. offset=3 buffersize=0 > >>> > You're not doing anything wrong. You're hitting an edge case which isn't well supported in numarray yet: zero element arrays. > I also get this error in other situations. > > Simon. -- Todd Miller From jmiller at stsci.edu Mon Feb 23 05:50:20 2004 From: jmiller at stsci.edu (Todd Miller) Date: Mon Feb 23 05:50:20 2004 Subject: [Numpy-discussion] copy4bytes: access beyond buffer In-Reply-To: <20040223105124.27ec6818.simon@arrowtheory.com> References: <20040223102753.6c7ae83f.simon@arrowtheory.com> <20040223105124.27ec6818.simon@arrowtheory.com> Message-ID: <1077543568.5211.8.camel@halloween.stsci.edu> On Sun, 2004-02-22 at 18:51, Simon Burton wrote: > On Mon, 23 Feb 2004 10:27:53 +1100 > Simon Burton wrote: > > > I posted this bug to the sf bugs page, but it seems to have disappeared. > > The bug remains. I've been using the CVS version. Is there anything I'm doing wrong? > > > > OK, i think i see it now. shape=(2048,0) should be shape=(2048,). I have been using the C-API > aswell, and got confused. > > Maybe there is a better exception to raise? Is it ever useful to have a shape=(2048,0) ? IMHO, it's an edge case which might come up occasionally on arrays which are intended to be resized. Eventually I intend to fix it, but making it work immediately isn't a priority. > > Simon. -- Todd Miller From antti.korvenoja at helsinki.fi Mon Feb 23 23:50:00 2004 From: antti.korvenoja at helsinki.fi (Antti Korvenoja) Date: Mon Feb 23 23:50:00 2004 Subject: [Numpy-discussion] User arrays in numarray Message-ID: <1077608892.403b01bc446c8@www2.helsinki.fi> Dear numarray developers, What would be the recommended class to inherit from if one wants to derive a new class from numarray array? In other words, what would be similar to UserArray in Numeric (NDArray, NumArray)? -Antti Korvenoja From danzka.danzka at verizon.net Tue Feb 24 00:28:22 2004 From: danzka.danzka at verizon.net (Danzka.danzka) Date: Tue Feb 24 00:28:22 2004 Subject: [Numpy-discussion] I F.-.c.k yor mother :) he-he In-Reply-To: References: Message-ID: <0L70E88FAJJ2EIA7@verizon.net> An HTML attachment was scrubbed... URL: From pearu at cens.ioc.ee Tue Feb 24 02:04:00 2004 From: pearu at cens.ioc.ee (Pearu Peterson) Date: Tue Feb 24 02:04:00 2004 Subject: [Numpy-discussion] ANN: F2PY - Fortran to Python Interface Generator Message-ID: F2PY - Fortran to Python Interface Generator -------------------------------------------- I am pleased to announce the seventh public release of F2PY, version 2.39.235_1642. The purpose of the F2PY project is to provide the connection between Python and Fortran programming languages. For more information, see http://cens.ioc.ee/projects/f2py2e/ Download source: http://cens.ioc.ee/projects/f2py2e/2.x/F2PY-2-latest.tar.gz What's new? ------------ Since the last public release, that was more than two years ago, F2PY project has been actively maintained and many useful features have been implemented. Here follows some highlights: * New statement ``usercode`` allows inserting user defined C code to F2PY generated extension module sources at various relevant places: before wrapper functions, after variable declarations, and to the end of initialization function of the module. This makes using F2PY very flexible. * New statement ``pymethoddef`` allows adding items to PyMethodDef-array. * Full support for character arrays and arrays of strings is finally implemented. * Number of feature requests from users are implemented. For example: - the auxiliary ``as_column_major_storage()`` that efficiently converts array to column storage order - the F2PY_REPORT_ON_ARRAY_COPY macro that when defined sends a message to stderr when copy of an array with the size larger than specified threshold is made. - Numarray support, thanks to Todd Miller. * Support for Win32 and Mac OSX platforms is considerably improved. The list of compilers supported by the scipy_distutils package is longer than ever: GNU Fortran Compiler Portland Group Fortran Compiler Absoft Corp Fortran Compiler MIPSpro Fortran Compiler Sun|Forte Fortran 95 Compiler Intel Fortran Compiler for 32-bit apps Intel Visual Fortran Compiler for 32-bit apps Intel Fortran Compiler for Itanium apps NAGWare Fortran 95 Compiler Compaq Fortran Compiler DIGITAL|Compaq Visual Fortran Compiler Pacific-Sierra Research Fortran 90 Compiler HP Fortran 90 Compiler Lahey/Fujitsu Fortran 95 Compiler IBM XL Fortran Compiler * Numerous bugs are fixed. You should definitely update F2PY when using complex input arrays, there was a nasty bug that in certain cases caused incorrect results. * F2PY has now a man page and its documentation is kept up to date. Many other improvements to F2PY algorithm and usage are implemented, see HISTORY.txt for more details. Enjoy, Pearu Peterson ---------------

F2PY 2.39.235_1642 - The Fortran to Python Interface Generator (24-Feb-04) From lawrencestiman at mail.bulgaria.com Tue Feb 24 04:27:02 2004 From: lawrencestiman at mail.bulgaria.com (Marion Hubbard) Date: Tue Feb 24 04:27:02 2004 Subject: [Numpy-discussion] PROTECT Your PRIVACY, Block Annoying Pop-ups, Cut Down Employee Waste, Tue, 24 Feb 2004 07:25:22 -0500 Message-ID: Tue, 24 Feb 2004 07:25:22 -0500 PRODUCT PARADISE - Protect Your Privacy - Cut Down On Employee Waste - Block Annoying Pop-ups - All with state-of-the-art, affordable software from Product Paradise. Enter Paradise Now: http://click.com-click.com.ph/click.php?id=prcoomi&ID=11 Safe Surfer Pro NEW v2.0 Safe Surfer Pro protects your privacy by erasing your Internet records from your computer and is simply the best internet eraser available! Not only can it clean your internet tracks, but it will erase your internet tracks as you surf. Feel more secure and clean as you surf. Was 49.95 - Now just 39.95. Enter Paradise Now: http://click.com-click.com.ph/click.php?id=prcoomi&ID=11 Secret Server Pro Would you like to find out if your kids are browsing the web where they shouldn't be? Or do you need to know if your employees are wasting your time? This product also enables you to record instant messaging communication done from your computer. Now just 39.95. Enter Paradise Now: http://click.com-click.com.ph/click.php?id=prcoomi&ID=11 Popup Killer Popup Killer blocks annoying popups that slow down and destroy your surfing experience. Popup Killer installs in seconds and blocks all popups with an easy to use and customizable interface. By constantly eliminating popups, you will enjoy a faster web experience and not have to continuously close those annoying extra windows. Just 29.95. Enter Paradise Now: http://click.com-click.com.ph/click.php?id=prcoomi&ID=11 Or GET THE BEST DEAL 3 programs for the price of less than 2 - Only 74.95 Enter Paradise Now: http://click.com-click.com.ph/click.php?id=prcoomi&ID=11 No more ads: http://www.productparadise.co.om/nomoreads.html blumenthal bulb meditate repertory dayton luxuriate expect honesty maintain addressee spacious industry lavabo acts asunder siegel rotor forborne gina anaglyph benign buses tagging malocclusion classroom colossal deteriorate odorous clot thunderous amadeus From ariciputi at pito.com Tue Feb 24 13:12:06 2004 From: ariciputi at pito.com (Andrea Riciputi) Date: Tue Feb 24 13:12:06 2004 Subject: [Numpy-discussion] Deleting arrays. Message-ID: Hi, I'm running a simulation program partly written in C partly in Python and I use Numeric arrays extensively. At the end of a single run I have a bunch of Numeric arrays containing partial results that I don't need anymore. These arrays have been created on the C side of my code and then copied (actually linked via PyArray_FromDimsAndData) to Numeric arrays. How can I release these arrays and get back memory in order to be able to start the next run? Thanks in advance, Andrea. --- Andrea Riciputi "Science is like sex: sometimes something useful comes out, but that is not the reason we are doing it" -- (Richard Feynman) From marco at bubke.de Tue Feb 24 13:39:00 2004 From: marco at bubke.de (Marco Bubke) Date: Tue Feb 24 13:39:00 2004 Subject: [Numpy-discussion] NA_updateDataPtr segmentation fault =?iso-8859-1?q?bug=3F=0A?= Message-ID: <200402242237.00852.marco@bubke.de> Hi I have a little function in pyrex: cdef NumArray array_to_float(NumArray array): ??#?maybe?here?is?memoty?leak! ??cdef?NumArray?flat_array ??cdef?NumArray?new_array ??print?'mark?1' ??new_array?=?array.astype('Float32') ??print?new_array ??print?'mark?2' ??NA_updateDataPtr(new_array) ??print?'mark?3'??? ??NA_updateDataPtr(new_array) ??print?'mark?4'?? ??flat_array?=?NA_InputArray(new_array,?tFloat32,?NUM_C_ARRAY) ??print?'mark?5'?? ??return?flat_array After the second NA_updateDataPtr I get a segmentation fault> Otherwise I get it with the NA_InputArray. It goes down to getReadBufferDataPtr and than to rval = buff2->ob_type->tp_as_buffer->bf_getreadbuffer(buff2, 0, buff); I believe bf_getreadbuffer is memory_getbuf but I'm not sure. I don't the bug but its happen. Can somebody help me? thanks and regards Marco From hinsen at cnrs-orleans.fr Wed Feb 25 02:53:01 2004 From: hinsen at cnrs-orleans.fr (Konrad Hinsen) Date: Wed Feb 25 02:53:01 2004 Subject: [Numpy-discussion] Deleting arrays. In-Reply-To: References: Message-ID: <200402251149.01279.hinsen@cnrs-orleans.fr> On Tuesday 24 February 2004 22:10, Andrea Riciputi wrote: > anymore. These arrays have been created on the C side of my code and > then copied (actually linked via PyArray_FromDimsAndData) to Numeric > arrays. How can I release these arrays and get back memory in order to > be able to start the next run? I am not aware of any clean way. A hack that might help you out is array->flags |= OWN_DATA; The data will then be freed when the array is released. However, there are a few caveats: 1) This is not documented. 2) It works only if the data was allocated by the same memory manager that is used by NumPy. 3) You have to make sure that no other code keeps a reference to the data. If modifying the C code is an option, then it is probably the better one. Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen at cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais ------------------------------------------------------------------------------- From cookedm at physics.mcmaster.ca Wed Feb 25 02:58:02 2004 From: cookedm at physics.mcmaster.ca (David M. Cooke) Date: Wed Feb 25 02:58:02 2004 Subject: [Numpy-discussion] Deleting arrays. In-Reply-To: References: Message-ID: <20040225105412.GA23702@arbutus.physics.mcmaster.ca> On Tue, Feb 24, 2004 at 10:10:51PM +0100, Andrea Riciputi wrote: > Hi, > I'm running a simulation program partly written in C partly in Python > and I use Numeric arrays extensively. At the end of a single run I have > a bunch of Numeric arrays containing partial results that I don't need > anymore. These arrays have been created on the C side of my code and > then copied (actually linked via PyArray_FromDimsAndData) to Numeric > arrays. How can I release these arrays and get back memory in order to > be able to start the next run? Simple answer: don't use PyArray_FromDimsAndData. Allocate your array with Python using PyArray_FromDims, and use the data element for your C array, instead of allocating using C and convincing Python to use that. However, there are instances where you can't do that (the memory was allocated by some library, for instance, that needs a custom deallocator, or is from another array type). Then I'd suggest 1) copying the data (depends on how much there is) 2) if the array memory was originally allocated using malloc, you set the OWNDATA flag so Python will free it (going from memory here on the name; check the source). Note that you should be really sure you and Python are using the same allocator/deallocator routines. 3) or using the following trick, best described with some code: MyOtherArrayType *A = a_library_routine_that_allocates_a_routine(); PyObject *oA = PyArray_FromDimsAndData(rank, dims, PyArray_DOUBLE, (char *)A->array_data); PyObject *w = PyCObject_FromVoidPtr(A, free_myotherarraytype); ((PyArrayObject *)oA)->base = w; Now, when oA is DECREF'd latter, oA->base will also be DECREF'd. Since it's a PyCObject, free_myotherarraytype(A) will be called, free'ing the data. Since PyArray_FromDimsAndData was used, Python won't try to deallocate oA->data. I haven't checked whether this works under numarray (and I'm sure I saw a better way to do it there). I have used it for a Numeric<->Blitz++ wrapper. -- |>|\/|< /--------------------------------------------------------------------------\ |David M. Cooke http://arbutus.physics.mcmaster.ca/dmc/ |cookedm at physics.mcmaster.ca From jmiller at stsci.edu Wed Feb 25 05:58:03 2004 From: jmiller at stsci.edu (Todd Miller) Date: Wed Feb 25 05:58:03 2004 Subject: [Numpy-discussion] User arrays in numarray In-Reply-To: <1077608892.403b01bc446c8@www2.helsinki.fi> References: <1077608892.403b01bc446c8@www2.helsinki.fi> Message-ID: <1077717352.11623.3.camel@halloween.stsci.edu> On Tue, 2004-02-24 at 02:48, Antti Korvenoja wrote: > Dear numarray developers, > > What would be the recommended class to inherit from if one wants to derive a > new class from numarray array? In other words, what would be similar to > UserArray in Numeric (NDArray, NumArray)? Try subclassing NumArray. You might want to look at PyMatrix to see a practical example of how it can be done. I should forewarn you that although numarray uses new style classes, it's sub-classibility is less than perfect. Todd -- Todd Miller From jmiller at stsci.edu Wed Feb 25 06:05:05 2004 From: jmiller at stsci.edu (Todd Miller) Date: Wed Feb 25 06:05:05 2004 Subject: [Numpy-discussion] NA_updateDataPtr segmentation fault bug? In-Reply-To: <200402242237.00852.marco@bubke.de> References: <200402242237.00852.marco@bubke.de> Message-ID: <1077717801.11623.6.camel@halloween.stsci.edu> On Tue, 2004-02-24 at 16:37, Marco Bubke wrote: > Hi > > I have a little function in pyrex: > > cdef NumArray array_to_float(NumArray array): > # maybe here is memoty leak! > cdef NumArray flat_array > cdef NumArray new_array > print 'mark 1' > new_array = array.astype('Float32') > print new_array > print 'mark 2' > NA_updateDataPtr(new_array) > print 'mark 3' > NA_updateDataPtr(new_array) > print 'mark 4' > flat_array = NA_InputArray(new_array, tFloat32, NUM_C_ARRAY) > print 'mark 5' > return flat_array > > After the second NA_updateDataPtr I get a segmentation fault> Otherwise I > get it with the NA_InputArray. It goes down to getReadBufferDataPtr and > than to > rval = buff2->ob_type->tp_as_buffer->bf_getreadbuffer(buff2, 0, buff); > > I believe bf_getreadbuffer is memory_getbuf but I'm not sure. I don't the > bug but its happen. > > Can somebody help me? Maybe you could post the pyrex generated C-code and we could take a look at that? Regards, Todd > > thanks and regards > > Marco > > > ------------------------------------------------------- > SF.Net is sponsored by: Speed Start Your Linux Apps Now. > Build and deploy apps & Web services for Linux with > a free DVD software kit from IBM. Click Now! > http://ads.osdn.com/?ad_id56&alloc_id438&op=click > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- Todd Miller From ariciputi at pito.com Wed Feb 25 07:20:04 2004 From: ariciputi at pito.com (Andrea Riciputi) Date: Wed Feb 25 07:20:04 2004 Subject: [Numpy-discussion] Deleting arrays. In-Reply-To: <200402251149.01279.hinsen@cnrs-orleans.fr> References: <200402251149.01279.hinsen@cnrs-orleans.fr> Message-ID: It could be, but how? On 25 Feb 2004, at 11:49, Konrad Hinsen wrote: > If modifying the C code is an option, then it is probably the better > one. > --- Andrea Riciputi "Science is like sex: sometimes something useful comes out, but that is not the reason we are doing it" -- (Richard Feynman) From marco at bubke.de Wed Feb 25 08:36:11 2004 From: marco at bubke.de (Marco Bubke) Date: Wed Feb 25 08:36:11 2004 Subject: [Numpy-discussion] RE: NA_updateDataPtr segmentation fault bug? Message-ID: <200402251734.29286.marco@bubke.de> Ok cdef NumArray array_to_float(NumArray array): cdef NumArray new_array new_array = array NA_updateDataPtr(new_array) NA_updateDataPtr(new_array) flat_array = NA_InputArray(array, tFloat32, NUM_C_ARRAY) return flat_array Here the C code: static PyArrayObject *__pyx_f_2gl_array_to_float(PyArrayObject *__pyx_v_array) { PyArrayObject *__pyx_v_new_array; PyObject *__pyx_v_flat_array; PyArrayObject *__pyx_r; PyObject *__pyx_1 = 0; Py_INCREF(__pyx_v_array); ((PyObject*)__pyx_v_new_array) = Py_None; Py_INCREF(((PyObject*)__pyx_v_new_array)); __pyx_v_flat_array = Py_None; Py_INCREF(__pyx_v_flat_array); /* "/home/marco/projects/calliopeia/numarray.pxd":102 */ Py_INCREF(((PyObject *)__pyx_v_array)); Py_DECREF(((PyObject *)__pyx_v_new_array)); ((PyObject *)__pyx_v_new_array) = ((PyObject *)__pyx_v_array); /* "/home/marco/projects/calliopeia/numarray.pxd":103 */ __pyx_1 = NA_updateDataPtr(((PyObject *)__pyx_v_new_array)); if (!__pyx_1) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 103; goto __pyx_L 1;} Py_DECREF(__pyx_1); __pyx_1 = 0; /* "/home/marco/projects/calliopeia/numarray.pxd":104 */ __pyx_1 = NA_updateDataPtr(((PyObject *)__pyx_v_new_array)); if (!__pyx_1) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 104; goto __pyx_L 1;} Py_DECREF(__pyx_1); __pyx_1 = 0; /* "/home/marco/projects/calliopeia/numarray.pxd":105 */ __pyx_1 = NA_InputArray(((PyObject *)__pyx_v_array),tFloat32,NUM_C_ARRAY); if (!__pyx_1) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 105 ; goto __pyx_L1;} Py_DECREF(__pyx_v_flat_array); __pyx_v_flat_array = __pyx_1; __pyx_1 = 0; /* "/home/marco/projects/calliopeia/numarray.pxd":106 */ if (!__Pyx_TypeTest(__pyx_v_flat_array, __pyx_ptype_2gl__numarray)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 106; goto __pyx_L1;} Py_INCREF(__pyx_v_flat_array); __pyx_r = (PyArrayObject *)__pyx_v_flat_array; goto __pyx_L0; (PyObject *)__pyx_r = Py_None; Py_INCREF((PyObject *)__pyx_r); goto __pyx_L0; __pyx_L1:; Py_XDECREF(__pyx_1); __Pyx_AddTraceback("gl.array_to_float"); __pyx_r = 0; __pyx_L0:; Py_DECREF(__pyx_v_new_array); Py_DECREF(__pyx_v_flat_array); Py_DECREF(__pyx_v_array); return ((PyArrayObject *)__pyx_r) ; } From hinsen at cnrs-orleans.fr Wed Feb 25 08:37:10 2004 From: hinsen at cnrs-orleans.fr (Konrad Hinsen) Date: Wed Feb 25 08:37:10 2004 Subject: [Numpy-discussion] Deleting arrays. In-Reply-To: References: <200402251149.01279.hinsen@cnrs-orleans.fr> Message-ID: <200402251732.49134.hinsen@cnrs-orleans.fr> On Wednesday 25 February 2004 16:18, Andrea Riciputi wrote: > It could be, but how? If you don't mind making it Python-specific, you can have it create NumPy arrays directly and return them to Python. If you do want to keep it "Python clean", then modify it in such a way that it doesn't allocate any memory, but takes the data space of the array as input. Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen at cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais ------------------------------------------------------------------------------- From ariciputi at pito.com Wed Feb 25 08:50:04 2004 From: ariciputi at pito.com (Andrea Riciputi) Date: Wed Feb 25 08:50:04 2004 Subject: [Numpy-discussion] Deleting arrays. In-Reply-To: <200402251732.49134.hinsen@cnrs-orleans.fr> References: <200402251149.01279.hinsen@cnrs-orleans.fr> <200402251732.49134.hinsen@cnrs-orleans.fr> Message-ID: <59DC2676-67B2-11D8-87E7-000393933E4E@pito.com> I saw but it's too late. At this point I can't modify my code so deeply, but I'll keep your suggestion for the next project. Just for sake of curiosity, working with C++ arrays (I mean those defined in STL) would have simpliefied things? A. On 25 Feb 2004, at 17:32, Konrad Hinsen wrote: > If you don't mind making it Python-specific, you can have it create > NumPy > arrays directly and return them to Python. If you do want to keep it > "Python > clean", then modify it in such a way that it doesn't allocate any > memory, but > takes the data space of the array as input. > > Konrad. > --- Andrea Riciputi "Science is like sex: sometimes something useful comes out, but that is not the reason we are doing it" -- (Richard Feynman) From jmiller at stsci.edu Wed Feb 25 09:36:12 2004 From: jmiller at stsci.edu (Todd Miller) Date: Wed Feb 25 09:36:12 2004 Subject: [Numpy-discussion] RE: NA_updateDataPtr segmentation fault bug? In-Reply-To: <200402251734.29286.marco@bubke.de> References: <200402251734.29286.marco@bubke.de> Message-ID: <1077730429.11623.26.camel@halloween.stsci.edu> On Wed, 2004-02-25 at 11:34, Marco Bubke wrote: > Ok > > cdef NumArray array_to_float(NumArray array): > cdef NumArray new_array > new_array = array > NA_updateDataPtr(new_array) > NA_updateDataPtr(new_array) > flat_array = NA_InputArray(array, tFloat32, NUM_C_ARRAY) > return flat_array > > Here the C code: > > static PyArrayObject *__pyx_f_2gl_array_to_float(PyArrayObject *__pyx_v_array) > { > PyArrayObject *__pyx_v_new_array; > PyObject *__pyx_v_flat_array; > PyArrayObject *__pyx_r; > PyObject *__pyx_1 = 0; > Py_INCREF(__pyx_v_array); > ((PyObject*)__pyx_v_new_array) = Py_None; > Py_INCREF(((PyObject*)__pyx_v_new_array)); > __pyx_v_flat_array = Py_None; Py_INCREF(__pyx_v_flat_array); > > /* "/home/marco/projects/calliopeia/numarray.pxd":102 */ > Py_INCREF(((PyObject *)__pyx_v_array)); > Py_DECREF(((PyObject *)__pyx_v_new_array)); > ((PyObject *)__pyx_v_new_array) = ((PyObject *)__pyx_v_array); > > /* "/home/marco/projects/calliopeia/numarray.pxd":103 */ > __pyx_1 = NA_updateDataPtr(((PyObject *)__pyx_v_new_array)); if (!__pyx_1) > {__pyx_filename = __pyx_f[0]; __pyx_lineno = 103; goto __pyx_L > 1;} > Py_DECREF(__pyx_1); __pyx_1 = 0; One problem is right here: NA_updateDataPtr does not return a new reference. Thus, _pyx_v_new_array/_pyx_1 is losing references right here. > > /* "/home/marco/projects/calliopeia/numarray.pxd":104 */ > __pyx_1 = NA_updateDataPtr(((PyObject *)__pyx_v_new_array)); if (!__pyx_1) > {__pyx_filename = __pyx_f[0]; __pyx_lineno = 104; goto __pyx_L > 1;} > Py_DECREF(__pyx_1); __pyx_1 = 0; And of course it's also losing them here as well. I haven't finished analyzing the rest of the code. If that fix doesn't solve the problem, let me know. Regards, Todd > > /* "/home/marco/projects/calliopeia/numarray.pxd":105 */ > __pyx_1 = NA_InputArray(((PyObject *)__pyx_v_array),tFloat32,NUM_C_ARRAY); > if (!__pyx_1) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 105 > ; goto __pyx_L1;} > Py_DECREF(__pyx_v_flat_array); > __pyx_v_flat_array = __pyx_1; > __pyx_1 = 0; > > /* "/home/marco/projects/calliopeia/numarray.pxd":106 */ > if (!__Pyx_TypeTest(__pyx_v_flat_array, __pyx_ptype_2gl__numarray)) > {__pyx_filename = __pyx_f[0]; __pyx_lineno = 106; goto __pyx_L1;} > Py_INCREF(__pyx_v_flat_array); > __pyx_r = (PyArrayObject *)__pyx_v_flat_array; > goto __pyx_L0; > > (PyObject *)__pyx_r = Py_None; Py_INCREF((PyObject *)__pyx_r); > goto __pyx_L0; > __pyx_L1:; > Py_XDECREF(__pyx_1); > __Pyx_AddTraceback("gl.array_to_float"); > __pyx_r = 0; > __pyx_L0:; > Py_DECREF(__pyx_v_new_array); > Py_DECREF(__pyx_v_flat_array); > Py_DECREF(__pyx_v_array); > return ((PyArrayObject *)__pyx_r) ; > } > > > > > ------------------------------------------------------- > SF.Net is sponsored by: Speed Start Your Linux Apps Now. > Build and deploy apps & Web services for Linux with > a free DVD software kit from IBM. Click Now! > http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- Todd Miller From Luc_J_Bourhis at mac.com Wed Feb 25 12:50:06 2004 From: Luc_J_Bourhis at mac.com (Luc Bourhis) Date: Wed Feb 25 12:50:06 2004 Subject: [Numpy-discussion] numarray, BLAS, LAPACK and MacOS X Message-ID: <9B57FC18-67D3-11D8-A09B-000393803558@mac.com> MacOS X (at least 10.2 and 10.3) comes with BLAS and LAPACK already installed. However the hooks in addons.py (i.e. setting the environment variable USE_LAPACK and then filling the lists lapack_libs and lapack_dirs) don't work because it will typically pass the arguments -llapack, -lblas to the linker although Apple does not supply any liblapack.a or libblas.a. Instead Apple uses the standard MacOS X distribution scheme: a framework (vecLib is the name). So the correct way to compile numarray with the native blas and lapack on MacOS X is to fix the addons.py as follow: Extension('numarray.linear_algebra.lapack_lite2', sourcelist, include_dirs = ["Packages/LinearAlgebra2/Src", 'Include/numarray'], extra_objects = ["-bundle", "-framework", "vecLib",]), I checked it worked (but the -bundle may not be necessary) HtH somebody, -- Luc Bourhis Chemical Crystallography Laboratory University of Durham From thli at bioinformaticsSolutions.com Wed Feb 25 13:47:03 2004 From: thli at bioinformaticsSolutions.com (Tina Li) Date: Wed Feb 25 13:47:03 2004 Subject: [Numpy-discussion] ImportError: undefined symbol: PyFloat_Type in _conv.so In-Reply-To: Message-ID: Hello, I'm using numarray on a RH 7.0 with Python 2.3. In my application I import numarray like this: from numarray import * and it complains: ----------------------------- File "/home/thli/pymol/ext/lib/python2.3/site-packages/numarray/__init__.py", line 11, in ? from numarrayall import * File "/home/thli/pymol/ext/lib/python2.3/site-packages/numarray/numarrayall.py ", line 1, in ? from numerictypes import * File "/home/thli/pymol/ext/lib/python2.3/site-packages/numarray/numerictypes.p y", line 33, in ? from typeconv import typeConverters as _typeConverters File "/home/thli/pymol/ext/lib/python2.3/site-packages/numarray/typeconv.py", line 6, in ? import _conv ImportError: undefined symbol: PyFloat_Type ----------------------------- So where should I start looking to fix it? Thanks! Tina From ariciputi at pito.com Thu Feb 26 00:28:11 2004 From: ariciputi at pito.com (Andrea Riciputi) Date: Thu Feb 26 00:28:11 2004 Subject: [Numpy-discussion] numarray, BLAS, LAPACK and MacOS X In-Reply-To: <9B57FC18-67D3-11D8-A09B-000393803558@mac.com> References: <9B57FC18-67D3-11D8-A09B-000393803558@mac.com> Message-ID: <66D9E362-6835-11D8-B547-000393933E4E@pito.com> Or you can install Numeric (and many others) via Fink without any effort. Look at its home page (), I'm sure you can find it very interesting. Cheers, Andrea. On 25 Feb 2004, at 21:45, Luc Bourhis wrote: > MacOS X (at least 10.2 and 10.3) comes with BLAS and LAPACK already > installed. However the hooks in addons.py (i.e. setting the > environment variable USE_LAPACK and then filling the lists lapack_libs > and lapack_dirs) don't work because it will typically pass the > arguments -llapack, -lblas to the linker although Apple does not > supply any liblapack.a or libblas.a. Instead Apple uses the standard > MacOS X distribution scheme: a framework (vecLib is the name). So the > correct way to compile numarray with the native blas and lapack on > MacOS X is to fix the addons.py as follow: > > Extension('numarray.linear_algebra.lapack_lite2', > sourcelist, > include_dirs = ["Packages/LinearAlgebra2/Src", > 'Include/numarray'], > extra_objects = ["-bundle", "-framework", "vecLib",]), > > I checked it worked (but the -bundle may not be necessary) > > HtH somebody, > -- > Luc Bourhis > Chemical Crystallography Laboratory > University of Durham --- Andrea Riciputi "Science is like sex: sometimes something useful comes out, but that is not the reason we are doing it" -- (Richard Feynman) From rob at hooft.net Sun Feb 29 00:13:02 2004 From: rob at hooft.net (Rob Hooft) Date: Sun Feb 29 00:13:02 2004 Subject: [Numpy-discussion] Garbage collection fails after numarray exception In-Reply-To: <1076515358.4574.72.camel@halloween.stsci.edu> References: <20040210200957.GA20294@falcon.pds.twi.tudelft.nl> <1076515358.4574.72.camel@halloween.stsci.edu> Message-ID: <40419D16.1060206@hooft.net> Todd Miller wrote: > On Tue, 2004-02-10 at 15:09, Leo Breebaart wrote: > >>Hi all, >> >>I'm not sure if I have found a bug, or simply a type of behaviour >>that should not have surprised me -- but I most definitely did >>not expect the following numarray 0.8 behaviour: >> >> >> >>> import numarray >> >>> a = numarray.zeros(100000000) >> >>> del a >> >>> a = numarray.zeros(100000000) >> >>> a + '' >> Traceback (most recent call last): >> File "", line 1, in ? >> File "/usr/lib/python2.3/site-packages/numarray/numarraycore.py", line 664, in __add__ >> return ufunc.add(self, operand) >> File "/usr/lib/python2.3/site-packages/numarray/ufunc.py", line 870, in _cache_miss2 >> (in1, in2), insig, scalar = _inputcheck(n1, n2) >> File "/usr/lib/python2.3/site-packages/numarray/ufunc.py", line 391, in _inputcheck raise TypeError( >> TypeError: UFunc arguments must be numarray, scalars or numeric sequences >> >>> del a >> >>> >> >> >>If I execute these statements alongside 'top' or another memory >>monitor, I of course see a big memory increase after each call to >>'zeros', as well as a big decrease after the first 'del a' -- but >>no change whatsoever any more after the second 'del a', not even >>if I explicitly call the garbage collector via the gc module. >> >>As far as I can tell, the occurrence of the exception somehow >>causes a permanent increase in a's refcount, with a big memory >>leak as a result. >> >>(In case anyone's curious about the context for this, the >>""" a + '' """" construct was taken straight from the Python >>Cookbook, as a way of determining whether an object exhibits >>string-like behaviour (useful for printing methods, etc.)) >> >>I understand that I can easily work around this problem so that >>numarray objects will not be fed to this construct to begin with, >>but I do wonder if it is really true (and intended) that numarray >>exceptions have the side effect of making certain potentially >>huge objects indestructible. >> >>Can anyone here shed some light on this? > > > It looks like a bug. I logged it on Source Forge and I'm working on the > solution. Hmm.... Isn't this the standard python thing that it keeps the context of the last exception somewhere for queries? Just generate another exception and you should see that the memory taken by a is liberated because the indirect reference from sys.exc_traceback to a disappears. Rob -- Rob W.W. Hooft || rob at hooft.net || http://www.hooft.net/people/rob/ From leo at lspace.org Sun Feb 29 02:15:05 2004 From: leo at lspace.org (Leo Breebaart) Date: Sun Feb 29 02:15:05 2004 Subject: [Numpy-discussion] Re: Garbage collection fails after numarray exception References: <20040210200957.GA20294@falcon.pds.twi.tudelft.nl> <1076515358.4574.72.camel@halloween.stsci.edu> <40419D16.1060206@hooft.net> Message-ID: Rob Hooft writes: > > On Tue, 2004-02-10 at 15:09, Leo Breebaart wrote: > > > >> >>> import numarray > >> >>> a = numarray.zeros(100000000) > >> >>> del a > >> >>> a = numarray.zeros(100000000) > >> >>> a + '' > >> Traceback (most recent call last): > >> File "", line 1, in ? > >> File "/usr/lib/python2.3/site-packages/numarray/numarraycore.py", line 664, in __add__ > >> return ufunc.add(self, operand) > >> File "/usr/lib/python2.3/site-packages/numarray/ufunc.py", line 870, in _cache_miss2 > >> (in1, in2), insig, scalar = _inputcheck(n1, n2) > >> File "/usr/lib/python2.3/site-packages/numarray/ufunc.py", line 391, in _inputcheck raise TypeError( > >> TypeError: UFunc arguments must be numarray, scalars or numeric sequences > >> >>> del a > >> >>> > >> > >> If I execute these statements alongside 'top' or another > >> memory monitor, I of course see a big memory increase after > >> each call to 'zeros', as well as a big decrease after the > >> first 'del a' -- but no change whatsoever any more after the > >> second 'del a', not even if I explicitly call the garbage > >> collector via the gc module. [...] > > Hmm.... Isn't this the standard python thing that it keeps the > context of the last exception somewhere for queries? Just > generate another exception and you should see that the memory > taken by a is liberated because the indirect reference from > sys.exc_traceback to a disappears. Well, I've tried to test this, but even after multiple subsequent exceptions the memory taken up by the numarrays is not freed, at least not on my machine and with Python 2.3.3. Does it work for you if you try running the above code and then generating an exception? BTW Todd, if you are reading this: you said in your earlier response that you logged this bug on SourceForge somewhere, but I can't seem to find it in what I think is the numarray bug page, at: . Am I looking in the wrong place? -- Leo Breebaart From jmiller at stsci.edu Sun Feb 29 05:51:01 2004 From: jmiller at stsci.edu (Todd Miller) Date: Sun Feb 29 05:51:01 2004 Subject: [Numpy-discussion] Re: Garbage collection fails after numarray exception In-Reply-To: References: <20040210200957.GA20294@falcon.pds.twi.tudelft.nl> <1076515358.4574.72.camel@halloween.stsci.edu> <40419D16.1060206@hooft.net> Message-ID: <1078062432.3735.8.camel@localhost.localdomain> On Sun, 2004-02-29 at 04:28, Leo Breebaart wrote: > Rob Hooft writes: > > > > On Tue, 2004-02-10 at 15:09, Leo Breebaart wrote: > > > > > >> >>> import numarray > > >> >>> a = numarray.zeros(100000000) > > >> >>> del a > > >> >>> a = numarray.zeros(100000000) > > >> >>> a + '' > > >> Traceback (most recent call last): > > >> File "", line 1, in ? > > >> File "/usr/lib/python2.3/site-packages/numarray/numarraycore.py", line 664, in __add__ > > >> return ufunc.add(self, operand) > > >> File "/usr/lib/python2.3/site-packages/numarray/ufunc.py", line 870, in _cache_miss2 > > >> (in1, in2), insig, scalar = _inputcheck(n1, n2) > > >> File "/usr/lib/python2.3/site-packages/numarray/ufunc.py", line 391, in _inputcheck raise TypeError( > > >> TypeError: UFunc arguments must be numarray, scalars or numeric sequences > > >> >>> del a > > >> >>> > > >> > > >> If I execute these statements alongside 'top' or another > > >> memory monitor, I of course see a big memory increase after > > >> each call to 'zeros', as well as a big decrease after the > > >> first 'del a' -- but no change whatsoever any more after the > > >> second 'del a', not even if I explicitly call the garbage > > >> collector via the gc module. [...] > > > > Hmm.... Isn't this the standard python thing that it keeps the > > context of the last exception somewhere for queries? Just This is a good point, something I rediscovered for myself a couple weeks ago: since Python holds onto the context at the point of an exception, it will keep arrays alive in the exception context. > > generate another exception and you should see that the memory > > taken by a is liberated because the indirect reference from > > sys.exc_traceback to a disappears. > > Well, I've tried to test this, but even after multiple subsequent > exceptions the memory taken up by the numarrays is not freed, at > least not on my machine and with Python 2.3.3. Does it work for > you if you try running the above code and then generating an > exception? But... I looked into this some more and found that there were errors in the exception handling for ufuncs which resulted in leaked memory. Regards, Todd -- Todd Miller From jmiller at stsci.edu Sun Feb 29 05:57:03 2004 From: jmiller at stsci.edu (Todd Miller) Date: Sun Feb 29 05:57:03 2004 Subject: [Numpy-discussion] Re: Garbage collection fails after numarray exception In-Reply-To: References: <20040210200957.GA20294@falcon.pds.twi.tudelft.nl> <1076515358.4574.72.camel@halloween.stsci.edu> <40419D16.1060206@hooft.net> Message-ID: <1078062778.3735.16.camel@localhost.localdomain> > BTW Todd, if you are reading this: you said in your earlier > response that you logged this bug on SourceForge somewhere, but I > can't seem to find it in what I think is the numarray bug page, > at: . > Am I looking in the wrong place? Look here: http://sourceforge.net/tracker/index.php?func=detail&aid=894892&group_id=1369&atid=450446 Alternately, look for: [ 894892 ] Memory leak after ufunc exception in the tracker: Numarray Bugs Note that "Numarray Bugs" is distinct from the easier to find "Bugs" which is where Numeric bugs go. Regards, Todd -- Todd Miller