Mailman 3 September 2003 - NumPy-Discussion

numarray boolean array formatting poll results
by Todd Miller 23 Sep '03

23 Sep '03

The thread on what to do with boolean array formatting in numarray has died out, so I thought I should explain (what I saw) as the results of the poll: (1) There was a clear majority of numarray users who actually prefer the status quo of 0,1 for representing arrays of boolean values. (2) There was a sizable minority that would like to see consistency with Python-2.3 by using False, True for representing arrays of boolean values. (3) There was a (good) suggestion of using F, T for representing boolean values. However, on reflection and reviewing the poll comments, F, T is not really as good as 0, 1 visually, nor is it likely to satisfy those people interested in consistency with Python, so it is unlikely to make most numarray users any happier. (4) Also suggested was supporting configurable boolean formats. This approach is reasonable to implement, but I have a reservation that adding support for multiple formats may cause as much confusion as it gives satisfaction. Is it *really* worth it? (more later) Given 1-4, I think the best thing to do for now is keep 0,1 as the only format for boolean array values. If someone wants False, True bad enough to submit a complete patch for supporting multiple formats (and defaulting to 0,1), we will still accept it. If anyone is still interested in doing this, the patch should include modified TeX documentation and pass all the self tests in setupall.py (all of core numarray and all current add-ons). The patch should also run cleanly on both Python-2.2 and Python-2.3. Thanks everyone for responding, Todd -- Todd Miller jmiller(a)stsci.edu STSCI / ESS / SSB

1 0

[ANN] PyTables 0.7.2 released
by Francesc Alted 22 Sep '03

22 Sep '03

Announcing PyTables 0.7.2 ------------------------- PyTables is a hierarchical database package designed to efficently manage very large amounts of data. PyTables is built on top of the HDF5 library and the numarray package. It features an object-oriented interface that, combined with natural naming and C-code generated from Pyrex sources, makes it a fast, yet extremely easy to use tool for interactively save and retrieve large amounts of data. Besides, it provides flexible indexed access on disk to anywhere in the data you want to go. On this release you will not find any exciting new features. It is mainly a maintenance release where the next issues has been addressed: - a memory leak was fixed - memory consumption is being addressed and lowered - much faster opening of files - Some important index patterns cases in table reads has been optimized More in detail: What's new ----------- - Fixed a nasty memory leak located on the C libraries (it was happening during HDF5 attribute writes). After that, the memory consumption when using large object trees has dropped quite a bit. However, there remains some small leaks that has been tracked down to the underlying numarray library. These leaks has been reported, and hopefully they should be fixed more sooner than later. - Table buffers are built dinamically now, so if Tables are not accessed for reading or writing this memory will not be booked. This will help to reduce the memory consumption. - The opening of files with lots of nodes has been accelerated between a factor 2 or 3. For example, a file with 10 groups and 3000 tables that takes 9.3 seconds to open in 0.7.1, now takes only 2.8 seconds. - The Table.read() method has been refactored and optimized and some parts of its code has been moved to Pyrex. In particular, in the special case of step=1, up to a factor 5 of speedup (reaching 160 MB/s on a Pentium4 @ 2 GHz) when reading table contents can be achieved now. - Done some cosmetic changes in the user manual, but, as no new features has been added, you won't need to read the manual again :-) What is a table? ---------------- A table is defined as a collection of records whose values are stored in fixed-length fields. All records have the same structure and all values in each field have the same data type. The terms "fixed-length" and "strict data types" seems to be quite a strange requirement for an language like Python, that supports dynamic data types, but they serve a useful function if the goal is to save very large quantities of data (such as is generated by many scientific applications, for example) in an efficient manner that reduces demand on CPU time and I/O resources. What is HDF5? ------------- For those people who know nothing about HDF5, it is is a general purpose library and file format for storing scientific data made at NCSA. HDF5 can store two primary objects: datasets and groups. A dataset is essentially a multidimensional array of data elements, and a group is a structure for organizing objects in an HDF5 file. Using these two basic constructs, one can create and store almost any kind of scientific data structure, such as images, arrays of vectors, and structured and unstructured grids. You can also mix and match them in HDF5 files according to your needs. Platforms --------- I'm using Linux as the main development platform, but PyTables should be easy to compile/install on other UNIX machines. This package has also passed all the tests on a UltraSparc platform with Solaris 7 and Solaris 8. It also compiles and passes all the tests on a SGI Origin2000 with MIPS R12000 processors and running IRIX 6.5. Regarding Windows platforms, PyTables has been tested with Windows 2000 and Windows XP, but it should also work with other flavors. An example? ----------- For online code examples, have a look at http://pytables.sourceforge.net/tut/tutorial1-1.html and http://pytables.sourceforge.net/tut/tutorial1-2.html Web site -------- Go to the PyTables web site for more details: http://pytables.sourceforge.net/ Share your experience --------------------- Let me know of any bugs, suggestions, gripes, kudos, etc. you may have. Have fun! -- Francesc Alted

1 0

Promoting Your Business
by Betty 20 Sep '03

20 Sep '03

Email Marketing is one of the most effective and inexpensive ways to promote your products and services. We offer a complete Email Marketing solution with quality service and the lowest prices. The result is that you will enjoy more success. 1. Targeted Email Addresses We can supply targeted email addresses according to your requirements, which are compiled only on your order. We will customize your customer email addresses. * We have millions of email addresses in a wide variety of categories. 2. Targeted Mailing If you are worried about any complications or consequences with sending out targeted emails, or want to avoid the work of sending out targeted emails. We will do it for you! We can send your email message to your targeted customers. * We can Bullet-Proof your Web Site. We also offer a wide variety of marketing software. For more details, you can refer to: www.marketingboosting.com Our services will help you get more business opportunities. Regards! Betty Jones Customer Support www.marketingboosting.com Sales(a)marketingboosting.com ************************************************************************ Receiving this email because you registered to receive special offers from one of our partners. If you would prefer not to receive future email, click: Http://optin.garyshawkey.com/optin/r.php3 ************************************************************************

1 0

R E S U M E
by ELECTRICIAN 19 Sep '03

19 Sep '03

Resume from: Rich for Job or Service " no job too small " E L E C T R I C I A N Tel. (408) 482-2102 rysio3(a)yahoo.com WIRING & INSTALLATION Hands on electrical installations perform fitting, mounting, laying cables on Commercial, Industrial, residential new & existing buildings. Electrical Power Supply for Lights, Plugs, Receptacles, Panels, & Fuse boxes, Emergency Generators wiring and testing, Transformers, Power Lines & conduit layout, bending and mounting, parking lighting, lamps, switches, SOLAR PROJECTS, posts and underground installations. Shopping Centers; grocery stories, hardware stories, restaurants & residential - housing areas, computer business & fast food units installation & buildings; Solar Panels, Sun Tracking, Flywheel Storage & electric cars systems modify, Natural Energy in Remote areas install. LOW VOLTAGE Office Home Yard Patio Parking 12 / 24 Volt audio & video equipment, Computer & data network wiring, data backup and UPS; Monitoring Video Control & backup tapes set up and mounting, electro-optical assemblies & subsystems. DC Power Supply, Switch & Motion sensors Alarm. Fire & safety systems install. Fiber Optics systems, PLC setup, Master Control Center, cable modems & cable TV install. Network, UPS Battery Backup mounting and charging systems; Power supply testing, troubleshooting, and analyzing to a components level. Electric Vehicles Design, Assembly & Installations. CC TV & Cameras, Security Systems & Sensors for Safety, Fire sprinklers and traffic Monitoring & Door Control. Telephones / Net move & install. TECHNICIAN Use lab & shop equipment, mechanical, electrical & electronic tools, measurement & testing equipment, video cameras & microscopes. Support scientists & electronic engineers. Mechanical & Electro-Mech. Design. OFFICE, ELECTRICAL AND MECHANICAL PROJECTS Electrical & Network Sketches, one line diagrams, and "as is" drawings update. Customizing Electronic and Electrical Components & Parts, Layouts electronic and electrical schematic, connectors and mechanical detailing. Quotes, supply, bids and job estimating. Customers contact, inspection, project mgmt & supervision of electricians & material handling; Use CAD, Windows and applications; ELECTRICAL & MAINTENANCE SERVICE US Citizen; open for travel .

1 0

Re: [Numpy-discussion] array sort questions
by CL WU 18 Sep '03

18 Sep '03

Great! It works much more efficiently. Thank you so much. Best, Chunlei Tim Hochberg wrote: > CL WU wrote: > >> Thank you, Tim. argsort() and take() does provide a easy way to sort >> an array based on any col or row. But for the second question, it >> doesn't return the result I want. >> As below, softrank or softrank1 are functions I am currently using >> for get the rank of a vector(first is more efficient). It returns the >> index of each value from original array/list in sorted array/list. > > > Hmmm. It seems that argsort and sortrank are inverses of a sort, so it > should be possible to do what you want efficiently, but I'm not sure how. > > <think> > > Ah, it appears to be quite simple. I believe: > > argsort(argsort(a)) > > is equivalent to your sortrank and should be much faster. > > regards, > > -tim > > >> I hope there is an efficient function in array level to do the same >> work. >> >>> from Numeric import * >> >>> a=array([5,2,3]) >> >>> argsort(a) >> array([1, 2, 0]) >> >>> def sortrank(list): >> ... n=len(list) >> ... li_a=[(i,list[i]) for i in range(n)] >> ... li_a.sort(lambda a,b:cmp(a[1],b[1])) >> ... li_b=[(i,li_a[i]) for i in range(n)] >> ... li_b.sort(lambda a,b:cmp(a[1][0],b[1][0])) >> ... return [x[0] for x in li_b] >> ... >>> sortrank(a) >> [2, 0, 1] >> >>> def sortrank2(li): >> ... li_sorted=li[:] >> ... li_sorted.sort() >> ... return [li_sorted.index(x) for x in li] >> >>> sortrank1(list(a)) >> [2, 0, 1] >> >> >> Thanks again. >> >> Chunlei >> >> Tim Hochberg wrote: >> >>> CL WU wrote: >>> >>>> Hi, group, >>>> I am new to numpy. I have 2 questions for array sort. >>>> >>>> 1. How to sort an array by its one column or one row? >>>> I know python build-in sort() can do it for list by passing own >>>> cmp function. but array function sort() will sort each column or >>>> row seperately,as I know. I don't want to convert array to list to >>>> sort and then convert back to array. >>> >>> >>> >>> >>> I think you want argsort plus take. For example, the following sorts >>> on the second column of a: >>> >>> a = array([[4,5,6], [1,2,3], [7,8,9]]) >>> arg = argsort(a[:,1]) >>> take(a, arg, 0) >>> >>>> 2. How to get the rank of a rank-0 array? The first "rank" means >>>> the order of each element after sorting, instead of the "dimension" >>>> meaning in numpy. Just like "rank()" function in splus. >>> >>> >>> >>> >>> If I understand you correctly, you want argsort as mentioned above. >>> >>> Regards, >>> >>> -tim >>> >>> >>>> >>>> Thank you >>>> >>>> Chunlei >>>> >>>> >>>> >>>> >>>> ------------------------------------------------------- >>>> This sf.net email is sponsored by:ThinkGeek >>>> Welcome to geek heaven. >>>> http://thinkgeek.com/sf >>>> _______________________________________________ >>>> Numpy-discussion mailing list >>>> Numpy-discussion(a)lists.sourceforge.net >>>> https://lists.sourceforge.net/lists/listinfo/numpy-discussion >>>> >>> >>> >>> >>> >>> >>> ------------------------------------------------------- >>> This sf.net email is sponsored by:ThinkGeek >>> Welcome to geek heaven. >>> http://thinkgeek.com/sf >>> _______________________________________________ >>> Numpy-discussion mailing list >>> Numpy-discussion(a)lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/numpy-discussion >>> >> >> >> >> >> ------------------------------------------------------- >> This sf.net email is sponsored by:ThinkGeek >> Welcome to geek heaven. >> http://thinkgeek.com/sf >> _______________________________________________ >> Numpy-discussion mailing list >> Numpy-discussion(a)lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/numpy-discussion >> > > > >

3 4

array sort questions
by CL WU 17 Sep '03

17 Sep '03

Hi, group, I am new to numpy. I have 2 questions for array sort. 1. How to sort an array by its one column or one row? I know python build-in sort() can do it for list by passing own cmp function. but array function sort() will sort each column or row seperately,as I know. I don't want to convert array to list to sort and then convert back to array. 2. How to get the rank of a rank-0 array? The first "rank" means the order of each element after sorting, instead of the "dimension" meaning in numpy. Just like "rank()" function in splus. Thank you Chunlei

2 2

numarray latest version
by Nadav Horesh 17 Sep '03

17 Sep '03

How comes that numarray 0.6.2 succeeds 0.7? Are there 2 active branches? Nadav.

2 1

Re: MacPython and Numeric
by Jack Jansen 15 Sep '03

15 Sep '03

Folks, I got this report on Numeric as included with MacPython (through the package manager) and it looks promising. But: I'm building Numeric simply by running setup.py, so in order to get the added performance Numeric's setup.py would need to be taught about veclib. Is anyone willing to look into this? On Sunday, September 14, 2003, at 09:35 PM, Alun Carr wrote: > Jack, > > I would like to thank you for all your hard work in producing > MacPython 2.3 for OS X. It's superb. > > One point to note: I think that you may have compiled the Numeric > stuff from scratch. Apple supply hand-tuned BLAS, LAPACK, and FFT code > in their vecLib framework. Despite the name, according to Apple's > documentation this framework is suitable for both G3 and G4 > processors, and it will automatically use vectorized code if it > detects a G4. > > From <http://developer.apple.com/hardware/ve/quickstart.html>: > >> If you are a developer who would like to make more use of AltiVec in >> your application, you should first explore the functionality that is >> available in Apple's vecLib.framework . This framework can be found >> in /System/Library/Frameworks on any MacOS X installation. There you >> will find many highly optimized routines for a diversity of tasks. >> There are single and double precision FFT's. Some of these are up to >> five times faster than FFT-W . There are also convolutions, 1D and 2D >> matrix operations and much, much more. MacOS X.2 (Jaguar) rounds this >> out with a full BLAS and LAPACK implementation in both single and >> double precision. These functions also work on G3 and earlier >> processors, meaning that you can call them safely regardless of what >> machine you are running on. > > For future releases of Numeric for MacPython, it might be worth using > vecLib, to (i) reduce the effort on your part, (ii) reduce the size of > the distribution, and (iii) to use code which has been optimized by > Apple for maximum performance. > > Anyway, thanks again for your efforts, and please keep up the good > work. > > Best regards, > > Alun > > == > > Dr Alun J. Carr Phone: +353-(0)1-716 1989 (Office) > Mechanical Engineering Dept +353-(0)1-716 1752 (BRC) > University College Dublin Fax: +353-(0)1-283 0534 > Belfield Email: alun.j.carr(a)ucd.ie (UCD) > Dublin 4 carraj(a)attglobal.net (Home) > Ireland > -- Jack Jansen, <Jack.Jansen(a)cwi.nl>, http://www.cwi.nl/~jack If I can't dance I don't want to be part of your revolution -- Emma Goldman

2 1

Boolean array formatting poll
by Todd Miller 11 Sep '03

11 Sep '03

Now that Python has a bool type, I have received a patch for numarray's arrayprint module which alters the formatting of Boolean arrays from: >>>> a=numarray.arange(6, type=numarray.Bool); a[::2] = 0; a array([0, 1, 0, 1, 0, 1], type=Bool) to: >>>> a=numarray.arange(6, type=numarray.Bool); a[::2] = 0; a array([False, True, False, True, False, True], type=Bool) Which format do you prefer? Please vote for or against changing the format: +1 (for) or -1 (against). -- Todd Miller jmiller(a)stsci.edu STSCI / ESS / SSB

10 13

Why stripping spaces in strings?
by Francesc Alted 11 Sep '03

11 Sep '03

Hi, A user of pytables recently asked me why it truncates the trailing spaces on strings. I have to explain him that the information is there, but the strings module of the numarray package deliberately strips off the trailing spaces when printing: >>> from numarray import strings >>> a=strings.array("hello world ") >>> print a ['hello world'] >>> print "<%s>" % a <['hello world']> >>> len(a[0]) 11 however: >>> print "<%s>" % a._data <hello world > >>> a._byteView() array([[104, 101, 108, 108, 111, 32, 119, 111, 114, 108, 100, 32, 32]], type=UInt8) >>> a._itemsize 13 In the beginning I thought that this was a nice feature, but when people is using numarray.strings objects as containers (for example for Pickled objects), that can be rather confusing, and I would prefer to see all the trailing blanks. But as I am not completely sure, I just would like to discuss the pro's and con's of the current printing apprach. Cheers, -- Francesc Alted

2 2