The thread on what to do with boolean array formatting in numarray has
died out, so I thought I should explain (what I saw) as the results of
(1) There was a clear majority of numarray users who actually prefer the
status quo of 0,1 for representing arrays of boolean values.
(2) There was a sizable minority that would like to see consistency with
Python-2.3 by using False, True for representing arrays of boolean
(3) There was a (good) suggestion of using F, T for representing boolean
values. However, on reflection and reviewing the poll comments, F, T
is not really as good as 0, 1 visually, nor is it likely to satisfy
those people interested in consistency with Python, so it is unlikely to
make most numarray users any happier.
(4) Also suggested was supporting configurable boolean formats. This
approach is reasonable to implement, but I have a reservation that
adding support for multiple formats may cause as much confusion as it
gives satisfaction. Is it *really* worth it? (more later)
Given 1-4, I think the best thing to do for now is keep 0,1 as the only
format for boolean array values.
If someone wants False, True bad enough to submit a complete patch for
supporting multiple formats (and defaulting to 0,1), we will still
accept it. If anyone is still interested in doing this, the patch
should include modified TeX documentation and pass all the self tests in
setupall.py (all of core numarray and all current add-ons). The patch
should also run cleanly on both Python-2.2 and Python-2.3.
Thanks everyone for responding,
Todd Miller jmiller(a)stsci.edu
STSCI / ESS / SSB
Announcing PyTables 0.7.2
PyTables is a hierarchical database package designed to efficently
manage very large amounts of data. PyTables is built on top of the
HDF5 library and the numarray package. It features an object-oriented
interface that, combined with natural naming and C-code generated from
Pyrex sources, makes it a fast, yet extremely easy to use tool for
interactively save and retrieve large amounts of data. Besides, it
provides flexible indexed access on disk to anywhere in the data you
want to go.
On this release you will not find any exciting new features. It is
mainly a maintenance release where the next issues has been addressed:
- a memory leak was fixed
- memory consumption is being addressed and lowered
- much faster opening of files
- Some important index patterns cases in table reads has been
More in detail:
- Fixed a nasty memory leak located on the C libraries (it was
happening during HDF5 attribute writes). After that, the
memory consumption when using large object trees has dropped
quite a bit. However, there remains some small leaks that
has been tracked down to the underlying numarray
library. These leaks has been reported, and hopefully they
should be fixed more sooner than later.
- Table buffers are built dinamically now, so if Tables are
not accessed for reading or writing this memory will not be
booked. This will help to reduce the memory consumption.
- The opening of files with lots of nodes has been accelerated
between a factor 2 or 3. For example, a file with 10 groups
and 3000 tables that takes 9.3 seconds to open in 0.7.1, now
takes only 2.8 seconds.
- The Table.read() method has been refactored and optimized
and some parts of its code has been moved to Pyrex. In
particular, in the special case of step=1, up to a factor 5
of speedup (reaching 160 MB/s on a Pentium4 @ 2 GHz) when
reading table contents can be achieved now.
- Done some cosmetic changes in the user manual, but, as no
new features has been added, you won't need to read the
manual again :-)
What is a table?
A table is defined as a collection of records whose values are stored
in fixed-length fields. All records have the same structure and all
values in each field have the same data type. The terms
"fixed-length" and "strict data types" seems to be quite a strange
requirement for an language like Python, that supports dynamic data
types, but they serve a useful function if the goal is to save very
large quantities of data (such as is generated by many scientific
applications, for example) in an efficient manner that reduces demand
on CPU time and I/O resources.
What is HDF5?
For those people who know nothing about HDF5, it is is a general
purpose library and file format for storing scientific data made at
NCSA. HDF5 can store two primary objects: datasets and groups. A
dataset is essentially a multidimensional array of data elements, and
a group is a structure for organizing objects in an HDF5 file. Using
these two basic constructs, one can create and store almost any kind of
scientific data structure, such as images, arrays of vectors, and
structured and unstructured grids. You can also mix and match them in
HDF5 files according to your needs.
I'm using Linux as the main development platform, but PyTables should
be easy to compile/install on other UNIX machines. This package has
also passed all the tests on a UltraSparc platform with Solaris 7 and
Solaris 8. It also compiles and passes all the tests on a SGI
Origin2000 with MIPS R12000 processors and running IRIX 6.5.
Regarding Windows platforms, PyTables has been tested with Windows
2000 and Windows XP, but it should also work with other flavors.
For online code examples, have a look at
Go to the PyTables web site for more details:
Share your experience
Let me know of any bugs, suggestions, gripes, kudos, etc. you may
-- Francesc Alted
Email Marketing is one of the most effective and inexpensive ways to
promote your products and services.
We offer a complete Email Marketing solution with quality service and
the lowest prices. The result is that you will enjoy more success.
1. Targeted Email Addresses
We can supply targeted email addresses according to your requirements,
which are compiled only on your order. We will customize your customer
* We have millions of email addresses in a wide variety of categories.
2. Targeted Mailing
If you are worried about any complications or consequences with sending
out targeted emails, or want to avoid the work of sending out targeted
emails. We will do it for you! We can send your email message to your
* We can Bullet-Proof your Web Site.
We also offer a wide variety of marketing software. For more details,
you can refer to: www.marketingboosting.com
Our services will help you get more business opportunities.
Receiving this email because you registered to receive special
offers from one of our partners. If you would prefer not to receive
future email, click: Http://optin.garyshawkey.com/optin/r.php3
Resume from: Rich for Job or Service " no job too small "
E L E C T R I C I A N
Tel. (408) 482-2102 rysio3(a)yahoo.com
WIRING & INSTALLATION
Hands on electrical installations perform fitting, mounting, laying cables on Commercial, Industrial, residential new & existing buildings.
Electrical Power Supply for Lights, Plugs, Receptacles, Panels, & Fuse boxes, Emergency Generators wiring and testing, Transformers, Power Lines & conduit layout, bending and mounting, parking lighting, lamps, switches, SOLAR PROJECTS, posts and underground installations.
Shopping Centers; grocery stories, hardware stories, restaurants & residential - housing areas, computer business & fast food units installation & buildings; Solar Panels, Sun Tracking, Flywheel Storage & electric cars systems modify, Natural Energy in Remote areas install.
LOW VOLTAGE Office Home Yard Patio Parking
12 / 24 Volt audio & video equipment, Computer & data network wiring, data backup and UPS; Monitoring Video Control & backup tapes set up and mounting, electro-optical assemblies & subsystems. DC Power Supply, Switch & Motion sensors Alarm. Fire & safety systems install.
Fiber Optics systems, PLC setup, Master Control Center, cable modems & cable TV install. Network, UPS Battery Backup mounting and charging systems; Power supply testing, troubleshooting, and analyzing to a components level. Electric Vehicles Design, Assembly & Installations.
CC TV & Cameras, Security Systems & Sensors for Safety, Fire sprinklers and traffic Monitoring & Door Control. Telephones / Net move & install.
Use lab & shop equipment, mechanical, electrical & electronic tools, measurement & testing equipment, video cameras & microscopes. Support scientists & electronic engineers. Mechanical & Electro-Mech. Design.
OFFICE, ELECTRICAL AND MECHANICAL PROJECTS
Electrical & Network Sketches, one line diagrams, and "as is" drawings update. Customizing Electronic and Electrical Components & Parts, Layouts electronic and electrical schematic, connectors and mechanical detailing. Quotes, supply, bids and job estimating.
Customers contact, inspection, project mgmt & supervision of electricians & material handling; Use CAD, Windows and applications;
ELECTRICAL & MAINTENANCE SERVICE US Citizen; open for travel
Great! It works much more efficiently. Thank you so much.
Tim Hochberg wrote:
> CL WU wrote:
>> Thank you, Tim. argsort() and take() does provide a easy way to sort
>> an array based on any col or row. But for the second question, it
>> doesn't return the result I want.
>> As below, softrank or softrank1 are functions I am currently using
>> for get the rank of a vector(first is more efficient). It returns the
>> index of each value from original array/list in sorted array/list.
> Hmmm. It seems that argsort and sortrank are inverses of a sort, so it
> should be possible to do what you want efficiently, but I'm not sure how.
> Ah, it appears to be quite simple. I believe:
> is equivalent to your sortrank and should be much faster.
>> I hope there is an efficient function in array level to do the same
>> >>> from Numeric import *
>> >>> a=array([5,2,3])
>> >>> argsort(a)
>> array([1, 2, 0])
>> >>> def sortrank(list):
>> ... n=len(list)
>> ... li_a=[(i,list[i]) for i in range(n)]
>> ... li_a.sort(lambda a,b:cmp(a,b))
>> ... li_b=[(i,li_a[i]) for i in range(n)]
>> ... li_b.sort(lambda a,b:cmp(a,b))
>> ... return [x for x in li_b]
>> ... >>> sortrank(a)
>> [2, 0, 1]
>> >>> def sortrank2(li):
>> ... li_sorted=li[:]
>> ... li_sorted.sort()
>> ... return [li_sorted.index(x) for x in li]
>> >>> sortrank1(list(a))
>> [2, 0, 1]
>> Thanks again.
>> Tim Hochberg wrote:
>>> CL WU wrote:
>>>> Hi, group,
>>>> I am new to numpy. I have 2 questions for array sort.
>>>> 1. How to sort an array by its one column or one row?
>>>> I know python build-in sort() can do it for list by passing own
>>>> cmp function. but array function sort() will sort each column or
>>>> row seperately,as I know. I don't want to convert array to list to
>>>> sort and then convert back to array.
>>> I think you want argsort plus take. For example, the following sorts
>>> on the second column of a:
>>> a = array([[4,5,6], [1,2,3], [7,8,9]])
>>> arg = argsort(a[:,1])
>>> take(a, arg, 0)
>>>> 2. How to get the rank of a rank-0 array? The first "rank" means
>>>> the order of each element after sorting, instead of the "dimension"
>>>> meaning in numpy. Just like "rank()" function in splus.
>>> If I understand you correctly, you want argsort as mentioned above.
>>>> Thank you
>>>> This sf.net email is sponsored by:ThinkGeek
>>>> Welcome to geek heaven.
>>>> Numpy-discussion mailing list
>>> This sf.net email is sponsored by:ThinkGeek
>>> Welcome to geek heaven.
>>> Numpy-discussion mailing list
>> This sf.net email is sponsored by:ThinkGeek
>> Welcome to geek heaven.
>> Numpy-discussion mailing list
I am new to numpy. I have 2 questions for array sort.
1. How to sort an array by its one column or one row?
I know python build-in sort() can do it for list by passing own cmp
function. but array function sort() will sort each column or row
seperately,as I know. I don't want to convert array to list to sort and
then convert back to array.
2. How to get the rank of a rank-0 array? The first "rank" means the
order of each element after sorting, instead of the "dimension" meaning
in numpy. Just like "rank()" function in splus.
I got this report on Numeric as included with MacPython (through the
package manager) and it looks
promising. But: I'm building Numeric simply by running setup.py, so in
order to get the added performance
Numeric's setup.py would need to be taught about veclib.
Is anyone willing to look into this?
On Sunday, September 14, 2003, at 09:35 PM, Alun Carr wrote:
> I would like to thank you for all your hard work in producing
> MacPython 2.3 for OS X. It's superb.
> One point to note: I think that you may have compiled the Numeric
> stuff from scratch. Apple supply hand-tuned BLAS, LAPACK, and FFT code
> in their vecLib framework. Despite the name, according to Apple's
> documentation this framework is suitable for both G3 and G4
> processors, and it will automatically use vectorized code if it
> detects a G4.
> From <http://developer.apple.com/hardware/ve/quickstart.html>:
>> If you are a developer who would like to make more use of AltiVec in
>> your application, you should first explore the functionality that is
>> available in Apple's vecLib.framework . This framework can be found
>> in /System/Library/Frameworks on any MacOS X installation. There you
>> will find many highly optimized routines for a diversity of tasks.
>> There are single and double precision FFT's. Some of these are up to
>> five times faster than FFT-W . There are also convolutions, 1D and 2D
>> matrix operations and much, much more. MacOS X.2 (Jaguar) rounds this
>> out with a full BLAS and LAPACK implementation in both single and
>> double precision. These functions also work on G3 and earlier
>> processors, meaning that you can call them safely regardless of what
>> machine you are running on.
> For future releases of Numeric for MacPython, it might be worth using
> vecLib, to (i) reduce the effort on your part, (ii) reduce the size of
> the distribution, and (iii) to use code which has been optimized by
> Apple for maximum performance.
> Anyway, thanks again for your efforts, and please keep up the good
> Best regards,
> Dr Alun J. Carr Phone: +353-(0)1-716 1989 (Office)
> Mechanical Engineering Dept +353-(0)1-716 1752 (BRC)
> University College Dublin Fax: +353-(0)1-283 0534
> Belfield Email: alun.j.carr(a)ucd.ie (UCD)
> Dublin 4 carraj(a)attglobal.net (Home)
Jack Jansen, <Jack.Jansen(a)cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma
Now that Python has a bool type, I have received a patch for numarray's
arrayprint module which alters the formatting of Boolean arrays from:
>>>> a=numarray.arange(6, type=numarray.Bool); a[::2] = 0; a
array([0, 1, 0, 1, 0, 1], type=Bool)
>>>> a=numarray.arange(6, type=numarray.Bool); a[::2] = 0; a
array([False, True, False, True, False, True], type=Bool)
Which format do you prefer? Please vote for or against changing the
format: +1 (for) or -1 (against).
Todd Miller jmiller(a)stsci.edu
STSCI / ESS / SSB
A user of pytables recently asked me why it truncates the trailing spaces on
strings. I have to explain him that the information is there, but the
strings module of the numarray package deliberately strips off the trailing
spaces when printing:
>>> from numarray import strings
>>> a=strings.array("hello world ")
>>> print a
>>> print "<%s>" % a
>>> print "<%s>" % a._data
<hello world >
array([[104, 101, 108, 108, 111, 32, 119, 111, 114, 108, 100, 32, 32]],
In the beginning I thought that this was a nice feature, but when people is
using numarray.strings objects as containers (for example for Pickled
objects), that can be rather confusing, and I would prefer to see all the
But as I am not completely sure, I just would like to discuss the pro's and
con's of the current printing apprach.