[Baypiggies] sorting a table by column
Chris.Clark at ingres.com
Thu Jun 23 19:31:44 CEST 2011
Tripathi, Bibha wrote:
> a huge table like an excel sheet, saved and accumulating more data
> more rows, may be more tables
> user chooses which column to sort on
> what's the best python data structure to use?
It probably depends on what "huge" means. If the "table" data fits in
memory (either physical or virtual) it probably isn't that big and doing
operations in Python (e.g. using list comprehension) is probably
appropriate. If it doesn't fit in memory you can't easily use list
comprehension and need to look into loops and generators, see
(I think Alex might have done something similar at PyCon a few years ago
There are some third party libs on PyPi that are worth checking out. I
keep wanting to find an excuse to kick the tires on
http://pypi.python.org/pypi/blist/ but I've not had cause to do so yet.
Search PyPi for "tree" and there are a lot of hits.
You can even use good old Schwartzian transforms (aka
decorate-sort-undecorate) to handle changes in columns if for some
reason there isn't a key argument to sort() provided by the structure
David Berthelot wrote:
> Looks like a typical SQL problem.
Agreed, without more information this sounds like a classic "ORDERY BY"
clause on a SELECT statement. Relational database really excel, lower
case "e", rather than upper case "E" :-), at this sort of thing..... If
you check my email address domain name, of course I'm going to say that
;-) Databases do a lot of the heavy lifting for you.
If you're doing some sort of BI analytics a database is probably your
best bet (I'm taking a guess you are based on your email address domain
name). Shameless promotion, take a gander at
and http://www.ingres.com/products/vectorwise (I don't work on
Vectorwise but I'm always blown away at how fast it is). If you are
avoiding a traditional DBMS for performance reasons VW may well surprise
More information about the Baypiggies