From mark at microenh.com  Thu Jan 15 20:35:11 2009
From: mark at microenh.com (Mark Erbaugh)
Date: Thu, 15 Jan 2009 14:35:11 -0500
Subject: [CentralOH] Manipulating Nested Sequences
Message-ID: <1232048111.8054.45.camel@quad>

I have some data in the form of nested sequences. Think of this as a
three-dimensional array.

data[country][category][product]

Though not required by Python, the each data element at each level is
the same length as would be required by a multi-dimensional array in
other languages.

I would like to rearrange the data so it is in the format of

data[product][country][category]

for example if the data were:

(
((A,a),(B,b),(C,c)),
((D,d),(E,e),(F,f)),
((G,g),(H,h),(I,i)),
((J,j),(K,k),(L,l))
)

I would like to transform it to

(
((A,B,C),(D,E,F),(G,H,I),(J,K,L)),
((a,b,c),(d,e,f),(g,h,i),(j,k,l)),
)

Is there a name for this particular operation?

Is there a generic way to do this for arrays with an arbitrary number of
dimensions?


From steven_h at acm.org  Thu Jan 15 21:39:42 2009
From: steven_h at acm.org (Steven Huwig)
Date: Thu, 15 Jan 2009 15:39:42 -0500
Subject: [CentralOH] Manipulating Nested Sequences
In-Reply-To: <1232048111.8054.45.camel@quad>
References: <1232048111.8054.45.camel@quad>
Message-ID: <7505f2a60901151239i1da1e715lb6d5661245af1024@mail.gmail.com>

I don't know what this is called. I think this does what you want.

import itertools
import operator
def foo(x):
    outarray = []
    try:
        for i in itertools.count():
            outrow = []
            for row in x:
                outrow.append(map(operator.itemgetter(i), row))
            outarray.append(outrow)
    except IndexError:
        pass
    return outarray

-- Steve

On Thu, Jan 15, 2009 at 2:35 PM, Mark Erbaugh <mark at microenh.com> wrote:
> I have some data in the form of nested sequences. Think of this as a
> three-dimensional array.
>
> data[country][category][product]
>
> Though not required by Python, the each data element at each level is
> the same length as would be required by a multi-dimensional array in
> other languages.
>
> I would like to rearrange the data so it is in the format of
>
> data[product][country][category]
>
> for example if the data were:
>
> (
> ((A,a),(B,b),(C,c)),
> ((D,d),(E,e),(F,f)),
> ((G,g),(H,h),(I,i)),
> ((J,j),(K,k),(L,l))
> )
>
> I would like to transform it to
>
> (
> ((A,B,C),(D,E,F),(G,H,I),(J,K,L)),
> ((a,b,c),(d,e,f),(g,h,i),(j,k,l)),
> )
>
> Is there a name for this particular operation?
>
> Is there a generic way to do this for arrays with an arbitrary number of
> dimensions?
>
> _______________________________________________
> CentralOH mailing list
> CentralOH at python.org
> http://mail.python.org/mailman/listinfo/centraloh
>

From wam at cisco.com  Thu Jan 15 22:55:46 2009
From: wam at cisco.com (William McVey)
Date: Thu, 15 Jan 2009 16:55:46 -0500
Subject: [CentralOH] Manipulating Nested Sequences
In-Reply-To: <1232048111.8054.45.camel@quad>
References: <1232048111.8054.45.camel@quad>
Message-ID: <1232056546.27167.103.camel@tardis>

On Thu, 2009-01-15 at 14:35 -0500, Mark Erbaugh wrote:
> I have some data in the form of nested sequences. Think of this as a
> three-dimensional array.
<snip>

> I would like to rearrange the data so it is in the format of
> data[product][country][category]

It's not clear to me why some of those nested datastructures aren't
dictionaries, but if you're really dealing with nested sequences, then I
would recommend reading the docs on the itertools module. The imap, izip
and starmap functions in particular should allow you to reshape
tuple/list based data structures in nearly any fashion you can think
of. 

> for example if the data were:
> 
> (
> ((A,a),(B,b),(C,c)),
> ((D,d),(E,e),(F,f)),
> ((G,g),(H,h),(I,i)),
> ((J,j),(K,k),(L,l))
> )
> 
> I would like to transform it to
> 
> (
> ((A,B,C),(D,E,F),(G,H,I),(J,K,L)),
> ((a,b,c),(d,e,f),(g,h,i),(j,k,l)),
> )

Just as a proof of concept, I've used the non-iterator based versions of
those functions to get you the transform you are looking for. It should
work with arbitrary sized inputs of the same structure format you
provided. However, for large datasets, use of the itertools variant of
these functions is recommended to minimize allocating memory to hold
temporary lists used during the transformation.
        
        >>> from pprint import pprint
        >>> data = (
                (("A","a"),("B","b"),("C","c")),
                (("D","d"),("E","e"),("F","f")),
                (("G","g"),("H","h"),("I","i")),
                (("J","j"),("K","k"),("L","l"))
        )
        >>> pprint(zip(*map(zip, *zip(*data))))
        [(('A', 'B', 'C'), ('D', 'E', 'F'), ('G', 'H', 'I'), ('J', 'K', 'L')),
         (('a', 'b', 'c'), ('d', 'e', 'f'), ('g', 'h', 'i'), ('j', 'k', 'l'))]
        

> Is there a name for this particular operation?

I think of these as matrix transformations. Which leads me to believe
numpy may have a different (less obtuse) method of getting the same
transformation using actual matrix operations. I'm not very familiar
with numpy, so I just went the functional programming route. For that
matter, there may be a more effective transformation using just zip and
map that I didn't see.

> Is there a generic way to do this for arrays with an arbitrary number of
> dimensions?

Well, you requested a particular transformation from a large number of
potential transformations. Each additional dimension gives even more
potential transformations. There is no real *general* solution (that I
know of) for solving this particular problem. There are general
techniques though,  (e.g. twisting the matrix and recombining elements)
which tools like zip and map can be used to do.

  -- William


From David.Byrne at cambridge-na.com  Fri Jan 23 21:27:07 2009
From: David.Byrne at cambridge-na.com (Byrne, David)
Date: Fri, 23 Jan 2009 15:27:07 -0500
Subject: [CentralOH] TurboGears developer
Message-ID: <BA8583383ABE464F87CFDFBC8FE46F6A032FC5E0@USCOL2K3EXVS04.cisgi.com>

I was wondering if anyone on this list resides in the Columbus area and
has experience in developing with TurboGears.  Please contact me at
David.Byrne at cambridge-na.com.

 

Thanks,

David Byrne

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/mailman/private/centraloh/attachments/20090123/25b41d81/attachment.htm>