Hi there, perhaps someone has a bright idea for this one: I want to concatenate ranges of numbers into a single array (for indexing). So I have generated an array "a" with starting positions, for example: a = [4, 0, 11] I have an array b with stop positions: b = [11, 4, 15] and I would like to generate an index array that takes 4..11, then 0..4, then 11..15. In reality, a and b have 10000+ elements and the arrays to be "sliced" are very large so I want to avoid any for loops etc. Any idea how this could be done? I thought some combination of *repeat* and adding of *arange* should do the trick but just cannot nail it down. Thanks in advance for any hints! Greetings, Raik
Raik Gruenberg wrote:
Hi there,
perhaps someone has a bright idea for this one:
I want to concatenate ranges of numbers into a single array (for indexing). So I have generated an array "a" with starting positions, for example:
a = [4, 0, 11]
I have an array b with stop positions:
b = [11, 4, 15]
and I would like to generate an index array that takes 4..11, then 0..4, then 11..15.
Does this help? Python 2.5.4 (r254:67916, Dec 23 2008, 15:10:54) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information.
a = [4, 0, 11] b = [11, 4, 15] zip(a,b) [(4, 11), (0, 4), (11, 15)]
Apologies if I'm stating the obvious.  jv
In reality, a and b have 10000+ elements and the arrays to be "sliced" are very large so I want to avoid any for loops etc. Any idea how this could be done? I thought some combination of *repeat* and adding of *arange* should do the trick but just cannot nail it down.
Thanks in advance for any hints!
Greetings, Raik
_______________________________________________ Numpydiscussion mailing list Numpydiscussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpydiscussion
Jim Vickroy wrote:
Raik Gruenberg wrote:
Hi there,
perhaps someone has a bright idea for this one:
I want to concatenate ranges of numbers into a single array (for indexing). So I have generated an array "a" with starting positions, for example:
a = [4, 0, 11]
I have an array b with stop positions:
b = [11, 4, 15]
and I would like to generate an index array that takes 4..11, then 0..4, then 11..15.
Does this help?
Python 2.5.4 (r254:67916, Dec 23 2008, 15:10:54) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information.
a = [4, 0, 11] b = [11, 4, 15] zip(a,b) [(4, 11), (0, 4), (11, 15)]
Mhm, I got this far. But how do I get from here to a single index array [ 4, 5, 6, ... 10, 0, 1, 2, 3, 11, 12, 13, 14 ] ? Greetings Raik
Apologies if I'm stating the obvious.
 jv
In reality, a and b have 10000+ elements and the arrays to be "sliced" are very large so I want to avoid any for loops etc. Any idea how this could be done? I thought some combination of *repeat* and adding of *arange* should do the trick but just cannot nail it down.
Thanks in advance for any hints!
Greetings, Raik
_______________________________________________ Numpydiscussion mailing list Numpydiscussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpydiscussion
_______________________________________________ Numpydiscussion mailing list Numpydiscussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpydiscussion
 ________________________________ Dr. Raik Gruenberg http://www.raiks.de/contact.html ________________________________
Raik Gruenberg wrote:
Jim Vickroy wrote:
Raik Gruenberg wrote:
Hi there,
perhaps someone has a bright idea for this one:
I want to concatenate ranges of numbers into a single array (for indexing). So I have generated an array "a" with starting positions, for example:
a = [4, 0, 11]
I have an array b with stop positions:
b = [11, 4, 15]
and I would like to generate an index array that takes 4..11, then 0..4, then 11..15.
Does this help?
Python 2.5.4 (r254:67916, Dec 23 2008, 15:10:54) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information.
a = [4, 0, 11] b = [11, 4, 15] zip(a,b) [(4, 11), (0, 4), (11, 15)]
Mhm, I got this far. But how do I get from here to a single index array
[ 4, 5, 6, ... 10, 0, 1, 2, 3, 11, 12, 13, 14 ] ?
not sure I understand your goal ... is this what you want:
[range(i,j) for i,j in zip(a,b)] [[4, 5, 6, 7, 8, 9, 10], [0, 1, 2, 3], [11, 12, 13, 14]]
Greetings Raik
Apologies if I'm stating the obvious.
 jv
In reality, a and b have 10000+ elements and the arrays to be "sliced" are very large so I want to avoid any for loops etc. Any idea how this could be done? I thought some combination of *repeat* and adding of *arange* should do the trick but just cannot nail it down.
Thanks in advance for any hints!
Greetings, Raik
_______________________________________________ Numpydiscussion mailing list Numpydiscussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpydiscussion
_______________________________________________ Numpydiscussion mailing list Numpydiscussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpydiscussion
Pierre GM wrote:
On Jan 30, 2009, at 1:11 PM, Raik Gruenberg wrote:
Mhm, I got this far. But how do I get from here to a single index array
[ 4, 5, 6, ... 10, 0, 1, 2, 3, 11, 12, 13, 14 ] ?
np.concatenate([np.arange(aa,bb) for (aa,bb) in zip(a,b)])
exactly! Now, the question was, is there a way to do this only using numpy functions (sum, repeat, ...), that means without any python "for" loop? Sorry about being so insistent on this one but, in my experience, eliminating those for loops makes a huge difference in terms of speed. The zip is probably also quite costly on a very large data set. Thanks! Raik
_______________________________________________ Numpydiscussion mailing list Numpydiscussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpydiscussion
 ________________________________ Dr. Raik Gruenberg http://www.raiks.de/contact.html ________________________________
On Jan 30, 2009, at 1:53 PM, Raik Gruenberg wrote:
Pierre GM wrote:
On Jan 30, 2009, at 1:11 PM, Raik Gruenberg wrote:
Mhm, I got this far. But how do I get from here to a single index array
[ 4, 5, 6, ... 10, 0, 1, 2, 3, 11, 12, 13, 14 ] ?
np.concatenate([np.arange(aa,bb) for (aa,bb) in zip(a,b)])
exactly! Now, the question was, is there a way to do this only using numpy functions (sum, repeat, ...), that means without any python "for" loop?
Can't really see it right now. Make np.arange(max(b)) and take the slices you need ? But you still have to look in 2 arrays to find the beginning and end of slices, so...
Sorry about being so insistent on this one but, in my experience, eliminating those for loops makes a huge difference in terms of speed. The zip is probably also quite costly on a very large data set.
yeah, but it's in a list comprehension, which may make things a tad faster. If you prefer, use itertools.izip instead of zip, but I wonder where the advantage would be. Anyway, are you sure this particular part is your bottleneck ? You know the saying about premature optimization...
participants (3)

Jim Vickroy

Pierre GM

Raik Gruenberg