A buildbot farm with shell access - for free ?

Hi, Just saw that on one ML: http://www.snakebite.org/ http://mail.python.org/pipermail/python-committers/2009-January/000331.html Bottom line: it looks like there is a set of machines which were donated to the PSF for buildbot *with shell access* so that people can fix problems appearing on some platforms. If you look at the email, there are some 'exotic' machines that mere mortals cannot have access to (like True64 on Itanium: to quote the email "massive quad Itanium 2 RX-5670s, chock full of 73GB 15k disks and no less than 78GB of RAM between the two servers; 32GB in one and 46GB in the other"). There are also windows machines available. It is said in the email that this is reserved to the python project, and prominent python projects like Twisted and Django. Would it be ok to try to be qualified as a prominent python project as well ? cheers, David

On Wed, Jan 28, 2009 at 7:11 PM, David Cournapeau < david@ar.media.kyoto-u.ac.jp> wrote:
Hi,
Just saw that on one ML:
http://www.snakebite.org/ http://mail.python.org/pipermail/python-committers/2009-January/000331.html
Bottom line: it looks like there is a set of machines which were donated to the PSF for buildbot *with shell access* so that people can fix problems appearing on some platforms. If you look at the email, there are some 'exotic' machines that mere mortals cannot have access to (like True64 on Itanium: to quote the email "massive quad Itanium 2 RX-5670s, chock full of 73GB 15k disks and no less than 78GB of RAM between the two servers; 32GB in one and 46GB in the other"). There are also windows machines available.
It is said in the email that this is reserved to the python project, and prominent python projects like Twisted and Django. Would it be ok to try to be qualified as a prominent python project as well ?
Ohhh... I love buildbots. Go for it. Chuck

Sounds like a great idea! On 1/28/09, David Cournapeau <david@ar.media.kyoto-u.ac.jp> wrote:
Hi,
Just saw that on one ML:
http://www.snakebite.org/ http://mail.python.org/pipermail/python-committers/2009-January/000331.html
Bottom line: it looks like there is a set of machines which were donated to the PSF for buildbot *with shell access* so that people can fix problems appearing on some platforms. If you look at the email, there are some 'exotic' machines that mere mortals cannot have access to (like True64 on Itanium: to quote the email "massive quad Itanium 2 RX-5670s, chock full of 73GB 15k disks and no less than 78GB of RAM between the two servers; 32GB in one and 46GB in the other"). There are also windows machines available.
It is said in the email that this is reserved to the python project, and prominent python projects like Twisted and Django. Would it be ok to try to be qualified as a prominent python project as well ?
cheers,
David _______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
-- Sent from my mobile device ----------------------------------------------------- Damian Eads Ph.D. Student Jack Baskin School of Engineering, UCSC E2-489 1156 High Street Machine Learning Lab Santa Cruz, CA 95064 http://www.soe.ucsc.edu/~eads

On Wed, Jan 28, 2009 at 6:11 PM, David Cournapeau <david@ar.media.kyoto-u.ac.jp> wrote:
It is said in the email that this is reserved to the python project, and prominent python projects like Twisted and Django. Would it be ok to try to be qualified as a prominent python project as well ?
That would be great.

Hi, I have to buidl a grid with 256 point by the command: a = arange(-15,16,2) L = len(a) cnstl = a.reshape(L,1)+1j*a My problem is that I have a big data array that contains the data round the points in cnstl. I want to slice the point to the closest cnstl point and also compute the error. The condition is in the middle of the two point in x and y axis. I can do it in a for loop. Since Python and numpy have a lot of magic, I want to find an efficient way to do. This problem arise from QAM 256 modulation. Thanks Frank _________________________________________________________________ Windows Live™ Hotmail®:…more than just e-mail. http://windowslive.com/explore?ocid=TXT_TAGLM_WL_t2_hm_justgotbetter_explore...

On Wed, Jan 28, 2009 at 23:52, frank wang <f.yw@hotmail.com> wrote:
Hi,
I have to buidl a grid with 256 point by the command: a = arange(-15,16,2) L = len(a) cnstl = a.reshape(L,1)+1j*a
My problem is that I have a big data array that contains the data round the points in cnstl. I want to slice the point to the closest cnstl point and also compute the error. The condition is in the middle of the two point in x and y axis. I can do it in a for loop. Since Python and numpy have a lot of magic, I want to find an efficient way to do. This problem arise from QAM 256 modulation.
Can you show us the for loop? I'm not really sure what you want to compute. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco

Here is the for loop that I am think about. Also, I do not know whether the where commands can handle the complicated logic. The where command basically find the data in the square around the point cnstl[j]. Let the data array is qam with size N Out = X error = X for i in arange(N): for j in arange(L): aa = np.where((real(X)<real(cnstl[j])+1) & (real(X)>real(cnstl[j])-1) & (imag(X)<imag(cnstl[j])+1) & (imag(X)>imag(cnstl[j]-1)) Out[aa]=cnstl[j] error[aa]=abs(X)**2 - abs(cnstl[j])**2 Thanks Frank
Date: Wed, 28 Jan 2009 23:57:16 -0600> From: robert.kern@gmail.com> To: numpy-discussion@scipy.org> Subject: Re: [Numpy-discussion] help on fast slicing on a grid> > On Wed, Jan 28, 2009 at 23:52, frank wang <f.yw@hotmail.com> wrote:> >> > Hi,> >> > I have to buidl a grid with 256 point by the command:> > a = arange(-15,16,2)> > L = len(a)> > cnstl = a.reshape(L,1)+1j*a> >> > My problem is that I have a big data array that contains the data round the> > points in cnstl. I want to slice the point to the closest cnstl point and> > also compute the error. The condition is in the middle of the two point in x> > and y axis. I can do it in a for loop. Since Python and numpy have a lot of> > magic, I want to find an efficient way to do. This problem arise from QAM> > 256 modulation.> > Can you show us the for loop? I'm not really sure what you want to compute.> > -- > Robert Kern> > "I have come to believe that the whole world is an enigma, a harmless> enigma that is made terrible by our own mad attempt to interpret it as> though it had an underlying truth."> -- Umberto Eco> _______________________________________________> Numpy-discussion mailing list> Numpy-discussion@scipy.org> http://projects.scipy.org/mailman/listinfo/numpy-discussion
Windows Live™: E-mail. Chat. Share. Get more ways to connect. http://windowslive.com/howitworks?ocid=TXT_TAGLM_WL_t2_allup_howitworks_0120...

On Thu, Jan 29, 2009 at 00:09, frank wang <f.yw@hotmail.com> wrote:
Here is the for loop that I am think about. Also, I do not know whether the where commands can handle the complicated logic. The where command basically find the data in the square around the point cnstl[j].
cnstl is a 2D array from your previous description.
Let the data array is qam with size N
I don't see qam anywhere. Did you mean X?
Out = X error = X
Don't you want something like zeros_like(X) for these?
for i in arange(N): for j in arange(L): aa = np.where((real(X)<real(cnstl[j])+1) & (real(X)>real(cnstl[j])-1) & (imag(X)<imag(cnstl[j])+1) & (imag(X)>imag(cnstl[j]-1)) Out[aa]=cnstl[j] error[aa]=abs(X)**2 - abs(cnstl[j])**2
I'm still confused. Can you show me a complete, working script with possibly fake data? -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco

Hi, Bob, Thanks for your help. I am sorry for my type error. qam array is the X array in my example. cntl is a complex array contains the point (x,y) axises. I will try to make a workable example. Also I will try to find out the zeros_like function. However, I guess that zeros_like(X) will create an array the same size as X. It it is. Then the two line Out=X and error=X should be Out=zeros_like(X) and error=zeros(X). Also, can where command handel the logic command? aa = np.where((real(X)<real(cnstl[j])+1) & (real(X)>real(cnstl[j])-1) & (imag(X)<imag(cnstl[j])+1) & (imag(X)>imag(cnstl[j]-1)) For example, cntl[j]=3+1j*5, then the where command is the same as: aa = np.where((real(X)<4) & (real(X)>2 )& (imag(X)<6) & (imag(X)>4)) Thanks Frank> Date: Thu, 29 Jan 2009 00:15:48 -0600> From: robert.kern@gmail.com> To: numpy-discussion@scipy.org> Subject: Re: [Numpy-discussion] help on fast slicing on a grid> > On Thu, Jan 29, 2009 at 00:09, frank wang <f.yw@hotmail.com> wrote:> > Here is the for loop that I am think about. Also, I do not know whether the> > where commands can handle the complicated logic.> > The where command basically find the data in the square around the point> > cnstl[j].> > cnstl is a 2D array from your previous description.> > > Let the data array is qam with size N> > I don't see qam anywhere. Did you mean X?> > > Out = X> > error = X> > Don't you want something like zeros_like(X) for these?> > > for i in arange(N):> > for j in arange(L):> > aa = np.where((real(X)<real(cnstl[j])+1) &> > (real(X)>real(cnstl[j])-1) & (imag(X)<imag(cnstl[j])+1) &> > (imag(X)>imag(cnstl[j]-1))> > Out[aa]=cnstl[j]> > error[aa]=abs(X)**2 - abs(cnstl[j])**2> > I'm still confused. Can you show me a complete, working script with> possibly fake data?> > -- > Robert Kern> > "I have come to believe that the whole world is an enigma, a harmless> enigma that is made terrible by our own mad attempt to interpret it as> though it had an underlying truth."> -- Umberto Eco> _______________________________________________> Numpy-discussion mailing list> Numpy-discussion@scipy.org> http://projects.scipy.org/mailman/listinfo/numpy-discussion _________________________________________________________________ Windows Live™: E-mail. Chat. Share. Get more ways to connect. http://windowslive.com/explore?ocid=TXT_TAGLM_WL_t2_allup_explore_012009

David Cournapeau wrote:
It is said in the email that this is reserved to the python project, and prominent python projects like Twisted and Django. Would it be ok to try to be qualified as a prominent python project as well ?
From my perspective NumPy and Sage both count as prominent Python
Give it some time. Nobody - not even the Python core devs - have access to the machines. It's going to take at least several weeks to get the infrastructure running and to establish a policy. projects. Heck, you are in competition with tools like Matlab and you ain't looking bad! Furthermore you could make better use of the machines than Django because you are using way more C extensions and esoteric libraries. I recommend you subscribe to the snakebite list and bring up your interest. You got my +1 already. For now the list is snakebite-list@googlegroups.com but it will move to another server (probably Python.org) eventually. Christian

I have created a test example for the question using for loop and hope someone can help me to get fast solution. My data set is about 2000000 data. However, I have the problem to run the code, the Out[i]=cnstl[j] line gives me error says: In [107]: Out[0]=cnstl[0]---------------------------------------------------------------------------TypeError Traceback (most recent call last) C:\Frank_share\qamslicer.py in <module>()----> 1 2 3 4 5 TypeError: can't convert complex to float; use abs(z) In [108]: cnstl.dtypeOut[108]: dtype('complex128') I do not know why that my data is complex128 already. Can anyone help to figure why? Thanks Frank from numpy import *a = arange(-15,16,2)cnstl=a.reshape(16,1)+1j*acnstl=cnstl.reshape(256,1) X = array([1.4 + 1j*2.7, -4.9 + 1j*8.3]) Out = array(X)error =array(X)for i in xrange(2): for j in xrange(256): a0 = real(X[i]) < (real(cnstl[j])+1) a1 = real(X[i]) > (real(cnstl[j])-1) a2 = imag(X[i]) > (imag(cnstl[j])-1) a3 = imag(X[i]) < (imag(cnstl[j])+1) if (a0 & a1 & a2 &a3): Out[i] = cnstl[j] error[i] = X[i] - cnstl[j] From: f.yw@hotmail.comTo: numpy-discussion@scipy.orgSubject: RE: [Numpy-discussion] help on fast slicing on a gridDate: Wed, 28 Jan 2009 23:28:47 -0700 Hi, Bob, Thanks for your help. I am sorry for my type error. qam array is the X array in my example. cntl is a complex array contains the point (x,y) axises. I will try to make a workable example. Also I will try to find out the zeros_like function. However, I guess that zeros_like(X) will create an array the same size as X. It it is. Then the two line Out=X and error=X should be Out=zeros_like(X) and error=zeros(X). Also, can where command handel the logic command? aa = np.where((real(X)<real(cnstl[j])+1) & (real(X)>real(cnstl[j])-1) & (imag(X)<imag(cnstl[j])+1) & (imag(X)>imag(cnstl[j]-1)) For example, cntl[j]=3+1j*5, then the where command is the same as: aa = np.where((real(X)<4) & (real(X)>2 )& (imag(X)<6) & (imag(X)>4)) Thanks Frank> Date: Thu, 29 Jan 2009 00:15:48 -0600> From: robert.kern@gmail.com> To: numpy-discussion@scipy.org> Subject: Re: [Numpy-discussion] help on fast slicing on a grid> > On Thu, Jan 29, 2009 at 00:09, frank wang <f.yw@hotmail.com> wrote:> > Here is the for loop that I am think about. Also, I do not know whether the> > where commands can handle the complicated logic.> > The where command basically find the data in the square around the point> > cnstl[j].> > cnstl is a 2D array from your previous description.> > > Let the data array is qam with size N> > I don't see qam anywhere. Did you mean X?> > > Out = X> > error = X> > Don't you want something like zeros_like(X) for these?> > > for i in arange(N):> > for j in arange(L):> > aa = np.where((real(X)<real(cnstl[j])+1) &> > (real(X)>real(cnstl[j])-1) & (imag(X)<imag(cnstl[j])+1) &> > (imag(X)>imag(cnstl[j]-1))> > Out[aa]=cnstl[j]> > error[aa]=abs(X)**2 - abs(cnstl[j])**2> > I'm still confused. Can you show me a complete, working script with> possibly fake data?> > -- > Robert Kern> > "I have come to believe that the whole world is an enigma, a harmless> enigma that is made terrible by our own mad attempt to interpret it as> though it had an underlying truth."> -- Umberto Eco> _______________________________________________> Numpy-discussion mailing list> Numpy-discussion@scipy.org> http://projects.scipy.org/mailman/listinfo/numpy-discussion Windows Live™: E-mail. Chat. Share. Get more ways to connect. Check it out. _________________________________________________________________ Hotmail® goes where you go. On a PC, on the Web, on your phone. http://www.windowslive-hotmail.com/learnmore/versatility.aspx#mobile?ocid=TX...

On Fri, Jan 30, 2009 at 12:58, frank wang <f.yw@hotmail.com> wrote:
I have created a test example for the question using for loop and hope someone can help me to get fast solution. My data set is about 2000000 data.
However, I have the problem to run the code, the Out[i]=cnstl[j] line gives me error says:
In [107]: Out[0]=cnstl[0] --------------------------------------------------------------------------- TypeError Traceback (most recent call last) C:\Frank_share\qamslicer.py in <module>() ----> 1 2 3 4 5 TypeError: can't convert complex to float; use abs(z) In [108]: cnstl.dtype Out[108]: dtype('complex128')
I do not know why that my data is complex128 already. Can anyone help to figure why?
It's an odd error message, certainly. The root of the problem is that you are attempting to put a (1,)-shaped array into a scalar. You don't want to do that. By the way, you don't want to use the variable name Out in IPython. It's already used to capture the output.
Thanks
Frank
from numpy import * a = arange(-15,16,2) cnstl=a.reshape(16,1)+1j*a cnstl=cnstl.reshape(256,1)
Change that line to cnstl = cnstl.ravel()
X = array([1.4 + 1j*2.7, -4.9 + 1j*8.3]) Out = array(X) error =array(X) for i in xrange(2): for j in xrange(256): a0 = real(X[i]) < (real(cnstl[j])+1) a1 = real(X[i]) > (real(cnstl[j])-1) a2 = imag(X[i]) > (imag(cnstl[j])-1) a3 = imag(X[i]) < (imag(cnstl[j])+1) if (a0 & a1 & a2 &a3): Out[i] = cnstl[j] error[i] = X[i] - cnstl[j]
After reindenting this correctly, I get the following results: In [22]: out Out[22]: array([ 1.+3.j, -5.+9.j]) In [23]: error Out[23]: array([ 0.4-0.3j, 0.1-0.7j]) Are those correct? -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco

Thanks for the correction. I will learn the ravel() function since I do not know it. Moving from Matlab world into python is tricky sometime. Your output In [22]: out> Out[22]: array([ 1.+3.j, -5.+9.j])> > In [23]: error> Out[23]: array([ 0.4-0.3j, 0.1-0.7j]) are correct answer. However, if my data set is large, this solution takes long time to run. Are there any python/numpy magic to speed it up? Thanks Frank> Date: Fri, 30 Jan 2009 14:52:27 -0600> From: robert.kern@gmail.com> To: numpy-discussion@scipy.org> Subject: Re: [Numpy-discussion] help on fast slicing on a grid> > On Fri, Jan 30, 2009 at 12:58, frank wang <f.yw@hotmail.com> wrote:> > I have created a test example for the question using for loop and hope> > someone can help me to get fast solution. My data set is about 2000000 data.> >> > However, I have the problem to run the code, the Out[i]=cnstl[j] line gives> > me error says:> >> > In [107]: Out[0]=cnstl[0]> > ---------------------------------------------------------------------------> > TypeError Traceback (most recent call last)> > C:\Frank_share\qamslicer.py in <module>()> > ----> 1> > 2> > 3> > 4> > 5> > TypeError: can't convert complex to float; use abs(z)> > In [108]: cnstl.dtype> > Out[108]: dtype('complex128')> >> > I do not know why that my data is complex128 already. Can anyone help to> > figure why?> > It's an odd error message, certainly. The root of the problem is that> you are attempting to put a (1,)-shaped array into a scalar. You don't> want to do that.> > By the way, you don't want to use the variable name Out in IPython.> It's already used to capture the output.> > > Thanks> >> > Frank> >> > from numpy import *> > a = arange(-15,16,2)> > cnstl=a.reshape(16,1)+1j*a> > cnstl=cnstl.reshape(256,1)> > Change that line to> > cnstl = cnstl.ravel()> > > X = array([1.4 + 1j*2.7, -4.9 + 1j*8.3])> > Out = array(X)> > error =array(X)> > for i in xrange(2):> > for j in xrange(256):> > a0 = real(X[i]) < (real(cnstl[j])+1)> > a1 = real(X[i]) > (real(cnstl[j])-1)> > a2 = imag(X[i]) > (imag(cnstl[j])-1)> > a3 = imag(X[i]) < (imag(cnstl[j])+1)> > if (a0 & a1 & a2 &a3):> > Out[i] = cnstl[j]> > error[i] = X[i] - cnstl[j]> > After reindenting this correctly, I get the following results:> > In [22]: out> Out[22]: array([ 1.+3.j, -5.+9.j])> > In [23]: error> Out[23]: array([ 0.4-0.3j, 0.1-0.7j])> > > Are those correct?> > -- > Robert Kern> > "I have come to believe that the whole world is an enigma, a harmless> enigma that is made terrible by our own mad attempt to interpret it as> though it had an underlying truth."> -- Umberto Eco> _______________________________________________> Numpy-discussion mailing list> Numpy-discussion@scipy.org> http://projects.scipy.org/mailman/listinfo/numpy-discussion _________________________________________________________________ Windows Live™ Hotmail®:…more than just e-mail. http://windowslive.com/explore?ocid=TXT_TAGLM_WL_t2_hm_justgotbetter_explore...

On Fri, Jan 30, 2009 at 22:41, frank wang <f.yw@hotmail.com> wrote:
Thanks for the correction. I will learn the ravel() function since I do not know it. Moving from Matlab world into python is tricky sometime.
Your output In [22]: out
Out[22]: array([ 1.+3.j, -5.+9.j])
In [23]: error Out[23]: array([ 0.4-0.3j, 0.1-0.7j])
are correct answer.
However, if my data set is large, this solution takes long time to run. Are there any python/numpy magic to speed it up?
from numpy import * a = arange(-15,16,2) cnstl = a[:,newaxis] + 1j*a cnstl = cnstl.ravel() X = array([1.4 + 1j*2.7, -3.9 + 1j*8.3]) out = around((X + 1+1j) / 2.0) * 2.0 - (1+1j) error = X - out print out print error -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco

Hi, Bob, Thanks. This solution works great. It really helps me a lot. Frank> Date: Fri, 30 Jan 2009 23:08:35 -0600> From: robert.kern@gmail.com> To: numpy-discussion@scipy.org> Subject: Re: [Numpy-discussion] help on fast slicing on a grid> > On Fri, Jan 30, 2009 at 22:41, frank wang <f.yw@hotmail.com> wrote:> > Thanks for the correction. I will learn the ravel() function since I do not> > know it. Moving from Matlab world into python is tricky sometime.> >> > Your output> > In [22]: out> >> Out[22]: array([ 1.+3.j, -5.+9.j])> >>> >> In [23]: error> >> Out[23]: array([ 0.4-0.3j, 0.1-0.7j])> >> > are correct answer.> >> > However, if my data set is large, this solution takes long time to run. Are> > there any python/numpy magic to speed it up?> > from numpy import *> > a = arange(-15,16,2)> cnstl = a[:,newaxis] + 1j*a> cnstl = cnstl.ravel()> X = array([1.4 + 1j*2.7, -3.9 + 1j*8.3])> > out = around((X + 1+1j) / 2.0) * 2.0 - (1+1j)> error = X - out> > print out> print error> > -- > Robert Kern> > "I have come to believe that the whole world is an enigma, a harmless> enigma that is made terrible by our own mad attempt to interpret it as> though it had an underlying truth."> -- Umberto Eco> _______________________________________________> Numpy-discussion mailing list> Numpy-discussion@scipy.org> http://projects.scipy.org/mailman/listinfo/numpy-discussion _________________________________________________________________ Windows Live™ Hotmail®:…more than just e-mail. http://windowslive.com/explore?ocid=TXT_TAGLM_WL_t2_hm_justgotbetter_explore...
participants (7)
-
Charles R Harris
-
Christian Heimes
-
Damian Eads
-
David Cournapeau
-
frank wang
-
Jarrod Millman
-
Robert Kern