A buildbot farm with shell access  for free ?
Hi,
Just saw that on one ML:
http://www.snakebite.org/ http://mail.python.org/pipermail/pythoncommitters/2009January/000331.html
Bottom line: it looks like there is a set of machines which were donated to the PSF for buildbot *with shell access* so that people can fix problems appearing on some platforms. If you look at the email, there are some 'exotic' machines that mere mortals cannot have access to (like True64 on Itanium: to quote the email "massive quad Itanium 2 RX5670s, chock full of 73GB 15k disks and no less than 78GB of RAM between the two servers; 32GB in one and 46GB in the other"). There are also windows machines available.
It is said in the email that this is reserved to the python project, and prominent python projects like Twisted and Django. Would it be ok to try to be qualified as a prominent python project as well ?
cheers,
David
On Wed, Jan 28, 2009 at 7:11 PM, David Cournapeau < david@ar.media.kyotou.ac.jp> wrote:
Hi,
Just saw that on one ML:
http://www.snakebite.org/ http://mail.python.org/pipermail/pythoncommitters/2009January/000331.html
Bottom line: it looks like there is a set of machines which were donated to the PSF for buildbot *with shell access* so that people can fix problems appearing on some platforms. If you look at the email, there are some 'exotic' machines that mere mortals cannot have access to (like True64 on Itanium: to quote the email "massive quad Itanium 2 RX5670s, chock full of 73GB 15k disks and no less than 78GB of RAM between the two servers; 32GB in one and 46GB in the other"). There are also windows machines available.
It is said in the email that this is reserved to the python project, and prominent python projects like Twisted and Django. Would it be ok to try to be qualified as a prominent python project as well ?
Ohhh... I love buildbots. Go for it.
Chuck
Sounds like a great idea!
On 1/28/09, David Cournapeau david@ar.media.kyotou.ac.jp wrote:
Hi,
Just saw that on one ML:
http://www.snakebite.org/ http://mail.python.org/pipermail/pythoncommitters/2009January/000331.html
Bottom line: it looks like there is a set of machines which were donated to the PSF for buildbot *with shell access* so that people can fix problems appearing on some platforms. If you look at the email, there are some 'exotic' machines that mere mortals cannot have access to (like True64 on Itanium: to quote the email "massive quad Itanium 2 RX5670s, chock full of 73GB 15k disks and no less than 78GB of RAM between the two servers; 32GB in one and 46GB in the other"). There are also windows machines available.
It is said in the email that this is reserved to the python project, and prominent python projects like Twisted and Django. Would it be ok to try to be qualified as a prominent python project as well ?
cheers,
David _______________________________________________ Numpydiscussion mailing list Numpydiscussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpydiscussion
On Wed, Jan 28, 2009 at 6:11 PM, David Cournapeau david@ar.media.kyotou.ac.jp wrote:
It is said in the email that this is reserved to the python project, and prominent python projects like Twisted and Django. Would it be ok to try to be qualified as a prominent python project as well ?
That would be great.
Hi,
I have to buidl a grid with 256 point by the command: a = arange(15,16,2) L = len(a) cnstl = a.reshape(L,1)+1j*a
My problem is that I have a big data array that contains the data round the points in cnstl. I want to slice the point to the closest cnstl point and also compute the error. The condition is in the middle of the two point in x and y axis. I can do it in a for loop. Since Python and numpy have a lot of magic, I want to find an efficient way to do. This problem arise from QAM 256 modulation.
Thanks
Frank _________________________________________________________________ Windows Live™ Hotmail®:…more than just email. http://windowslive.com/explore?ocid=TXT_TAGLM_WL_t2_hm_justgotbetter_explore...
On Wed, Jan 28, 2009 at 23:52, frank wang f.yw@hotmail.com wrote:
Hi,
I have to buidl a grid with 256 point by the command: a = arange(15,16,2) L = len(a) cnstl = a.reshape(L,1)+1j*a
My problem is that I have a big data array that contains the data round the points in cnstl. I want to slice the point to the closest cnstl point and also compute the error. The condition is in the middle of the two point in x and y axis. I can do it in a for loop. Since Python and numpy have a lot of magic, I want to find an efficient way to do. This problem arise from QAM 256 modulation.
Can you show us the for loop? I'm not really sure what you want to compute.
Here is the for loop that I am think about. Also, I do not know whether the where commands can handle the complicated logic. The where command basically find the data in the square around the point cnstl[j].
Let the data array is qam with size N
Out = X error = X for i in arange(N): for j in arange(L): aa = np.where((real(X)<real(cnstl[j])+1) & (real(X)>real(cnstl[j])1) & (imag(X)<imag(cnstl[j])+1) & (imag(X)>imag(cnstl[j]1)) Out[aa]=cnstl[j] error[aa]=abs(X)**2  abs(cnstl[j])**2
Thanks
Frank
Date: Wed, 28 Jan 2009 23:57:16 0600> From: robert.kern@gmail.com> To: numpydiscussion@scipy.org> Subject: Re: [Numpydiscussion] help on fast slicing on a grid> > On Wed, Jan 28, 2009 at 23:52, frank wang f.yw@hotmail.com wrote:> >> > Hi,> >> > I have to buidl a grid with 256 point by the command:> > a = arange(15,16,2)> > L = len(a)> > cnstl = a.reshape(L,1)+1j*a> >> > My problem is that I have a big data array that contains the data round the> > points in cnstl. I want to slice the point to the closest cnstl point and> > also compute the error. The condition is in the middle of the two point in x> > and y axis. I can do it in a for loop. Since Python and numpy have a lot of> > magic, I want to find an efficient way to do. This problem arise from QAM> > 256 modulation.> > Can you show us the for loop? I'm not really sure what you want to compute.> >  > Robert Kern> > "I have come to believe that the whole world is an enigma, a harmless> enigma that is made terrible by our own mad attempt to interpret it as> though it had an underlying truth.">  Umberto Eco> _______________________________________________> Numpydiscussion mailing list> Numpydiscussion@scipy.org> http://projects.scipy.org/mailman/listinfo/numpydiscussion
_________________________________________________________________ Windows Live™: Email. Chat. Share. Get more ways to connect. http://windowslive.com/howitworks?ocid=TXT_TAGLM_WL_t2_allup_howitworks_0120...
On Thu, Jan 29, 2009 at 00:09, frank wang f.yw@hotmail.com wrote:
Here is the for loop that I am think about. Also, I do not know whether the where commands can handle the complicated logic. The where command basically find the data in the square around the point cnstl[j].
cnstl is a 2D array from your previous description.
Let the data array is qam with size N
I don't see qam anywhere. Did you mean X?
Out = X error = X
Don't you want something like zeros_like(X) for these?
for i in arange(N): for j in arange(L): aa = np.where((real(X)<real(cnstl[j])+1) & (real(X)>real(cnstl[j])1) & (imag(X)<imag(cnstl[j])+1) & (imag(X)>imag(cnstl[j]1)) Out[aa]=cnstl[j] error[aa]=abs(X)**2  abs(cnstl[j])**2
I'm still confused. Can you show me a complete, working script with possibly fake data?
Hi, Bob,
Thanks for your help.
I am sorry for my type error. qam array is the X array in my example.
cntl is a complex array contains the point (x,y) axises.
I will try to make a workable example. Also I will try to find out the zeros_like function. However, I guess that zeros_like(X) will create an array the same size as X. It it is. Then the two line Out=X and error=X should be Out=zeros_like(X) and error=zeros(X).
Also, can where command handel the logic command?
aa = np.where((real(X)<real(cnstl[j])+1) & (real(X)>real(cnstl[j])1) & (imag(X)<imag(cnstl[j])+1) & (imag(X)>imag(cnstl[j]1))
For example, cntl[j]=3+1j*5, then the where command is the same as:
aa = np.where((real(X)<4) & (real(X)>2 )& (imag(X)<6) & (imag(X)>4))
Thanks
Frank> Date: Thu, 29 Jan 2009 00:15:48 0600> From: robert.kern@gmail.com> To: numpydiscussion@scipy.org> Subject: Re: [Numpydiscussion] help on fast slicing on a grid> > On Thu, Jan 29, 2009 at 00:09, frank wang f.yw@hotmail.com wrote:> > Here is the for loop that I am think about. Also, I do not know whether the> > where commands can handle the complicated logic.> > The where command basically find the data in the square around the point> > cnstl[j].> > cnstl is a 2D array from your previous description.> > > Let the data array is qam with size N> > I don't see qam anywhere. Did you mean X?> > > Out = X> > error = X> > Don't you want something like zeros_like(X) for these?> > > for i in arange(N):> > for j in arange(L):> > aa = np.where((real(X)<real(cnstl[j])+1) &> > (real(X)>real(cnstl[j])1) & (imag(X)<imag(cnstl[j])+1) &> > (imag(X)>imag(cnstl[j]1))> > Out[aa]=cnstl[j]> > error[aa]=abs(X)**2  abs(cnstl[j])**2> > I'm still confused. Can you show me a complete, working script with> possibly fake data?> >  > Robert Kern> > "I have come to believe that the whole world is an enigma, a harmless> enigma that is made terrible by our own mad attempt to interpret it as> though it had an underlying truth.">  Umberto Eco> _______________________________________________> Numpydiscussion mailing list> Numpydiscussion@scipy.org> http://projects.scipy.org/mailman/listinfo/numpydiscussion _________________________________________________________________ Windows Live™: Email. Chat. Share. Get more ways to connect. http://windowslive.com/explore?ocid=TXT_TAGLM_WL_t2_allup_explore_012009
I have created a test example for the question using for loop and hope someone can help me to get fast solution. My data set is about 2000000 data.
However, I have the problem to run the code, the Out[i]=cnstl[j] line gives me error says:
In [107]: Out[0]=cnstl[0]TypeError Traceback (most recent call last) C:\Frank_share\qamslicer.py in <module>()> 1 2 3 4 5 TypeError: can't convert complex to float; use abs(z) In [108]: cnstl.dtypeOut[108]: dtype('complex128') I do not know why that my data is complex128 already. Can anyone help to figure why?
Thanks
Frank
from numpy import *a = arange(15,16,2)cnstl=a.reshape(16,1)+1j*acnstl=cnstl.reshape(256,1) X = array([1.4 + 1j*2.7, 4.9 + 1j*8.3]) Out = array(X)error =array(X)for i in xrange(2): for j in xrange(256): a0 = real(X[i]) < (real(cnstl[j])+1) a1 = real(X[i]) > (real(cnstl[j])1) a2 = imag(X[i]) > (imag(cnstl[j])1) a3 = imag(X[i]) < (imag(cnstl[j])+1) if (a0 & a1 & a2 &a3): Out[i] = cnstl[j] error[i] = X[i]  cnstl[j]
From: f.yw@hotmail.comTo: numpydiscussion@scipy.orgSubject: RE: [Numpydiscussion] help on fast slicing on a gridDate: Wed, 28 Jan 2009 23:28:47 0700
Hi, Bob, Thanks for your help. I am sorry for my type error. qam array is the X array in my example. cntl is a complex array contains the point (x,y) axises. I will try to make a workable example. Also I will try to find out the zeros_like function. However, I guess that zeros_like(X) will create an array the same size as X. It it is. Then the two line Out=X and error=X should be Out=zeros_like(X) and error=zeros(X). Also, can where command handel the logic command? aa = np.where((real(X)<real(cnstl[j])+1) & (real(X)>real(cnstl[j])1) & (imag(X)<imag(cnstl[j])+1) & (imag(X)>imag(cnstl[j]1)) For example, cntl[j]=3+1j*5, then the where command is the same as: aa = np.where((real(X)<4) & (real(X)>2 )& (imag(X)<6) & (imag(X)>4)) Thanks Frank> Date: Thu, 29 Jan 2009 00:15:48 0600> From: robert.kern@gmail.com> To: numpydiscussion@scipy.org> Subject: Re: [Numpydiscussion] help on fast slicing on a grid> > On Thu, Jan 29, 2009 at 00:09, frank wang f.yw@hotmail.com wrote:> > Here is the for loop that I am think about. Also, I do not know whether the> > where commands can handle the complicated logic.> > The where command basically find the data in the square around the point> > cnstl[j].> > cnstl is a 2D array from your previous description.> > > Let the data array is qam with size N> > I don't see qam anywhere. Did you mean X?> > > Out = X> > error = X> > Don't you want something like zeros_like(X) for these?> > > for i in arange(N):> > for j in arange(L):> > aa = np.where((real(X)<real(cnstl[j])+1) &> > (real(X)>real(cnstl[j])1) & (imag(X)<imag(cnstl[j])+1) &> > (imag(X)>imag(cnstl[j]1))> > Out[aa]=cnstl[j]> > error[aa]=abs(X)**2  abs(cnstl[j])**2> > I'm still confused. Can you show me a complete, working script with> possibly fake data?> >  > Robert Kern> > "I have come to believe that the whole world is an enigma, a harmless> enigma that is made terrible by our own mad attempt to interpret it as> though it had an underlying truth.">  Umberto Eco> _______________________________________________> Numpydiscussion mailing list> Numpydiscussion@scipy.org> http://projects.scipy.org/mailman/listinfo/numpydiscussion
Windows Live™: Email. Chat. Share. Get more ways to connect. Check it out. _________________________________________________________________ Hotmail® goes where you go. On a PC, on the Web, on your phone. http://www.windowslivehotmail.com/learnmore/versatility.aspx#mobile?ocid=TX...
On Fri, Jan 30, 2009 at 12:58, frank wang f.yw@hotmail.com wrote:
I have created a test example for the question using for loop and hope someone can help me to get fast solution. My data set is about 2000000 data.
However, I have the problem to run the code, the Out[i]=cnstl[j] line gives me error says:
In [107]: Out[0]=cnstl[0]
TypeError Traceback (most recent call last) C:\Frank_share\qamslicer.py in <module>() > 1 2 3 4 5 TypeError: can't convert complex to float; use abs(z) In [108]: cnstl.dtype Out[108]: dtype('complex128')
I do not know why that my data is complex128 already. Can anyone help to figure why?
It's an odd error message, certainly. The root of the problem is that you are attempting to put a (1,)shaped array into a scalar. You don't want to do that.
By the way, you don't want to use the variable name Out in IPython. It's already used to capture the output.
Thanks
Frank
from numpy import * a = arange(15,16,2) cnstl=a.reshape(16,1)+1j*a cnstl=cnstl.reshape(256,1)
Change that line to
cnstl = cnstl.ravel()
X = array([1.4 + 1j*2.7, 4.9 + 1j*8.3]) Out = array(X) error =array(X) for i in xrange(2): for j in xrange(256): a0 = real(X[i]) < (real(cnstl[j])+1) a1 = real(X[i]) > (real(cnstl[j])1) a2 = imag(X[i]) > (imag(cnstl[j])1) a3 = imag(X[i]) < (imag(cnstl[j])+1) if (a0 & a1 & a2 &a3): Out[i] = cnstl[j] error[i] = X[i]  cnstl[j]
After reindenting this correctly, I get the following results:
In [22]: out Out[22]: array([ 1.+3.j, 5.+9.j])
In [23]: error Out[23]: array([ 0.40.3j, 0.10.7j])
Are those correct?
Thanks for the correction. I will learn the ravel() function since I do not know it. Moving from Matlab world into python is tricky sometime.
Your output In [22]: out> Out[22]: array([ 1.+3.j, 5.+9.j])> > In [23]: error> Out[23]: array([ 0.40.3j, 0.10.7j]) are correct answer.
However, if my data set is large, this solution takes long time to run. Are there any python/numpy magic to speed it up?
Thanks
Frank> Date: Fri, 30 Jan 2009 14:52:27 0600> From: robert.kern@gmail.com> To: numpydiscussion@scipy.org> Subject: Re: [Numpydiscussion] help on fast slicing on a grid> > On Fri, Jan 30, 2009 at 12:58, frank wang f.yw@hotmail.com wrote:> > I have created a test example for the question using for loop and hope> > someone can help me to get fast solution. My data set is about 2000000 data.> >> > However, I have the problem to run the code, the Out[i]=cnstl[j] line gives> > me error says:> >> > In [107]: Out[0]=cnstl[0]> > > > TypeError Traceback (most recent call last)> > C:\Frank_share\qamslicer.py in <module>()> > > 1> > 2> > 3> > 4> > 5> > TypeError: can't convert complex to float; use abs(z)> > In [108]: cnstl.dtype> > Out[108]: dtype('complex128')> >> > I do not know why that my data is complex128 already. Can anyone help to> > figure why?> > It's an odd error message, certainly. The root of the problem is that> you are attempting to put a (1,)shaped array into a scalar. You don't> want to do that.> > By the way, you don't want to use the variable name Out in IPython.> It's already used to capture the output.> > > Thanks> >> > Frank> >> > from numpy import *> > a = arange(15,16,2)> > cnstl=a.reshape(16,1)+1j*a> > cnstl=cnstl.reshape(256,1)> > Change that line to> > cnstl = cnstl.ravel()> > > X = array([1.4 + 1j*2.7, 4.9 + 1j*8.3])> > Out = array(X)> > error =array(X)> > for i in xrange(2):> > for j in xrange(256):> > a0 = real(X[i]) < (real(cnstl[j])+1)> > a1 = real(X[i]) > (real(cnstl[j])1)> > a2 = imag(X[i]) > (imag(cnstl[j])1)> > a3 = imag(X[i]) < (imag(cnstl[j])+1)> > if (a0 & a1 & a2 &a3):> > Out[i] = cnstl[j]> > error[i] = X[i]  cnstl[j]> > After reindenting this correctly, I get the following results:> > In [22]: out> Out[22]: array([ 1.+3.j, 5.+9.j])> > In [23]: error> Out[23]: array([ 0.40.3j, 0.10.7j])> > > Are those correct?> >  > Robert Kern> > "I have come to believe that the whole world is an enigma, a harmless> enigma that is made terrible by our own mad attempt to interpret it as> though it had an underlying truth.">  Umberto Eco> _______________________________________________> Numpydiscussion mailing list> Numpydiscussion@scipy.org> http://projects.scipy.org/mailman/listinfo/numpydiscussion _________________________________________________________________ Windows Live™ Hotmail®:…more than just email. http://windowslive.com/explore?ocid=TXT_TAGLM_WL_t2_hm_justgotbetter_explore...
On Fri, Jan 30, 2009 at 22:41, frank wang f.yw@hotmail.com wrote:
Thanks for the correction. I will learn the ravel() function since I do not know it. Moving from Matlab world into python is tricky sometime.
Your output In [22]: out
Out[22]: array([ 1.+3.j, 5.+9.j])
In [23]: error Out[23]: array([ 0.40.3j, 0.10.7j])
are correct answer.
However, if my data set is large, this solution takes long time to run. Are there any python/numpy magic to speed it up?
from numpy import *
a = arange(15,16,2) cnstl = a[:,newaxis] + 1j*a cnstl = cnstl.ravel() X = array([1.4 + 1j*2.7, 3.9 + 1j*8.3])
out = around((X + 1+1j) / 2.0) * 2.0  (1+1j) error = X  out
print out print error
Hi, Bob,
Thanks. This solution works great. It really helps me a lot.
Frank> Date: Fri, 30 Jan 2009 23:08:35 0600> From: robert.kern@gmail.com> To: numpydiscussion@scipy.org> Subject: Re: [Numpydiscussion] help on fast slicing on a grid> > On Fri, Jan 30, 2009 at 22:41, frank wang f.yw@hotmail.com wrote:> > Thanks for the correction. I will learn the ravel() function since I do not> > know it. Moving from Matlab world into python is tricky sometime.> >> > Your output> > In [22]: out> >> Out[22]: array([ 1.+3.j, 5.+9.j])> >>> >> In [23]: error> >> Out[23]: array([ 0.40.3j, 0.10.7j])> >> > are correct answer.> >> > However, if my data set is large, this solution takes long time to run. Are> > there any python/numpy magic to speed it up?> > from numpy import *> > a = arange(15,16,2)> cnstl = a[:,newaxis] + 1j*a> cnstl = cnstl.ravel()> X = array([1.4 + 1j*2.7, 3.9 + 1j*8.3])> > out = around((X + 1+1j) / 2.0) * 2.0  (1+1j)> error = X  out> > print out> print error> >  > Robert Kern> > "I have come to believe that the whole world is an enigma, a harmless> enigma that is made terrible by our own mad attempt to interpret it as> though it had an underlying truth.">  Umberto Eco> _______________________________________________> Numpydiscussion mailing list> Numpydiscussion@scipy.org> http://projects.scipy.org/mailman/listinfo/numpydiscussion _________________________________________________________________ Windows Live™ Hotmail®:…more than just email. http://windowslive.com/explore?ocid=TXT_TAGLM_WL_t2_hm_justgotbetter_explore...
David Cournapeau wrote:
It is said in the email that this is reserved to the python project, and prominent python projects like Twisted and Django. Would it be ok to try to be qualified as a prominent python project as well ?
Give it some time. Nobody  not even the Python core devs  have access to the machines. It's going to take at least several weeks to get the infrastructure running and to establish a policy.
From my perspective NumPy and Sage both count as prominent Python
projects. Heck, you are in competition with tools like Matlab and you ain't looking bad! Furthermore you could make better use of the machines than Django because you are using way more C extensions and esoteric libraries.
I recommend you subscribe to the snakebite list and bring up your interest. You got my +1 already. For now the list is snakebitelist@googlegroups.com but it will move to another server (probably Python.org) eventually.
Christian
participants (7)

Charles R Harris

Christian Heimes

Damian Eads

David Cournapeau

frank wang

Jarrod Millman

Robert Kern