Mailman 3 September 2013 - SciPy-User

May I joint this list?
by Yuxiang Wang Sept. 19, 2013

Sept. 19, 2013

Thank you so much! -- Yuxiang "Shawn" Wang Gerling Research Lab University of Virginia yw5aj(a)virginia.edu +1 (434) 284-0836

2 1

how to get the indexes of non zero rows of sparse matric efficiently when using scipy?
by teccy Sept. 18, 2013

Sept. 18, 2013

for example, a 500x1000 sparse matrix with 300 non zero rows and 5000 non zero items, what i want is the index for the 300 rows, not the index for 5000 items, how to do this efficiently, seems scipy haven't provicde such kind of API ...

2 1

Re: [SciPy-User] solve_banded usage (once again) (Wagner Sebastian)
by federico vaggi Sept. 18, 2013

Sept. 18, 2013

Hey Wagner, not sure how helpful this is, but if you dig into the SciPy source code, you'll see that solve_banded is a fairly thin wrapper around the gbsv function from lapack. If you go look at the lapack source code: http://software.intel.com/sites/products/documentation/hpc/mkl/mklman/GUID-… You can get a better idea of the parameters there. Not sure if this is helpful - I've never really had to use that function myself, but at least it might give you some pointers. Federico

2 1

ANN: SfePy 2013.3
by Robert Cimrman Sept. 18, 2013

Sept. 18, 2013

I am pleased to announce release 2013.3 of SfePy. Description ----------- SfePy (simple finite elements in Python) is a software for solving systems of coupled partial differential equations by the finite element method. The code is based on NumPy and SciPy packages. It is distributed under the new BSD license. Home page: http://sfepy.org Downloads, mailing list, wiki: http://code.google.com/p/sfepy/ Git (source) repository, issue tracker: http://github.com/sfepy Highlights of this release -------------------------- - implementation of Mesh topology data structures in C - implementation of regions based on C Mesh (*) - MultiProblem solver for conjugate solution of subproblems - new advanced examples (vibro-acoustics, Stokes flow with slip conditions) (*) Warning: region selection syntax has been changed in a principal way, see [1]. Besides the simple renaming, all regions meant for boundary conditions or boundary/surface integrals need to have their kind set explicitly to 'facet' (or 'edge' in 2D, 'face' in 3D). [1] http://sfepy.org/doc-devel/users_guide.html#regions For full release notes see http://docs.sfepy.org/doc/release_notes.html#id1 (rather long and technical). Best regards, Robert Cimrman and Contributors (*) (*) Contributors to this release (alphabetical order): Vladimír Lukeš

1 0

Cannot subscribe to mailing lists
by Yuxiang Wang Sept. 18, 2013

Sept. 18, 2013

Hi all, I tried to subscribe to mailing list on page: http://www.scipy.org/scipylib/mailing-lists.html However, I think the links are broken. Would anyone have any idea about where to report it? Thanks! -Shawn

2 1

RAM problem during code execution - Numpy arrays
by Josè Luis Mietta Sept. 18, 2013

Sept. 18, 2013

Hi experts. I need your help with a RAM porblem during execution of my script. I wrote the next code. I use SAGE. In 1-2 hours of execution time the RAM of my laptop (8gb) is filled and the sistem crash: fromscipy.stats importuniform importnumpy asnp cant_de_cadenas =[700,800,900]cantidad_de_cadenas=np.array([])forkkkkk incant_de_cadenas:cantidad_de_cadenas=np.append(cantidad_de_cadenas,kkkkk)cantidad_de_cadenas=np.transpose(cantidad_de_cadenas)b=10h=b Longitud=1numero_experimentos=150densidad_de_cadenas =cantidad_de_cadenas/(b**2)prob_perc=np.array([])tiempos=np.array([])S_int=np.array([])S_medio=np.array([])desviacion_standard=np.array([])desviacion_standard_nuevo=np.array([])anisotropia_macroscopica_porcentual=np.array([])componente_y=np.array([])componente_x=np.array([])importtime forN incant_de_cadenas:empieza=time.clock()PERCOLACION=np.array([])size_medio_intuitivo =np.array([])size_medio_nuevo =np.array([])std_dev_size_medio_intuitivo =np.array([])std_dev_size_medio_nuevo =np.array([])comp_y =np.array([])comp_x =np.array([])foru inxrange(numero_experimentos):perco =Falsearray_x1=uniform.rvs(loc=-b/2,scale=b,size=N)array_y1=uniform.rvs(loc=-h/2,scale=h,size=N)array_angle=uniform.rvs(loc=-0.5*(np.pi),scale=np.pi,size=N)array_pendiente_x=1./np.tan(array_angle)random=uniform.rvs(loc=-1,scale=2,size=N)lambda_sign=np.zeros([N])fort inxrange(N):ifrandom[t]<0:lambda_sign[t]=-1else:lambda_sign[t]=1array_lambdas=(lambda_sign*Longitud)/np.sqrt(1+array_pendiente_x**2)array_x2=array_x1 +array_lambdas*array_pendiente_x array_y2=array_y1 +array_lambdas*1array_x1 =np.append(array_x1,[-b/2,b/2,-b/2,-b/2])array_y1 =np.append(array_y1,[-h/2,-h/2,-h/2,h/2])array_x2 =np.append(array_x2,[-b/2,b/2,b/2,b/2])array_y2 =np.append(array_y2,[h/2,h/2,-h/2,h/2])M =np.zeros([N+4,N+4])forj inxrange(N+4):ifj>0:x_A1B1 =array_x2[j]-array_x1[j]y_A1B1 =array_y2[j]-array_y1[j]x_A1A2 =array_x1[0:j]-array_x1[j]y_A1A2 =array_y1[0:j]-array_y1[j]x_A2A1 =-1*x_A1A2 y_A2A1 =-1*y_A1A2 x_A2B2 =array_x2[0:j]-array_x1[0:j]y_A2B2 =array_y2[0:j]-array_y1[0:j]x_A1B2 =array_x2[0:j]-array_x1[j]y_A1B2 =array_y2[0:j]-array_y1[j]x_A2B1 =array_x2[j]-array_x1[0:j]y_A2B1 =array_y2[j]-array_y1[0:j]p1 =x_A1B1*y_A1A2 -y_A1B1*x_A1A2 p2 =x_A1B1*y_A1B2 -y_A1B1*x_A1B2 p3 =x_A2B2*y_A2B1 -y_A2B2*x_A2B1 p4 =x_A2B2*y_A2A1 -y_A2B2*x_A2A1 condicion_1=p1*p2 condicion_2=p3*p4 fork inxrange (j):ifcondicion_1[k]<=0andcondicion_2[k]<=0:M[j,k]=1delcondicion_1 delcondicion_2 ifj+1<N+4:x_A1B1 =array_x2[j]-array_x1[j]y_A1B1 =array_y2[j]-array_y1[j]x_A1A2 =array_x1[j+1:]-array_x1[j]y_A1A2 =array_y1[j+1:]-array_y1[j]x_A2A1 =-1*x_A1A2 y_A2A1 =-1*y_A1A2 x_A2B2 =array_x2[j+1:]-array_x1[j+1:]y_A2B2 =array_y2[j+1:]-array_y1[j+1:]x_A1B2 =array_x2[j+1:]-array_x1[j]y_A1B2 =array_y2[j+1:]-array_y1[j]x_A2B1 =array_x2[j]-array_x1[j+1:]y_A2B1 =array_y2[j]-array_y1[j+1:]p1 =x_A1B1*y_A1A2 -y_A1B1*x_A1A2 p2 =x_A1B1*y_A1B2 -y_A1B1*x_A1B2 p3 =x_A2B2*y_A2B1 -y_A2B2*x_A2B1 p4 =x_A2B2*y_A2A1 -y_A2B2*x_A2A1 condicion_1=p1*p2 condicion_2=p3*p4 fork inxrange ((N+4)-j-1):ifcondicion_1[k]<=0andcondicion_2[k]<=0:M[j,k+j+1]=1delcondicion_1 delcondicion_2 M[N,N+2]=0M[N,N+3]=0M[N+1,N+2]=0M[N+1,N+3]=0M[N+2,N]=0M[N+2,N+1]=0M[N+3,N]=0M[N+3,N+1]=0CD=np.array([])POPOPO=[]forg inxrange(N):lala=0r=Falsewhilelala<=len(POPOPO)-1:esta=g inPOPOPO[lala]ifesta isTrue:lala=len(POPOPO)r=Trueelse:lala=lala+1ifr isFalse:L=np.array([g])fors inxrange(N):ifM[g,s]!=0:L=np.append(L,s)x=0whilex<=N:forl inxrange(N):z=l inL d=L[x]ifz isFalseandM[d,l]!=0:L=np.append(L,l)ifx+1<len(L):x+=1else:x=N+1.q=len (L)CD=np.append(CD,q)POPOPO.append(L)M_horizontal=M.copy()M_horizontal[:,N+2]=np.zeros(N+4)M_horizontal[:,N+3]=np.zeros(N+4)M_horizontal[N+2]=np.zeros(N+4)M_horizontal[N+3]=np.zeros(N+4)L=np.array([N])fors inxrange(N+4):ifM_horizontal[N,s]!=0:L=np.append(L,s)x=0whilex<=N+4:forl inxrange(N+4):z=l inL d=L[x]ifz isFalseandM_horizontal[d,l]!=0:L=np.append(L,l)ifx+1<len(L):x+=1else:x=(N+4)+1.LV1_in_L =N inL LV2_in_L=(N+1)inL ifLV1_in_L isTrueandLV2_in_L isTrue:perc_horiz=Trueelse:perc_horiz=FalseM_vertical=M.copy()M_vertical[:,N]=np.zeros(N+4)M_vertical[:,N+1]=np.zeros(N+4)M_vertical[N]=np.zeros(N+4)M_vertical[N+1]=np.zeros(N+4)L=np.array([N+2])fors inxrange(N+4):ifM_vertical[N+2,s]!=0:L=np.append(L,s)x=0whilex<=N+4:forl inxrange(N+4):z=l inL d=L[x]ifz isFalseandM_vertical[d,l]!=0:L=np.append(L,l)ifx+1<len(L):x+=1else:x=(N+4)+1.LH1_in_L =(N+2)inL LH2_in_L=(N+3)inL ifLH1_in_L isTrueandLH2_in_L isTrue:perc_ver =Trueelse:perc_ver =Falseifperc_ver isTrueorperc_horiz isTrue:PERCOLACION=np.append(PERCOLACION,1)perco=TrueD =np.array([])W =np.array([])forc inxrange (int(min(CD)),int(max(CD)+1),1):D=np.append(D,c)frec =sum (CD ==c)W =np.append(W,frec)ifperco isTrue:posicion=np.argmax(D)D=np.delete(D,posicion)W=np.delete(W,posicion)iflen(D)==0andlen(W)==0:S_medio_intuitivo_exp_u=0S_medio_nuevo_exp_u =0std_dev_exp_u =0std_dev_nuevo_exp_u =0else:S_medio_intuitivo_exp_u =np.average (D,weights=W)peso_nuevo=D*W S_medio_nuevo_exp_u =np.average (D,weights=peso_nuevo)tipos=sum(W)X=W*((D-S_medio_intuitivo_exp_u)**2)S=sum(X)std_dev_exp_u =np.sqrt(S/(tipos-1.))tipos_nuevo=sum(peso_nuevo)X_nuevo=peso_nuevo*((D-S_medio_nuevo_exp_u)**2)S_nuevo=sum(X_nuevo)std_dev_nuevo_exp_u =np.sqrt(S_nuevo/(tipos_nuevo-1.))componente_longitudinal=Longitud*np.abs(np.cos(array_angle))comp_y=np.append(comp_y,sum(componente_longitudinal)/N)componente_transversal=Longitud*np.abs(np.sin(array_angle))comp_x=np.append(comp_x,sum(componente_transversal)/N)std_dev_size_medio_intuitivo=np.append(std_dev_size_medio_intuitivo,std_dev_exp_u)std_dev_size_medio_nuevo=np.append(std_dev_size_medio_nuevo,std_dev_nuevo_exp_u)size_medio_intuitivo=np.append(size_medio_intuitivo,S_medio_intuitivo_exp_u)size_medio_nuevo=np.append(size_medio_nuevo,S_medio_nuevo_exp_u)percolation_probability=sum(PERCOLACION)/numero_experimentos prob_perc=np.append(prob_perc,percolation_probability)S_int =np.append (S_int,sum(size_medio_intuitivo)/numero_experimentos)S_medio=np.append (S_medio,sum(size_medio_nuevo)/numero_experimentos)desviacion_standard =np.append (desviacion_standard,sum(std_dev_size_medio_intuitivo)/numero_experimentos)desviacion_standard_nuevo=np.append (desviacion_standard_nuevo,sum(std_dev_size_medio_nuevo)/numero_experimentos)tiempos=np.append(tiempos,time.clock()-empieza)componente_y=np.append(componente_y,sum(comp_y)/numero_experimentos)componente_x=np.append(componente_x,sum(comp_x)/numero_experimentos)anisotropia_macroscopica_porcentual=100*(1-(componente_y/componente_x)) I tryed with gc and gc.collect() and 'del'command for deleting arrays after his use and nothing work! What am I doing wrong? Why the memory becomes full while running (starts with 10% of RAM used and in 1-2hour is totally full used)? Please help me, I'm totally stuck! Thanks a lot!

2 1

[ANN] OpenOpt suite v 0.51
by Dmitrey Sept. 15, 2013

Sept. 15, 2013

Hi all, new OpenOpt suite v 0.51 has been released: Some improvements for FuncDesigner automatic differentiation and QP FuncDesigner now can model sparse (MI)(QC)QP Octave QP solver has been connected MATLAB solvers linprog ( LP ), quadprog ( QP ), lsqlin ( LLSP ), bintprog ( MILP ) New NLP solver: knitro Some elements of 2nd order interval analysis, mostly for interalg Some interalg improvements interalg can directly handle (MI)LP and (possibly nonconvex) (MI)(QC)QP New classes: knapsack problem ( KSP ), bin packing problem ( BPP ), dominating set problem ( DSP ) FuncDesigner can model SOCP SpaceFuncs has been adjusted for recent versions of Python and NumPy visit http://openopt.org for more details. Regards, D.

1 0

Z
by Richard Johns Sept. 13, 2013

Sept. 13, 2013

Sent from my iPhone

1 0

Ipython parallel and PBS
by James Sept. 13, 2013

Sept. 13, 2013

Dear all, I'm having a lot of trouble setting up IPython parallel on a PBS cluster, and I would really appreciate anybody helping. The architecture is a standard PBS cluster - a head node with slave nodes. I connect to the head node from my laptop over ssh. The client (laptop) -> Head node connection seems simple enough. The problem is the engines. Ignoring the laptop for a moment, I'll just focus on running ipython on the head node, with the engines on a slave node. I assume this is a correct method of working? I did the following on the head node, following instructions at http://ipython.org/ipython-doc/stable/parallel/parallel_process.html#using-…: $ ipython profile create --parallel --profile=pbs Files are as follows: $cat ipcluster_config.py c = get_config() c.IPClusterStart.controller_launcher_class = 'PBSControllerLauncher' c.IPClusterEngines.engine_launcher_class = 'PBSEngineSetLauncher' c.PBSLauncher.queue = 'long' c.IPClusterEngines.n = 2 # Run 2 cores on 1 node or 2 nodes with all cores? Not sure. $ cat ipengine_config.py c = get_config() Then execute on the head node: $ ipcluster start --profile=pbs -n 2 2013-09-10 15:02:46,771.771 [IPClusterStart] Using existing profile dir: u'/home/username/.ipython/profile_pbs' 2013-09-10 15:02:46.777 [IPClusterStart] Starting ipcluster with [daemon=False] 2013-09-10 15:02:46.778 [IPClusterStart] Creating pid file: /home/username/.ipython/profile_pbs/pid/ipcluster.pid 2013-09-10 15:02:46.778 [IPClusterStart] Starting Controller with PBSControllerLauncher 2013-09-10 15:02:46.792 [IPClusterStart] Job submitted with job id: '2830' 2013-09-10 15:02:47.793 [IPClusterStart] Starting 2 Engines with PBSEngineSetLauncher 2013-09-10 15:02:47.808 [IPClusterStart] Job submitted with job id: '2831' Then the queue shows $ qstat Job id Name User Time Use S Queue ------------------------- ---------------- --------------- -------- - ----- 2830[].master ipcontroller username 0 Q long 2831[].master ipengine username 0 Q long And they just hang there, queued forever. I assume the engines at least should be running? Full information through "qstat -f" doesn't give the reason for the queuing. Normally it would do. There are more than 4 nodes available. $qstat -f Job Id: 2831[].master.domain Job_Name = ipengine Job_Owner = username(a)master.domain job_state = Q queue = long server = [head node's domain address] Checkpoint = u ctime = Tue Sep 10 15:02:47 2013 Error_Path = master.domain:/home/username/ipengine.e2831 Hold_Types = n Join_Path = n Keep_Files = n Mail_Points = a mtime = Tue Sep 10 15:02:47 2013 Output_Path = master.domain:/home/username/ipengine.o2831 Priority = 0 qtime = Tue Sep 10 15:02:47 2013 Rerunable = True [...] etime = Tue Sep 10 15:02:47 2013 submit_args = ./pbs_engines job_array_request = 1-2 fault_tolerant = False submit_host = master.domain init_work_dir = /home/username It also seems strange that the ipcontroller is launched through PBS. I thought this should be on the head node, so I changed 'PBSControllerLauncher' to 'LocalControllerLauncher'. Then it doesn't queue, but I don't know if what I'm doing is correct. Any help would be really greatly appreciated. Thank you. James

2 3

Roll your own python distributions
by David Baddeley Sept. 12, 2013

Sept. 12, 2013

Hi all, I'm wondering if anyone knows of an easy (or relatively easy) way of putting together a scientific python distribution with a one-click installer. I've got a python package with _lots_ of dependencies and would like to give users (with relatively limited computer skills) a simple way of installing python, my package, and all the dependencies. I have previously told people to download EPD, upgrade wxpython, and install a couple of additional packages (which is already pushing it in terms of what the users are comfortable with). The switch to canopy (with the accompanying move to a package management system in which one has to manually select which packages to install) makes this infeasible. The alternative distributions (PythonXY, Anaconda etc ...) are all either 32 bit only, or lack many of the packages I need, meaning that I'd need to get users to download a much longer list of additional packages. I want a python distribution, rather than just a py2exed version as parts of my code don't work well with py2exe. Has anyone encountered this situation, and what did you do? many thanks, David

9 10