Mailman 3 November 2013 - SciPy-User

How to display a RGB image (MFISH)?
by jp April 3, 2023

April 3, 2023

Hi, I have written a script to try to combine five images into a three channels RGB images<http://dip4fish.blogspot.com/2011/05/trying-to-combine-mfish-images-into-rg…> . The resulting RGB image looks like a grey scale image instead of a color one. I have three np.array Rnorm, Gnorm, Bnorm which are copied into a rgb array: *rgb = np.zeros((shape[0],shape[1],3),dtype=float) mxr=np.max(R) mxg=np.max(G) mxb=np.max(B) Rnorm=np.uint8((255*(R/mxr)))Gnorm=np.uint8((255*(R/mxg)))Bnorm=np.uint8((255*(R/mxb)))#copy each RGB component in an RGB array rgb[:,:,0]=Rnorm rgb[:,:,1]=Gnorm rgb[:,:,2]=Bnorm* *pylab.subplot(224, aspect='equal',frameon=False, xticks=[], yticks=[]) pylab.imshow(rgb) pylab.show()* Any advice? Thank you Jean-Patrick

3 4

Invitation to connect on LinkedIn
by Jelle Feringa via LinkedIn July 28, 2022

July 28, 2022

LinkedIn ------------ Jelle Feringa requested to add you as a connection on LinkedIn: ------------------------------------------ Jose, I'd like to add you to my professional network on LinkedIn. - Jelle Accept invitation from Jelle Feringa http://www.linkedin.com/e/-3wy1w2-gpkzwpgn-5n/Q6WKH0LACopGJkAw_6fSqajo6R7VM… View invitation from Jelle Feringa http://www.linkedin.com/e/-3wy1w2-gpkzwpgn-5n/Q6WKH0LACopGJkAw_6fSqajo6R7VM… ------------------------------------------ DID YOU KNOW your LinkedIn profile helps you control your public image when people search for you? Setting your profile as public means your LinkedIn profile will come up when people enter your name in leading search engines. Take control of your image! http://www.linkedin.com/e/-3wy1w2-gpkzwpgn-5n/ewp/inv-22/ -- (c) 2011, LinkedIn Corporation

2 1

Help required regding Eigenvectors
by Sudheer Phani March 14, 2022

March 14, 2022

Hello All, I have been encountering some strange problems with the calculation of eigen vectors using python. The problem is with the sign of the eigen vectors calculated. On Very few occasions the eigen vectors calculated using python have an opposite sign than the actual eigenvectors, though the magnitude is correct. Can any one of you plz help me in this regard and suggest me some solution if you have encoutered similar problems . Any help in this regd is appreciated Thanks in Advace regds Sudheer

5 5

Allan Variance
by Daniele Nicolodi Feb. 4, 2014

Feb. 4, 2014

Hello, does anyone know or have written and is willing to share some code to compute the Allan variance, modified Allan variance and friends, ideally both from frequency than from phase data? I'm writing my own code, it is not that difficult, but I would prefer the time required for testing and validation doing something more fun. If there is no available code implementing Allan variance, would it be interesting to include in in SciPy? Thank you so much. Best, Daniele

3 7

Re: [SciPy-User] SciPy-User Digest, Vol 120, Issue 12
by Michal Romaniuk Jan. 24, 2014

Jan. 24, 2014

Hi, > Hi, > > On Mon, Aug 19, 2013 at 7:44 AM, Michal Romaniuk > <michal.romaniuk06(a)imperial.ac.uk> wrote: >> Hi, >> >> I'm saving a large batch of data using savemat and although I get no >> errors, the files produced are not readable for either matlab or scipy. >> Is there a limit on file size? > > Ah - yes there is - the individual matrices in the mat file cannot be > larger than 4GB. Is it possible you hit this limit? > > Sorry, I only realized this when Richard Llewellyn pointed this out a > couple of weeks ago on the list: > > http://scipy-user.10969.n7.nabble.com/SciPy-User-scipy-io-loadmat-throws-Ty… > > The current scipy code has an error message for matrices that are too large. > > Cheers, > > Matthew Well, I managed to work around the problem to some extent by setting do_compression=True. Now Matlab can read those files (so they must be valid to some extent) but SciPy can't (even though they were written with SciPy). I get this error: PATH/lib/python2.6/site-packages/scipy/io/matlab/mio.pyc in loadmat(file_name, mdict, appendmat, **kwargs) 173 variable_names = kwargs.pop('variable_names', None) 174 MR = mat_reader_factory(file_name, appendmat, **kwargs) --> 175 matfile_dict = MR.get_variables(variable_names) 176 if mdict is not None: 177 mdict.update(matfile_dict) PATH/lib/python2.6/site-packages/scipy/io/matlab/mio5.pyc in get_variables(self, variable_names) 290 continue 291 try: --> 292 res = self.read_var_array(hdr, process) 293 except MatReadError, err: 294 warnings.warn( PATH/lib/python2.6/site-packages/scipy/io/matlab/mio5.pyc in read_var_array(self, header, process) 253 `process`. 254 ''' --> 255 return self._matrix_reader.array_from_header(header, process) 256 257 def get_variables(self, variable_names=None): PATH/lib/python2.6/site-packages/scipy/io/matlab/mio5_utils.so in scipy.io.matlab.mio5_utils.VarReader5.array_from_header (scipy/io/matlab/mio5_utils.c:5401)() PATH/lib/python2.6/site-packages/scipy/io/matlab/mio5_utils.so in scipy.io.matlab.mio5_utils.VarReader5.array_from_header (scipy/io/matlab/mio5_utils.c:4849)() PATH/lib/python2.6/site-packages/scipy/io/matlab/mio5_utils.so in scipy.io.matlab.mio5_utils.VarReader5.read_real_complex (scipy/io/matlab/mio5_utils.c:5602)() ValueError: total size of new array must be unchanged The size of the main array is about 9 GB before compression, but the compressed files are less than 500 MB and closer to 400 MB. There are some other arrays in the file too but they are much smaller. Any ideas on how I could get SciPy to read this data back? Right now I can only think of storing the data in single precision format... Thanks, Michal

3 4

log normal distribution random number array generation
by Josè Luis Mietta Dec. 4, 2013

Dec. 4, 2013

Hi experts! I wanna generate a random number array of size=N using a log normal distribution. From http://en.wikipedia.org/wiki/Log-normal_distribution i wanna use the parameters mu and sigma. I know that I must do: form scipy.stats import lognorm new_array = lognorm.rvs(......, size=N) What must I set like parameters (loc, s, scale, etc.) for use mu and sigma distribution parameters. In the same way: what must I do in new_array = norm.rvs(......, size=N) for generate a array of random numbers using a gaussian distribution with parameters mu and sigma? Waitign for your answers. Thanks a lot!

4 3

Matplotlib 1.3.1: plot(matrix("1, 2, 3")) -> RuntimeError: maximum recursion depth exceeded
by Klaus Dec. 3, 2013

Dec. 3, 2013

Hi, I am working with python 2.7.5 using - numpy.__version__: 1.7.1 - - matplotlib.__version__: 1.3.1 When I start "ipython2 --pylab" and execute the following code x = matrix("1,2,3") > plot(x) I get the error message [...] > /usr/lib/python2.7/site-packages/matplotlib/units.pyc in > get_converter(self, x) > 146 except AttributeError: > 147 # not a masked_array > --> 148 converter = self.get_converter(xravel[0]) > 149 return converter > 150 > /usr/lib/python2.7/site-packages/numpy/matrixlib/defmatrix.py in > __getitem__(self, index) > 303 > 304 try: > --> 305 out = N.ndarray.__getitem__(self, index) > 306 finally: > 307 self._getitem = False > RuntimeError: maximum recursion depth exceeded In the older matplotlib version 1.3.0 this error was not present. Any help is highly appreciated!

2 1

Repeated Measure ANOVA
by Horea Christian Dec. 3, 2013

Dec. 3, 2013

Hi guys, I would like to compare reaction times for a series of experimental conditions. My data comes from ~100 trial repetitions over 10 participants (yielding ~1000 trials). I was told that just doing an ANOVA on this dataset would be improper, because the 1000 measurements are not truly independent - and that the proper way to do this is called a repeated measure ANOVA. I have tried to look for a scipy function for this and found nothing. In this relevant discussion<http://scipy-user.10969.n7.nabble.com/stats-repeated-measures-ANOVA-td15997…>a participant pointed the following out: "Repeated measures" ANOVA is just a misnomer for using the "randomized > block design" as a substitute for not knowing MANOVA or Hotelling's T- > square test, and as such leads to conclusions that are very hard to > interpret. The real value of repeated measures ANOVA in medical > litterature is often to inform the reader that the authors don't > understand the statistics they use ;-) I would like to know whether I'm looking for the right thing at all, and if yes how I could accomplish this with scipy

2 1

sum of array for masked area only
by questions anon Nov. 29, 2013

Nov. 29, 2013

Hi All, I am not completely sure if this is the correct place for this question, I have a separate text file for daily rainfall data that covers the whole country. I would like to calculate the monthly mean, min, max and the mean of the sum for one state. The mean, max and min are just the mean, max and min for all data in that month however the sum data needs to work out the total for the month across the array and then sum that value. I use gdal tools to mask out the rest of the country and I use numpy tools for the summary stats. I can get the max, min and mean for the state, but the mean of the sum keeps giving me a result for the whole country rather than just the state, even though I am performing the analysis on the state only data. I am not sure if this is a masking issue or a numpy calculation issue. The mask works fine for the other summary statistics. Any feedback will be greatly appreciated! import numpy as np import matplotlib.pyplot as plt from numpy import ma as MA from mpl_toolkits.basemap import Basemap from datetime import datetime as dt from datetime import timedelta import os from StringIO import StringIO from osgeo import gdal, gdalnumeric, ogr, osr import glob import matplotlib.dates as mdates import sys shapefile=r"/Users/state.shp" ## Create masked array from shapefile xmin,ymin,xmax,ymax=[111.975,-9.975, 156.275,-44.525] ncols,nrows=[886, 691] #Your rows/cols maskvalue = 1 xres=(xmax-xmin)/float(ncols) yres=(ymax-ymin)/float(nrows) geotransform=(xmin,xres,0,ymax,0, -yres) 0 src_ds = ogr.Open(shapefile) src_lyr=src_ds.GetLayer() dst_ds = gdal.GetDriverByName('MEM').Create('',ncols, nrows, 1 ,gdal.GDT_Byte) dst_rb = dst_ds.GetRasterBand(1) dst_rb.Fill(0) #initialise raster with zeros dst_rb.SetNoDataValue(0) dst_ds.SetGeoTransform(geotransform) err = gdal.RasterizeLayer(dst_ds, [maskvalue], src_lyr) dst_ds.FlushCache() mask_arr=dst_ds.GetRasterBand(1).ReadAsArray() np.set_printoptions(threshold='nan') mask_arr[mask_arr == 255] = 1 newmask=MA.masked_equal(mask_arr,0) ### calculate monthly summary stats for state Only rainmax=[] rainmin=[] rainmean=[] rainsum=[] yearmonthlist=[] yearmonth_int=[] errors=[] OutputFolder=r"/outputfolder" GLOBTEMPLATE = r"/daily-rainfall/combined/rainfall-{year}/r{year}{month:02}??.txt" def accumulate_month(year, month): files = glob.glob(GLOBTEMPLATE.format(year=year, month=month)) monthlyrain=[] for ifile in files: try: f=np.genfromtxt(ifile,skip_header=6) except: print "ERROR with file:", ifile errors.append(ifile) f=np.flipud(f) stateonly_f=np.ma.masked_array(f, mask=newmask.mask) # this masks data to state print "stateonly_f:", stateonly_f.max(), stateonly_f.mean(), stateonly_f.sum() monthlyrain.append(stateonly_f) yearmonth=dt(year,month,1) yearmonthlist.append(yearmonth) yearmonthint=str(year)+str(month) d=dt.strptime(yearmonthint, '%Y%m') print d date_string=dt.strftime(d,'%Y%m') yearmonthint=int(date_string) yearmonth_int.append(yearmonthint) r_sum=np.sum(monthlyrain, axis=0) r_mean_of_sum=MA.mean(r_sum) r_max, r_mean, r_min=MA.max(monthlyrain), MA.mean(monthlyrain), MA.min(monthlyrain) rainmax.append(r_max) rainmean.append(r_mean) rainmin.append(r_min) rainsum.append(r_mean_of_sum) print " state only:", yearmonthint,r_max, r_mean, r_min, r_mean_of_sum

2 2

scipy.optimize.minimize constraint optimization
by Kevin Kunzmann Nov. 29, 2013

Nov. 29, 2013

Hi, I have a question concerning constraint optimization with scipy. After setting up my problem using scipy.optimize.minimize and the SLSQP method the procedure failed to converge and the final iteration was a bit awkward. So I inspected the iteration steps closely and the problem seems to be, that the bounds / constraints are not respected in every step. This blows up some function evaluations, as they are only well defined for the feasible set. Is there a way of doing general purpose, nonlinear, constraint optimization, that respects the feasible set at ALL times in scipy (or at least in python?). The Matlab implementations of fmincon using sqp and trust-region-reflective do implement that feature http://www.mathworks.de/de/help/optim/ug/writing-constraints.html#br9p_ry but i would rather stick with python if possible... If that feature is available already in scipy an comment on that in the help would be superb :) thank you, Kevin

3 2