Mailman 3 September 2010 - SciPy-User

Stereoscopic analysis
by majoris.ursae＠yahoo.co.uk Sept. 15, 2010

Sept. 15, 2010

Hello everybody, I would be grateful if someone give me some hints on how to extract average height from two overlapping aerial photographs preferably in Python environment. Thank you!

2 1

ANN: job opening at TradeLink Securities
by John Hunter Sept. 14, 2010

Sept. 14, 2010

We are looking to hire a programmer/researcher to help research and develop trading ideas, and to develop infrastructure to put these trading strategies into production. We are looking for someone who is bright and curious with a quantitative background and a strong interest in writing good code and building systems that work. Familiarity with basic probability and statistics is a definite plus. We do not require a financial background, but are looking for someone with an enthusiasm to dive into this industry and learn a lot. We do most of our data modeling and production software in python, utilizing mostly open source tools. We have a lot of ideas to test and hopefully put into production, and you'll be working with a fast paced and friendly small team of traders and programmers. Applying: Please submit a resume and cover letter to qsjobs(a)trdlnk.com with the subject header "TLS Job". In your cover letter, please address how your background, experience and skills will fit into the position described above. We are looking for a full-time, on-site candidate only. About Us: TradeLink Securities, headquartered in Chicago, IL, is a NASDAQ market maker and registered broker dealer with FINRA. The firm specializes in both quantitative and fundamental investing in the US and abroad. TradeLink Securities is a division of TradeLink Holdings, a diversified alternative investment and proprietary trading firm organized in 1979. For more information visit our website at http://tradelinkll.com.

1 1

how to get indices
by Marcel Blattner Sept. 13, 2010

Sept. 13, 2010

Hi there I have a small problem. I have for example a csc_matrix A. I want to find indices of values bigger then a treshold. In numpy I simply can do: a,b = A > threshold...is there something similar in Scipy? Thanks Marcel

2 2

How to disable warning display ?
by Mohammad Abdollahi Sept. 12, 2010

Sept. 12, 2010

Dear All, I keep getting a long list of stupid warnings when i'm running my code. I was just wondering if anybody knows how to disable warning printing at the output, as it takes a lot of time. i just dont want them to be printed. any idea ? Kind Regards

2 1

Re: [SciPy-User] [Numpy-discussion] scan array to extract min-max values (with if condition)
by Massimo Di Stefano Sept. 11, 2010

Sept. 11, 2010

Thanks Pierre, i tried it and all works fine and fast. my apologize :-( i used a wrong "if" statment to represent my needs if mydata[i,0] < E or mydata[i,0] > W or mydata[i,1] < N or mydata[i,1] > S : ^^ totally wrong for my needs^^ this "if" instead : if W < mydata[i,0] < E and S < mydata[i,1] < N: should reflect your example : yselect = (data[:,1] <= N) & (data[:,1] >= S) xselect = (data[:,0] <= E) & (data[:,0] >= W) selected_data = data[xselect & yselect] a question, how to code a masked array, as in the Brett's code, to reflect the new (right) if statment ? i'm asking this to try to learn how to use masked array. thanks a lot for your support!!! Massimo. Il giorno 11/set/2010, alle ore 23.00, Pierre GM ha scritto: > > On Sep 11, 2010, at 9:53 PM, Massimo Di Stefano wrote: > >> have you hints on how to get working the "MASK" code ? >> as it is now it pick all the points in the "mydata" array. > > Brett's code for the mask matched the loop of your post. However, taking a second look at it, I don't see why it would work. Mind trying something like > > # Selection on the NS axis > yselect = (mydata[:,1] <= N) & (mydata[:,1] >= S) > # selection on the EW axis > xselect = (mydata[:,1] <= E) & (mydata[:,1] >= W) > # Global selection > selected_data = mydata[xselect & yselect] > > Now you can check in selected data that the coordinates are indeed in the rectangle that you want, and take the min/max of your data as needed. > let me know how it goes > > P. > > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion(a)scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion

1 0

Re: [SciPy-User] [Numpy-discussion] scan array to extract min-max values (with if condition)
by Massimo Di Stefano Sept. 11, 2010

Sept. 11, 2010

Brett, i tried a different way to solve the problem, using : ############# import os fpath = '/Users/sasha/py/' input_fp = open( os.path.join(fpath, 'BE3730072600WC20050817.txt'), 'r' ) input_file = input_fp.readlines() N = 234560.94503118 S = 234482.56929822 E = 921336.53116178 W = 921185.3779625 xL = [] yL = [] zL = [] for index, line in enumerate( input_file ): if index == 0: print 'skipping header line...' else: x, y, z = line.split(',') xL.append( float(x) * 0.3048006096012 ) yL.append( float(y) * 0.3048006096012 ) zL.append( float(z) * 0.3048006096012 ) xLr = [] yLr = [] zLr = [] for coords in zip(xL, yL, zL): if W < coords[0] < E and S < coords[1] < N: xLr.append( coords[0] ) yLr.append( coords[1] ) zLr.append( coords[2] ) elements = len(xLr) minZ = min(zLr) maxZ = max(zLr) ############ using the same input file i posted early, it give me 966 elements instead of 158734 elements gived by your "MASK" example the input file contains 158734 elements, this means the mask code : > mask |= mydata[:,0] < E > mask |= mydata[:,0] > W > mask |= mydata[:,1] < N > mask |= mydata[:,1] > S is not working as aspected have you hints on how to get working the "MASK" code ? as it is now it pick all the points in the "mydata" array. thanks! Massimo. Il giorno 11/set/2010, alle ore 16.19, Brett Olsen ha scritto: > On Sat, Sep 11, 2010 at 7:45 AM, Massimo Di Stefano > <massimodisasha(a)gmail.com> wrote: >> Hello All, >> >> i need to extract data from an array, that are inside a >> rectangle area defined as : >> >> N, S, E, W = 234560.94503118, 234482.56929822, 921336.53116178, 921185.3779625 >> >> the data are in a csv (comma delimited text file, with 3 columns X,Y,Z) >> >> #X,Y,Z >> 3020081.5500,769999.3100,0.0300 >> 3020086.2000,769991.6500,0.4600 >> 3020099.6600,769996.2700,0.9000 >> ... >> ... >> >> i read it using " numpy.loadtxt " >> >> data : >> >> http://www.geofemengineering.it/data/csv.txt 5,3 mb (158735 rows) >> >> to extract data that are inside the boundy-box area (N, S, E, W) i'm using a loop >> inside a function like : >> >> import numpy as np >> >> def getMinMaxBB(data, N, S, E, W): >> mydata = data * 0.3048006096012 >> for i in range(len(mydata)): >> if mydata[i,0] < E or mydata[i,0] > W or mydata[i,1] < N or mydata[i,1] > S : >> if i == 0: >> newdata = np.array((mydata[i,0],mydata[i,1],mydata[i,2]), float) >> else : >> newdata = np.vstack((newdata,(mydata[i,0], mydata[i,1], mydata[i,2]))) >> results = {} >> results['Max_Z'] = newdata.max(0)[2] >> results['Min_Z'] = newdata.min(0)[2] >> results['Num_P'] = len(newdata) >> return results >> >> >> N, S, E, W = 234560.94503118, 234482.56929822, 921336.53116178, 921185.3779625 >> data = '/Users/sasha/csv.txt' >> mydata = np.loadtxt(data, comments='#', delimiter=',') >> out = getMinMaxBB(mydata, N, S, E, W) >> >> print out > > Use boolean arrays to index the parts of your array that you want to look at: > > def newGetMinMax(data, N, S, E, W): > mydata = data * 0.3048006096012 > mask = np.zeros(mydata.shape[0], dtype=bool) > mask |= mydata[:,0] < E > mask |= mydata[:,0] > W > mask |= mydata[:,1] < N > mask |= mydata[:,1] > S > results = {} > results['Max_Z'] = mydata[mask,2].max() > results['Min_Z'] = mydata[mask,2].min() > results['Num_P'] = mask.sum() > return results > > This runs about 5000 times faster on my machine. > > Brett > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion(a)scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion

1 0

scan array to extract min-max values (with if condition)
by Massimo Di Stefano Sept. 11, 2010

Sept. 11, 2010

Hello All, i need to extract data from an array, that are inside a rectangle area defined as : N, S, E, W = 234560.94503118, 234482.56929822, 921336.53116178, 921185.3779625 the data are in a csv (comma delimited text file, with 3 columns X,Y,Z) #X,Y,Z 3020081.5500,769999.3100,0.0300 3020086.2000,769991.6500,0.4600 3020099.6600,769996.2700,0.9000 ... ... i read it using " numpy.loadtxt " data : http://www.geofemengineering.it/data/csv.txt 5,3 mb (158735 rows) to extract data that are inside the boundy-box area (N, S, E, W) i'm using a loop inside a function like : import numpy as np def getMinMaxBB(data, N, S, E, W): mydata = data * 0.3048006096012 for i in range(len(mydata)): if mydata[i,0] < E or mydata[i,0] > W or mydata[i,1] < N or mydata[i,1] > S : if i == 0: newdata = np.array((mydata[i,0],mydata[i,1],mydata[i,2]), float) else : newdata = np.vstack((newdata,(mydata[i,0], mydata[i,1], mydata[i,2]))) results = {} results['Max_Z'] = newdata.max(0)[2] results['Min_Z'] = newdata.min(0)[2] results['Num_P'] = len(newdata) return results N, S, E, W = 234560.94503118, 234482.56929822, 921336.53116178, 921185.3779625 data = '/Users/sasha/csv.txt' mydata = np.loadtxt(data, comments='#', delimiter=',') out = getMinMaxBB(mydata, N, S, E, W) print out This method works, but maybe is not soo fast, have you any hints on how to improve code with better performance ? thanks!!! Massimo.

2 1

MannWhitneyU test critical values
by kossnair＠web.de Sept. 11, 2010

Sept. 11, 2010

2 2

AttributeError with scikits.timeseries.tstables
by Tim Michelsen Sept. 11, 2010

Sept. 11, 2010

Hello, saving a masked array (time series) to a HDF5 file through the pytables interface with scikits.timeseries.tstables I received the following error message: File "C:\Python26\lib\site-packages\scikits\timeseries\lib\tstables.py", line 714, in createTimeSeriesTable chunkshape=chunkshape, byteorder=byteorder) File "C:\Python26\lib\site-packages\scikits\timeseries\lib\tstables.py", line 419, in __init__ _log=_log) File "C:\Python26\lib\site-packages\scikits\timeseries\lib\tstables.py", line 292, in __init__ self.attrs.special_attrs = self._update_special_attrs(maskedarray) File "C:\Python26\lib\site-packages\scikits\timeseries\lib\tstables.py", line 425, in _update_special_attrs special_attrs['recshape'] = tuple(a._varshape) AttributeError: 'MaskedArray' object has no attribute '_varshape' I would love to get help on why the hdf write process fails here. Do I need to fill missing values? Thanks in advance, Timmie

1 1

naming columns in array
by Dan Lussier Sept. 11, 2010

Sept. 11, 2010

I was wondering if there was possible to change the dtpye of an array once it exits. Here is my situation. I have either a N x 1 column, or a N x M array where I would like to be able to name the columns after the array exists. I have tried working with record arrays a bit, but no luck. If someone might be able to scratch a quick example I'd be grateful. Thanks!

3 2