Reading and Comparing Two Files

What is the efficient numpy way to compare data from two different files? For the nth line in each file I want to operate on the numbers. I've been using loadtxt() data_5x5 = N.loadtxt("file5") data_8x8 = N.loadtxt("file8") for line in data_5x5: pos5 = N.array([line[0], line[1], line[2]]) This works fine for one file but how to I get the same line's worth of data from the other file?

2010/2/28 Robert Love <rblove_lists@comcast.net>:
What is the efficient numpy way to compare data from two different files? For the nth line in each file I want to operate on the numbers. I've been using loadtxt()
data_5x5 = N.loadtxt("file5")
data_8x8 = N.loadtxt("file8")
for line in data_5x5: pos5 = N.array([line[0], line[1], line[2]])
I believe there are several ways of doing that, and mine might not be the most efficient at all: for line5, line8 in zip(data_5x5, data_8x8): # line5 and line8 are row vectors of paired lines pass complete = numpy.hstack(data_5x5, data_8x8) # If data_5x5.shape[0] == data_8x8.shape[0], i.e., same number of rows. for line in complete: # complete is comprised of concatenated row vectors. pass for idx in xrange(0, min(data_5x5.shape[0], data_8x8.shape[0])): line5 = data_5x5[idx] line8 = data_8x8[idx] # Do sth with the vectors. Or: a1 = data_5x5[idx, (0, 1, 2)] # Extract items 0, 1, 2 of line idx of first file. a2 = data_8x8[idx, (0, 42)] # Extract items 0, 42 of line idx of second file. ... Friedrich

On Sun, Feb 28, 2010 at 7:24 AM, Friedrich Romstedt <friedrichromstedt@gmail.com> wrote:
2010/2/28 Robert Love <rblove_lists@comcast.net>:
What is the efficient numpy way to compare data from two different files? For the nth line in each file I want to operate on the numbers. I've been using loadtxt()
data_5x5 = N.loadtxt("file5")
data_8x8 = N.loadtxt("file8")
for line in data_5x5: pos5 = N.array([line[0], line[1], line[2]])
If you just want to compare row by row when you already have the arrays, you can just use numpy, e.g. based on first 3 columns: (data_8x8[:,:3] == data_5x5[:,:3]).all(1) but from your question it's not clear to me what you actually want to compare Josef
I believe there are several ways of doing that, and mine might not be the most efficient at all:
for line5, line8 in zip(data_5x5, data_8x8): # line5 and line8 are row vectors of paired lines pass
complete = numpy.hstack(data_5x5, data_8x8) # If data_5x5.shape[0] == data_8x8.shape[0], i.e., same number of rows. for line in complete: # complete is comprised of concatenated row vectors. pass
for idx in xrange(0, min(data_5x5.shape[0], data_8x8.shape[0])): line5 = data_5x5[idx] line8 = data_8x8[idx] # Do sth with the vectors. Or: a1 = data_5x5[idx, (0, 1, 2)] # Extract items 0, 1, 2 of line idx of first file. a2 = data_8x8[idx, (0, 42)] # Extract items 0, 42 of line idx of second file.
...
Friedrich _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

On Feb 28, 2010, at 6:24 AM, Friedrich Romstedt wrote:
2010/2/28 Robert Love <rblove_lists@comcast.net>:
What is the efficient numpy way to compare data from two different files? For the nth line in each file I want to operate on the numbers. I've been using loadtxt()
data_5x5 = N.loadtxt("file5")
data_8x8 = N.loadtxt("file8")
for line in data_5x5: pos5 = N.array([line[0], line[1], line[2]])
I believe there are several ways of doing that, and mine might not be the most efficient at all:
for line5, line8 in zip(data_5x5, data_8x8): # line5 and line8 are row vectors of paired lines pass
complete = numpy.hstack(data_5x5, data_8x8) # If data_5x5.shape[0] == data_8x8.shape[0], i.e., same number of rows. for line in complete: # complete is comprised of concatenated row vectors. pass
for idx in xrange(0, min(data_5x5.shape[0], data_8x8.shape[0])): line5 = data_5x5[idx] line8 = data_8x8[idx] # Do sth with the vectors. Or: a1 = data_5x5[idx, (0, 1, 2)] # Extract items 0, 1, 2 of line idx of first file. a2 = data_8x8[idx, (0, 42)] # Extract items 0, 42 of line idx of second file.
Thank you, I will try this last method listed. I need to actually compute with the values from the two files to perform my comparison and the time tag is in different formats. Your method will get me access to the contents of two files.
participants (3)
-
Friedrich Romstedt
-
josef.pktd@gmail.com
-
Robert Love