skip lines at the end of file with loadtxt

i'm tring to generate an array reading a txt file from internet. my target is to use python instead of matlab, to replace this steps in matlab : url=['http://www.cdc.noaa.gov/Correlation/amon.us.long.data']; urlwrite(url,'file.txt'); i'm using this code : urllib.urlretrieve('http://www.cdc.noaa.gov/Correlation/amon.us.long.data', 'file.txt') a = np.loadtxt('file.txt', skiprows=1) but it fails becouse of the txt description at the end of the file, do you know if exist a way to skip the X lines at the end, something like "skipmultiplerows='1,-4'" (to skip the first and the last 4 rows in the file) or i have to use some sort of string manipulation (readlines?) instead ? Thanks! --Massimo

Maybe try genfromtxt instead of loadtxt, it has a skip_footer option. -=- Olivier 2011/10/25 Massimo Di Stefano <massimodisasha@gmail.com>
i'm tring to generate an array reading a txt file from internet.
my target is to use python instead of matlab, to replace this steps in matlab :
url=['http://www.cdc.noaa.gov/Correlation/amon.us.long.data']; urlwrite(url,'file.txt');
i'm using this code :
urllib.urlretrieve('http://www.cdc.noaa.gov/Correlation/amon.us.long.data', 'file.txt') a = np.loadtxt('file.txt', skiprows=1)
but it fails becouse of the txt description at the end of the file,
do you know if exist a way to skip the X lines at the end,
something like "skipmultiplerows='1,-4'" (to skip the first and the last 4 rows in the file)
or i have to use some sort of string manipulation (readlines?) instead ?
Thanks!
--Massimo
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

Many thanks Oliver! i missed it in the description, works great :-) --Massimo. Il giorno 25/ott/2011, alle ore 15.33, Olivier Delalleau ha scritto:
Maybe try genfromtxt instead of loadtxt, it has a skip_footer option.
-=- Olivier
2011/10/25 Massimo Di Stefano <massimodisasha@gmail.com> i'm tring to generate an array reading a txt file from internet. my target is to use python instead of matlab, to replace this steps in matlab :
url=['http://www.cdc.noaa.gov/Correlation/amon.us.long.data']; urlwrite(url,'file.txt');
i'm using this code :
urllib.urlretrieve('http://www.cdc.noaa.gov/Correlation/amon.us.long.data', 'file.txt') a = np.loadtxt('file.txt', skiprows=1)
but it fails becouse of the txt description at the end of the file,
do you know if exist a way to skip the X lines at the end,
something like "skipmultiplerows='1,-4'" (to skip the first and the last 4 rows in the file)
or i have to use some sort of string manipulation (readlines?) instead ?
Thanks!
--Massimo
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

On Tue, Oct 25, 2011 at 12:28 PM, Massimo Di Stefano <massimodisasha@gmail.com> wrote:
urllib.urlretrieve('http://www.cdc.noaa.gov/Correlation/amon.us.long.data', 'file.txt') a = np.loadtxt('file.txt', skiprows=1)
but it fails becouse of the txt description at the end of the file,
It's always hard to stop reading before the end of a file, since we don't know when that's going to happen; I guess it would require a buffered approach. Fortunately, there is an easy workaround in your case. All the text lines start with " ", so simply do: np.loadtxt("amon.us.long.data", comments=" ") You should see a speedup of about 3 times over genfromtxt (loadtxt does much less under the hood). Stéfan
participants (3)
-
Massimo Di Stefano
-
Olivier Delalleau
-
Stéfan van der Walt