Problem with reading CSV file from URL, last record truncated.

KB keith at
Tue Aug 4 00:54:15 CEST 2009


I am trying to download from a URL, a CSV using the following:

import re
import urllib, urllib2, cookielib
import mechanize
import csv
import numpy
import os

def return_ranking():

	cj = mechanize.MSIECookieJar(delayload=True)
	cj.load_from_registry()  # finds cookie index file from registry

	# set things up for cookies

	opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))


	reply ='


	table = csv.reader(fin)

	for row in table:
		print row


I need to use cookies etc (mechanize/urllib2) for a different, more
complex URL but since it wasn't working, I went back to a simple Yahoo
example (above) which I have working with urllib (not urllib2).

The behaviour I am seeing is that the last record is being truncated:

(sample output)
['2009-04-08', '17.29', '17.33', '16.94', '17.13', '45389100',
['2009-04-07', '17.20', '17.25', '16.58', '16.85', '59902600',

A friend said I should do the above writing out to a file and have
csvreader read in the file, but as you can see, to no avail!

Any help greatly appreciated! Note that urllib.urlretrieve works
perfectly but I give up the ability to import cookies from my registry
which is all important (AFAIK anyway mechanize requires urllib2).

Any help greatly appreciated.

More information about the Python-list mailing list