steve at holdenweb.com
Wed Oct 3 16:51:42 CEST 2007
Tim Chase wrote:
>>> for lineNum, line in enumerate(f): ...
>>> However, lineNum soon overflows and starts counting backwards. How do
>>> i force enumerate to return long integer?
>> Just how "soon" exactly do you read sys.maxint lines from a file? I
>> should have thought that it would take a significant amount of time to
>> read 2,147,483,647 lines ...
> A modestly (but not overwhelmingly) long time:
> (defining our own xrange-ish generator that can handle things larger
> than longs)
> >>> def xxrange(x):
> ... i = 0
> ... while i < x:
> ... yield i
> ... i += 1
> >>> for i,j in enumerate(xxrange(2**33)): assert i==j
> Traceback (most recent call last):
> File "<stdin>", line 1, in ?
> It took me about an 60-90 minutes to hit the assertion on a dual-core
> 2.8ghz machine under otherwise-light-load. If batch-processing lengthy
> log files or other large data such as genetic data, it's entirely
> possible to hit this limit as the OP discovered.
I wouldn't dream of suggesting it's impossible. I just regard "soon" as
less than an hour in commuter's terms, I suppose.
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
Sorry, the dog ate my .sigline
More information about the Python-list