How Can I Increase the Speed of a Large Number of Date Conversions
eddie at holyrood.ed.ac.uk
Fri Jun 8 14:38:35 CEST 2007
vdicarlo <vdicarlo at gmail.com> writes:
>I am a programming amateur and a Python newbie who needs to convert
>about 100,000,000 strings of the form "1999-12-30" into ordinal dates
>for sorting, comparison, and calculations. Though my script does a ton
>of heavy calculational lifting (for which numpy and psyco are a
>blessing) besides converting dates, it still seems to like to linger
>in the datetime and time libraries. (Maybe there's a hot module in
>there with a cute little function and an impressive set of
>Anyway, others in this group have said that the date and time
>libraries are a little on the slow side but, even if that's true, I'm
>thinking that the best code I could come up with to do the conversion
>looks so clunky that I'm probably running around the block three times
>just to go next door. Maybe someone can suggest a faster, and perhaps
>Here's my code, in which I've used a sample date string instead of its
>variable name for the sake of clarity. Please don't laugh loud enough
>for me to hear you in Davis, California.
>dateTuple = time.strptime("2005-12-19", '%Y-%m-%d')
> dateTuple = dateTuple[:3]
> date = datetime.date(dateTuple, dateTuple,
> ratingDateOrd = date.toordinal()
>P.S. Why is an amateur newbie trying to convert 100,000,000 date
>strings into ordinal dates? To win try to win a million dollars, of
>course! In case you haven't seen it, the contest is at www.netflixprize.com.
>There are currently only about 23,648 contestants on 19,164 teams from
>151 different countries competing, so I figure my chances are pretty
I can't help noticing that dates in 'yyyy-mm-dd' format are already sortable
>>> '1999-12-30' > '1999-12-29'
depending on the pattern of access the slightly slower compare speed _might_
compensate for the conversion speed. Worth a try.
More information about the Python-list