[Tutor] (de)serialization questions

[Albert-Jan Roskam]

Hi Lee,

Thanks for your response.

Maybe my main question is as follows: what permanent object is most suitable to 
store a large amount of entries (maybe too many to fit into the computer's 
memory), which can be looked up very fast.

Eventually, I want to create two objects:
1-one to look up street name and city using zip code
2-one to look up zip code using street name, apartment number and city

I stored object1 in a marshalled dictionary. Its length is about 450.000 (I live 
in Holland, not THAT many streets). Look-ups are incredibly fast (it has to, 
because it's part of an autocompletion feature of a data entry program). I 
haven't got the street number data needed for object2 yet, but it's going to be 
much larger. Many streets have different zip codes for odd or even numbers, or 
the zip codes are divided into street number ranges (for long streets).

You suggest to simply use a file. I like simple solutions, but doesn't that, by 
definition, require a slow, linear search? I chose a dictionary for object1 
because hash tables are very fast in terms of lookup times. Another option might 
be to use the bisect module. Not sure if using a binary search algoritm is much 
faster with these still relatively small numbers. It's interesting for sure, but 
I don't know whether using bisect would justify the added amount of complexity 
and bug risks.

Funny you should mention sqlite: I was just considering it yesterday. Gosh, 
Python has so much interesting stuff to offer! 


Cheers!!
Albert-Jan


~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
All right, but apart from the sanitation, the medicine, education, wine, public 
order, irrigation, roads, a fresh water system, and public health, what have the 
Romans ever done for us?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~




________________________________
From: Lee Harr <missive at hotmail.com>
To: tutor at python.org
Sent: Fri, October 1, 2010 12:37:58 AM
Subject: Re: [Tutor] (de)serialization questions


> I have data about zip codes, street and city names (and perhaps later also of
> street numbers). I made a dictionary of the form {zipcode: (street, city)}

One dictionary with all of the data?

That does not seem like it will work. What happens when
2 addresses have the same zip code?


> Are there forms of object permanence that do not read all data into memory?

How about a file?

You could write your data one record per line, and then
read it in one line at a time also.



You could also use a csv file, or maybe use a database
like sqlite.

                          
_______________________________________________
Tutor maillist  -  Tutor at python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor



      
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20101001/218b48cc/attachment.html>