Using ZODB (or something else) for storing lots of metadata about RSS/Atom feeds and posts

Bruno Desthuilliers bdesth.quelquechose at free.quelquepart.fr
Sat Dec 19 06:15:33 EST 2009


Thomas Doggette a écrit :
> I'm working on (or rather, at this point, planning) a project that
> will involve keeping track of every post in a large number of Atom
> feeds, as well as a lot of metadata about how posts are linked and how
> users interact with them.
> 
> The idea of having all of these be persistent objects is very
> appealing, but I'm not sure the ZODB is ideal for what I'm doing. Can
> people with more experience working with data sets like this give me
> some advice on what they've had luck using?

My own experience with the ZODB for large number of small objects, lot
of metadata and indexes, frequent updates, and querying the whole thing,
has been at least very disappointing.

RDBMS are possibly not the perfect solution for each and any kind of
problem datastructures, but unless you have to deal with complex
heterogenous trees or graphs, they sure are a safe bet for most data
storage, retrieval and analysis jobs.

Also, I don't have ant experience with Google's AppEngine's datastore
nor Apache CouchDB, but given the description of your project they might
be appropriate solutions.

My 2 cents.




More information about the Python-list mailing list