There is one thing that TxMongo does that PyMongo, Motor and the rest don't do: handle deadlines and timeouts. We're (HGST/WesternDigital) using MongoDB (with WiredTiger) in real-time applications and one of our requirements was to honour application set deadlines (or timeouts) on a per-call basis. So if you are interested in that, we have it. :) For those just tuning in... Timeout: how many seconds the call is allowed run before raising an error. Deadline: a time (unix timestamp) in the future that the call is allowed to run before raising an error. We take advantage of Twisted's native deferred cancellation mechanism in addition to validation checks that no modifications happen to MongoDB when a TimeExceeded is raised. We've been very satisfied with how MongoDB has performed (post 3.0/WT) along with TxMongo and Twisted. Cheers, Bret On Fri, Oct 9, 2015 at 7:52 PM, Jonathan Vanasco <twisted-python@2xlp.com> wrote:
On Oct 9, 2015, at 1:06 PM, Clayton Daley wrote:
I (OP) was looking into these libraries/options as part of a side project that's been delayed by real work. I'm also relegating Mongo to a caching layer after stumbling upon Sarah Mei's "Why You Should Never Use MongoDB" and realizing I was going to have the exact same problems
Thanks for the update!
FWIW, (sorry, non-twisted!) if you want to avoid mongodb:
1. We use redis for KV caching, and its really, really, really fast. 2. Riak can use LevelDB as a backing store -- leveldb supports snappy compression on documents 3. If you need the document search, i really like Elastic
I don't really like/trust mongodb from past experience myself. I would normally not use it for anything other than a read-through cache... except for my current need. We have a web spider and need to archive the snapshots. WireTiger, which was bought by mongodb and integrated into the 3.x branch, is the only storage backend I've found that will do "bucket level compression". I forget if it's applied on pages or blocks, but it will basically apply zlib or snappy compression onto sets of documents. I looked for weeks on something that could do that, it was the only option. We did a lot of storage tests on our typical data, and bucketed compression saves way more space than document compression or document compression using a seed.
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python