Distributed Cache Server?

[ken.faulkner at gmail.com]

har.... nah, think I'll give BT a miss for this purpose.

Yeah, know it was a loaded question (no specifics), but was really only
thinking out aloud.

The plans are:

- Multiple copies of data on different machines (just thinking 2 copies
of the data)
- Load spread over multiple machines... anything between 2->lots
(again, not being specific).
- Data itself I'm expecting to be low.... a few Gb at tops (but
ofcourse, as this extends I might want to increase the size)
- Haven't thought a LOT about automatic handling of errors...... yet.
- and its assumed to be on a trusted network.

Apart from memcached, I haven't seen anything similar to what I'm after
(plus to be perfectly honest..... I wouldn't mind coding it up anyway)
;)

Was just "feeling about" to see if anything else is out there.

KenF

ps. btw, only part of it is theoretical, a lot of this I've already
coded, but not targetted towards large scale (100's machines etc)... So
am thinking what I'd need to consider to increase this to a larger
scale.




Roger Binns wrote:
> <ken.faulkner at gmail.com> wrote in message news:1131252445.366398.250160 at g43g2000cwa.googlegroups.com...
> > Does anyone know if a "distributed caching system" has been developed
> > for use with Python?
>
> BitTorrent :-)
>
> > Yes, "distributed caching system" is a bit of a general term, but am
> > really just talking about something as simple as key + value (arbitrary
> > class) which can be split over a number of machines in an efficient
> > manner.
>
> You'll need to define what the sweet spot is that you are aiming for.
> Are we talking tens of thousands of keys or billions?  How big is the
> data (megabytes, gigabytes, terabytes?)  Do you need transactional
> integrity (eg when are updates seen by other readers)?  Do you want
> redundancy (data duplicated on multiple machines)?  How many machines
> are we talking about?  Should failure be automatically detected?  Is
> there a need for security or treating the machines as untrusted?
> 
> Roger