[sapug] Stateless Iteration

Daryl Tester Daryl.Tester at iocane.com.au
Mon Jun 12 04:20:23 CEST 2006

Chris Foote wrote:

>  	http://pyro.sourceforge.net
> It has lots of advantages over XMLRPC, particularly speed and TCP
> connection reuse, but has just one disadvantage - Python specificity.

D'oh.  My fault, I should have been more specific in my
generalisation (bear in mind I'm thinking more abstractly than
concretely here - I'm after a general protocol to realise this
on any number of technologies).

XML-RPC highlights the issue, but isn't the cause as such. One
of the reasons I'm looking at using XML-RPC is to allow me to
get Python into some areas where it hasn't previously been
allowed, so while Pyro is cool, it won't solve that aspect of
the problem.  The client most likely won't be written in Python.

I guess (and I'm still mulling over the problem definition in
my head) that what I'm aiming for is to retain as little state
as possible in the transport (could be XML-RPC, but could also
easily be traditional RPC, or REST) and the backend server so
that the server could be restarted without loss of state,
because it won't have any to lose (helps minimise the need
for persistent session management).

Dammit, this where my thinking is breaking down - I'm not
able to articulate the problem in all its gory details.

OK, a concrete example.  I offer the following service

customer.fetch(next, limit)

customer.fetch() returns the first customer (as an array of
1) and an ID of some sort.  This returned ID can be fed into
customer.fetch() as parameter "next" to return the next
customer ID (in this example, it could be the primary key
of the customer - it would map easily) and also return
another ID, which if subsequently fed into customer.fetch()
returns the next customer, and so forth.  The "limit"
parameter allows you to return a bunch of customers, as
round trips can be expensive (and as before, the returned
ID would point to the customer immediately after "limit"
customers).  In this example, no state is maintained by the
server, so it could be rebooted in between requests and the
client could be none the wiser. If the client was rebooted
(or otherwise disappeared), the server doesn't have any
session details to clean up.  Sweet and simple.  (There
are other issues here that I'm glossing over, concurrent
updates leaping to mind).

The problem arises when I'm talking to something that's not
a table with a primary key, like some sort of computed list
(or a table without a primary key :-).  "next" needs to be
something that can be used by the server to reconstruct
enough state to be able to resume the list evaluation from
where it left off.  "Next" is a nonce, and given a complicated
enough nonce this could be done - i.e. the nonce is the remainder
of the list - but then the client wouldn't need to go back
to the server to fetch the remainder of the list when it
already has it (it could be hidden by the client API, but
then my goal in the first place was to lazy evaluate the list
so it wasn't pulled across the link in its entirety).

So I guess what I'm after is some method/pattern where, given
an arbitrary function running on a server, I can extract enough
state from it to be handed over to the client (ignoring for the
moment issues like security) so that when the client says "give
me more data", the server can reconstruct the previous state
from the client supplied data, and take off from there.

And it's taken me 2.5 hours to get to this point, but I think
I understand my problem better now (you're all acting as my
personal sounding board :-).

  Daryl Tester, IOCANE Pty. Ltd.

More information about the sapug mailing list