Execution state persistence for workflow application

[Bengt Richter]

On Mon, 24 Nov 2003 08:11:57 +0100, Paolo Losi <p.losi at netline.it> wrote:

>Hi all,
>	I'm pretty new to the python language so please excuse me
>if this is FAQ... I'm very glad to be part of the list! :-)
>
>I'm looking into a way to implement a generic workflow framework with python.
>The basic idea is to use python scripts as the way to specify workflow
>behavior. The framework should not only use scripts as a specification language
>but is going to leverage on python interpreter for the execution of the
>scripts.
>One of the possible solution is to have a one to one mapping
>between a interpreter process and a workflow instance.
>
I'm not clear on what all that meant ;-)

for word_or_phrase in ["workflow framework", "workflow behavior", "workflow",
                       "specification language", "intepreter process", "workflow instance"]:
    if you_please(): explain_what_you_mean_by(word_or_phrase)

;-)

>Since I want to be able to model also long running processes I need to
>cope with Execution state persistence. In other words..
>I'd like to stop the framework and restart it having all workflow
>instance processes resume exactly where they left.
>
>Ideally I would be nice to have dumpexec() e loadexec() builtin functions
>to dump and reload the state of the interpreter. I've not seen anything
>like that unfortunately...
That seems pretty large-grained. What happens if you have a power failure
in the middle of dumpexec()?

IOW, is sounds to me like you need something like a transactional database system to log
your state changes, so you can pick up from a consistent state no matter what.
The question then is how to design your system so that it has states that
can be recorded and recovered that way. I think there would be a lot of overhead
in capturing an entire system checkpoint image every time your app trasitioned to
a new state, even if such a snapshot function were available.

So OTTOMH ISTM you will wind up designing some state machine that can be initialized
from a TDB, and which will log state changes incrementally in smaller chunks than
all-encompassing blobs. I don't know what kind of control state you want to persist,
but I would guess it should be in terms of getting from one data transaction to the
next, not python loop variables and heap states and such.

>
>I've tried to look at exec statement but it doesn't seem to be good
>for execution persistence...
>
>Questions are:
>
>- does there exist a python framework for workflow that use
"workflow" is a bit too generic for me to guess what you have in mind.

>   python script as a specification language and that
>   support "long running" workflows? I don't want to reinvent the wheel...
ditto
>
>- does the implementation idea quickly depicted makes sense to you?
>
>- how do you suggest to implement execution persistence? Is there
>   any "standard" solution?
First thing in seeking help thinking about stuff is to define terms,
perhaps informally by examples or metaphors, etc., since everyone doesn't
speak the vernacular of your current focus.

HTH

Regards,
Bengt Richter