execnet-1.0.8 and a green experiment
holger at merlinux.eu
Sat Jul 31 00:34:04 CEST 2010
I just released execnet-1.0.8, see the Changelog for details, below.
What is execnet?
execnet is a small pure python library that manages distributed code execution
across Python versions and host boundaries. It provides a self-bootstrapping
share-nothing model with channels to send/receive data messages. It also
helps with managing a cluster of (local or remote) processes and comes
with tons of examples and API docs:
green version of execnet
For those interested i hacked up a green version of execnet
which is API-compatible (and less stable) than the "main"
version. It requires the eventlet library and is thus not fully
self-bootstrapping (you need to install eventlet on all remote nodes
you want to execute code on). There is no release but you can peruse
and "python setup.py develop" to play with it. It will never spawn
any operating system threads. Rather remote_exec() will spawn
greenlets/micro-threads of which you can easily have a few thousand.
It probably only is stable enough for simple scenarios at this point.
Changelog 1.0.8 (compared to 1.0.7):
- new ``gateway.remote_exec(func, **kwargs)`` style fo executing
a pure function with parameters. The function on the remote
side also needs to accept a ``channel`` which allows it to
communicate back and forth. Thanks to Ronny Pfannschmidt
for implementing it with special kudos to Maciej Fijalkowski
for writing a "pure-function" checker so that on Python2.6
onwards non-pure functions will be rejected.
- enhance rsyncing to also sync permissions (stat().st_mode)
of directories and files.
(should also resolve http://bitbucket.org/hpk42/py-trunk/issue/68/)
- fix rsyncing of symlinks, thanks to Charles Solar
(should also resolve http://bitbucket.org/hpk42/py-trunk/issue/70/)
- update internal usage of apipkg to 1.0b6
- remote_exec(module) now makes sure that the linecache is updated
before reading and sending the source. thanks Ronny, Matt.
- removed all trailing whitespace from source files
More information about the Python-announce-list