execnet-1.0.8 and a green experiment
holger krekel
holger at merlinux.eu
Sat Jul 31 00:34:04 CEST 2010
I just released execnet-1.0.8, see the Changelog for details, below.
What is execnet?
---------------------
execnet is a small pure python library that manages distributed code execution
across Python versions and host boundaries. It provides a self-bootstrapping
share-nothing model with channels to send/receive data messages. It also
helps with managing a cluster of (local or remote) processes and comes
with tons of examples and API docs:
http://codespeak.net/execnet
green version of execnet
--------------------------------
For those interested i hacked up a green version of execnet
which is API-compatible (and less stable) than the "main"
version. It requires the eventlet library and is thus not fully
self-bootstrapping (you need to install eventlet on all remote nodes
you want to execute code on). There is no release but you can peruse
http://code.google.com/p/execnet-green/source/checkout
and "python setup.py develop" to play with it. It will never spawn
any operating system threads. Rather remote_exec() will spawn
greenlets/micro-threads of which you can easily have a few thousand.
It probably only is stable enough for simple scenarios at this point.
best,
holger
Changelog 1.0.8 (compared to 1.0.7):
- new ``gateway.remote_exec(func, **kwargs)`` style fo executing
a pure function with parameters. The function on the remote
side also needs to accept a ``channel`` which allows it to
communicate back and forth. Thanks to Ronny Pfannschmidt
for implementing it with special kudos to Maciej Fijalkowski
for writing a "pure-function" checker so that on Python2.6
onwards non-pure functions will be rejected.
- enhance rsyncing to also sync permissions (stat().st_mode)
of directories and files.
(should also resolve http://bitbucket.org/hpk42/py-trunk/issue/68/)
- fix rsyncing of symlinks, thanks to Charles Solar
(should also resolve http://bitbucket.org/hpk42/py-trunk/issue/70/)
- update internal usage of apipkg to 1.0b6
- remote_exec(module) now makes sure that the linecache is updated
before reading and sending the source. thanks Ronny, Matt.
- removed all trailing whitespace from source files
More information about the Python-announce-list
mailing list