[Twisted-Python] Released Ampoule - 0.0.5
Ampoule is a process pool implementation written on top of Twisted Matrix. Its name comes from the use of AMP as the default communication protocol between the pool and all its children. It's different from other alternative solutions because it provides an API very close to that of the Twisted ThreadPool. As an helper function it also provides a deferToAMPProcess function that creates the ProcessPool and submits jobs to it. There has been 2 released since the last announce so I'm going to report both of them here. Ampoule 0.0.5 (2008-11-12) ========================== Changes -------- - Fixed bug #259264, this fix introduces a number of changes in the architecture of the project: 1. Removed childReactor argument form the process pool and added a starter argument. 2. Introduced the concept of a starter object whose role is to start subprocesses with given parameters. This makes it easier to specify particular parameters (eg. new env variables) to the child processes without needing to override many methods in the pool using closures. 3. main.py is completely changed and now provides the ProcessStarter object which is a default implementation of IStarter. 4. IStarter interface currently documents only 2 methods: startAMPProcess startPythonProcess in the future it's possible that we will add an additional: startProcess that starts whichever process we want without requiring python, also this might end up with the separation of ProcessPool in at least 2 logical levels: the ProcessPool and a dispatcher that talks with the children, in this way it would be possible to create custom ProcessPools without changing much code or requiring any special requirement on the children. - Introduced a callRemote method on the ProcessPool that is basically the same as doWork. Introduced for symmetry between all the RPC libraries in Twisted. - reactor short name and ampoule child class are now passed as the 2 last arguments rather than the first 2. So if you have written any custom bootstrap code be sure to change sys.argv[1] and sys.argv[2] into sys.argv[-2] and sys.argv[-1] respectively. Ampoule 0.0.4 (2008-11-03) ========================== Changes -------- - Ampoule parent process and child processes now talk using FDs 3 (in) and 4 (out) in order to avoid problems with libraries that mistakenly send error lines to stdout (yes, I'm looking at you gtk+) -- Valentino Volonghi aka Dialtone Now running MacOS X 10.5 Home Page: http://www.twisted.it http://www.adroll.com
Hmm, this sounds interesting. I'm wondering about a use case for it. Would Ampoule be a way to take advantage of multi-core processors? Thanks, Don On Wed, Nov 12, 2008 at 12:09 PM, Valentino Volonghi <dialtone@gmail.com>wrote:
Ampoule is a process pool implementation written on top of Twisted Matrix. Its name comes from the use of AMP as the default communication protocol between the pool and all its children.
It's different from other alternative solutions because it provides an API very close to that of the Twisted ThreadPool. As an helper function it also provides a deferToAMPProcess function that creates the ProcessPool and submits jobs to it.
There has been 2 released since the last announce so I'm going to report both of them here.
Ampoule 0.0.5 (2008-11-12) ==========================
Changes -------- - Fixed bug #259264, this fix introduces a number of changes in the architecture of the project: 1. Removed childReactor argument form the process pool and added a starter argument. 2. Introduced the concept of a starter object whose role is to start subprocesses with given parameters. This makes it easier to specify particular parameters (eg. new env variables) to the child processes without needing to override many methods in the pool using closures. 3. main.py is completely changed and now provides the ProcessStarter object which is a default implementation of IStarter. 4. IStarter interface currently documents only 2 methods: startAMPProcess startPythonProcess in the future it's possible that we will add an additional: startProcess that starts whichever process we want without requiring python, also this might end up with the separation of ProcessPool in at least 2 logical levels: the ProcessPool and a dispatcher that talks with the children, in this way it would be possible to create custom ProcessPools without changing much code or requiring any special requirement on the children.
- Introduced a callRemote method on the ProcessPool that is basically the same as doWork. Introduced for symmetry between all the RPC libraries in Twisted.
- reactor short name and ampoule child class are now passed as the 2 last arguments rather than the first 2. So if you have written any custom bootstrap code be sure to change sys.argv[1] and sys.argv[2] into sys.argv[-2] and sys.argv[-1] respectively.
Ampoule 0.0.4 (2008-11-03) ==========================
Changes -------- - Ampoule parent process and child processes now talk using FDs 3 (in) and 4 (out) in order to avoid problems with libraries that mistakenly send error lines to stdout (yes, I'm looking at you gtk+)
-- Valentino Volonghi aka Dialtone Now running MacOS X 10.5 Home Page: http://www.twisted.it http://www.adroll.com
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
On Nov 12, 2008, at 7:51 PM, Don Smith wrote:
Hmm, this sounds interesting. I'm wondering about a use case for it. Would Ampoule be a way to take advantage of multi-core processors?
Yes, multi-process programming is a way to achieve that advantage, it also supports remote process pools so you can actually take advantage of multiple computers. -- Valentino Volonghi aka Dialtone Now running MacOS X 10.5 Home Page: http://www.twisted.it http://www.adroll.com
participants (2)
-
Don Smith -
Valentino Volonghi