[Tutor] running multiple concurrent processes

Prasad, Ramit ramit.prasad at jpmorgan.com
Tue Oct 30 21:17:25 CET 2012

richard kappler wrote:
> As I sit through the aftermath of Sandy, I have resumed my personal quest to learn python. One of the things I
> am struggling with is running multiple processes. I read the docs on threading and am completely lost so am
> turning to the most excellent tutors here (and thanks for all the help, past, present and future btw!).
> In what ways can one run multiple concurrent processes and which would be considered the "best" way or is that
> case dependent?
> Example:
> I'm working on programming a robot in Python. The bot has an Arduino board that receives sensor input and sends
> the data to a laptop which is the "brain" of the bot via pySerial and uses this incoming sensor data to help
> determine state. State is used in decision making. The laptop runs a program that we'll call the Master Control
> Program in a nod to Tron. The bot also has a chat program, computer vision, some AI it uses to mine the web for
> information, several other functions. Each of these  concurrent programs (thus far all python programs) must run
> continuously and feed data to the MCP which receives the data, makes decisions and sends out appropriate action
> commands such as for movement, change of state, conversation direction, what to research, etc.
> So, to return to the original question, how does one run multiple concurrent processes within python?
> regards, Richard

The typical ways to do parallel execution is to use the 
multiprocessing module or use some type of threading (threading module).

I am not really sure what your bot does, given that it seems to 
include an interface to some chat program, does it really even
need to do things in the background?  Most chat bots can just
listen and then take action only when necessary.

1. Look for chat line to take action.
2. Get data from sources (Arduino, web, etc) to decide State/action.
3. Do action.
4. Repeat 1-3.

Your program does sound fairly complicated and it may very well
need to run everything in parallel. If so, you could use 
threading with callbacks that would modify the current "state"
as needed. You could also setup various programs that update
a database (SQLlite) and then the MCP queries the db to decide
on the action. An upside is that it is easier to (re-)start/test 
any single component while a downside is that monitoring needs
to occur for each component. 

I would seriously evaluate whether you need true parallel
processing or just something that is "fast enough" to seem like it.
The more I think about it, the more I like the use of a SQLlite

Ramit Prasad

This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  

More information about the Tutor mailing list