Multiprocessing bug, is my editor (SciTE) impeding my progress?

[John Ladasky]

Hi, folks,

Back in 2002, I got back into programming after a nine-year hiatus.  I
needed a new programming language, was guided to Python 2.2, and was
off to the races.  I chose the SciTE program editor, and I have been
using it ever since.  I'm now using Python 2.6 on Ubuntu Linux 10.10.

My programming needs have grown more sophisticated, but I'm still
using SciTE.  Pretty much all of my recent posts to comp.lang.python
have concerned multiprocessing.  I put together a decent system for my
current project, and had it all working.  Then I realized that I
needed to refactor and expand some code, which I did -- and somehow, I
generated a bug that I simply cannot understand.  I've been puzzling
over it for three days.

The error is occurring inside one of my subprocesses.  As far as I
know, SciTE is limited in what it can do in this situation.  The
program does not return when a subprocess generates an exception.  I
see the error message, but then the program simply hangs.

I have tried invoking the subprocess directly without scheduling it
through multiprocessing.Pool.  It works fine.  So the problem is
occurring inside Pool.

I tried opening my code in IDLE, and figured I could step through it,
or at least call functions one line at a time.  It appears that
multiprocessing code is not compatible with IDLE.  IDLE simply crashes
when I try to invoke any of the important functions.

I know, you want me to post a minimal example.  Most of the time,
that's possible, and I do it.  Trust me, this time it isn't.  I have
about 500 lines of code, split across three files.  These implement a
neural network, some test data, and multiprocessing methods for
network evaluation.  I made several concerted changes to the code, and
turned a working system into this:

=============================================

Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python2.6/threading.py", line 532, in
__bootstrap_inner
    self.run()
  File "/usr/lib/python2.6/threading.py", line 484, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/usr/lib/python2.6/multiprocessing/pool.py", line 225, in
_handle_tasks
    put(task)
TypeError: expected string or Unicode object, NoneType found

=============================================

Here's what I think would help me debug this error: I would like to
catch the TypeError, and examine the contents of task.  I need to
accomplish this WITHOUT adding a try...except block to the Python
library file multiprocessing/pool.py.  I don't know whether this is
possible, because the traceback isn't clear about where my OWN code
calls the code which is generating the error.

After that, if the cause of the error still is not obvious, I might
need to go back to the working program.  Somehow I want to examine the
contents of task when the program works, and no TypeError is being
generated.  By comparing the two, I hope to see a difference.  From
that, I should be able to figure out how I have broken what is being
fed to Pool.__init__ and/or MapResult.__init__.

Any suggestions how I might best accomplish this task?  Does this
error message look familiar to anyone?

More generally, should I consider graduating from SciTE?  I have had a
look at a few of the more comprehensive IDE's over the years, and I'll
have to say that I found them to be intimidating.  I found it to be a
huge chore just to open a single Python script and run it inside an
IDE.  It seems like you had to know how to set up a complete, multi-
script project before you could even accomplish simple tasks.  That
steep learning curve is the reason that I didn't choose Java as my
programming language.

So, if any of you have pertinent recommendations in the IDE
department, please feel free to guide me that way.

Thanks!