problem with simple multiprocessing script on OS X

[Benjamin Kaplan]

On Tue, Aug 24, 2010 at 3:31 PM, Darren Dale <dsdale24 at gmail.com> wrote:
> On Aug 23, 9:58 am, Darren Dale <dsdal... at gmail.com> wrote:
>> The following script runs without problems on Ubuntu and Windows 7.
>> h5py is a package wrapping the hdf5 library (http://code.google.com/p/
>> h5py/):
>>
>> from multiprocessing import Pool
>> import h5py
>>
>> def update(i):
>>     print i
>>
>> def f(i):
>>     "hello foo"
>>     return i*i
>>
>> if __name__ == '__main__':
>>     pool = Pool()
>>     for i in range(10):
>>         pool.apply_async(f, [i], callback=update)
>>     pool.close()
>>     pool.join()
>>
>> On OS X 10.6 (tested using python-2.6.5 from MacPorts), I have to
>> comment out the as-yet unused h5py import, otherwise I get a
>> traceback:
>>
>> Exception in thread Thread-1:
>> Traceback (most recent call last):
>>   File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/
>> lib/python2.6/threading.py", line 532, in __bootstrap_inner
>>     self.run()
>>   File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/
>> lib/python2.6/threading.py", line 484, in run
>>     self.__target(*self.__args, **self.__kwargs)
>>   File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/
>> lib/python2.6/multiprocessing/pool.py", line 226, in _handle_tasks
>>     put(task)
>> PicklingError: Can't pickle <type 'function'>: attribute lookup
>> __builtin__.function failed
>
>
> This is a really critical bug for me, but I'm not sure how to proceed.
> Can I file a bug report on the python bugtracker if the only code I
> can come up with to illustrate the problem requires a lame import of a
> third party module?
> --

It's working fine for me, OS X 10.6.4, Python 2.6 and h5py from Macports.