problem with simple multiprocessing script on OS X
thomas at jollybox.de
Tue Aug 24 22:32:29 CEST 2010
On Tuesday 24 August 2010, it occurred to Darren Dale to exclaim:
> On Aug 23, 9:58 am, Darren Dale <dsdal... at gmail.com> wrote:
> > The following script runs without problems on Ubuntu and Windows 7.
> > h5py is a package wrapping the hdf5 library (http://code.google.com/p/
> > h5py/):
> > from multiprocessing import Pool
> > import h5py
> > def update(i):
> > print i
> > def f(i):
> > "hello foo"
> > return i*i
> > if __name__ == '__main__':
> > pool = Pool()
> > for i in range(10):
> > pool.apply_async(f, [i], callback=update)
> > pool.close()
> > pool.join()
> > On OS X 10.6 (tested using python-2.6.5 from MacPorts), I have to
> > comment out the as-yet unused h5py import, otherwise I get a
> > traceback:
What on earth is h5py doing there? If what you're telling us is actually
happening, and the code works 1:1 on Linux and Windows, but fails on OSX, and
you're using the same versions of h5py and Python, then the h5py
initialization code is not only enticing multiprocessing to try to pickle
something other than usual, but it is also doing that due to some platform-
dependent witchcraft, and I doubt there's very much separating the OSX
versions from the Linux versions of anything involved.
> This is a really critical bug for me, but I'm not sure how to proceed.
> Can I file a bug report on the python bugtracker if the only code I
> can come up with to illustrate the problem requires a lame import of a
> third party module?
I doubt this is an issue with Python. File a bug on the h5py tracker and see
what they say. The people there might at least have some vague inkling of what
may be going on.
More information about the Python-list