freebsd and multiprocessing

[Tim Arnold]

On Mar 2, 12:59 pm, Tim Arnold <a_j... at bellsouth.net> wrote:
> On Mar 2, 11:52 am, Philip Semanchuk <phi... at semanchuk.com> wrote:
> > On Mar 2, 2010, at 11:31 AM, Tim Arnold wrote:
>
> > > Hi,
> > > I'm intending to use multiprocessing on a freebsd machine (6.3
> > > release, quad core, 8cpus, amd64). I see in the doc that on this
> > > platform I can't use synchronize:
>
> > > ImportError: This platform lacks a functioning sem_open
> > > implementation, therefore, the required synchronization primitives
> > > needed will not function, see issue 3770.
>
> > > As far as I can tell, I have no need to synchronize the processes--I
> > > have several processes run separately and I need to know when they're
> > > all finished; there's no communication between them and each owns its
> > > own log file for output.
>
> > > Is anyone using multiprocessing on FreeBSD and run into any other
> > > gotchas?
>
> > Hi Tim,
> > I don't use multiprocessing but I've written two low-level IPC  
> > packages, one for SysV IPC and the other for POSIX IPC.
>
> > I think that multiprocessing prefers POSIX IPC (which is where  
> > sem_open() comes from). I don't know what it uses if that's not  
> > available, but SysV IPC seems a likely alternative. I must emphasize,  
> > however, that that's a guess on my part.
>
> > FreeBSD didn't have POSIX IPC support until 7.0, and that was sort of  
> > broken until 7.2. As it happens, I was testing my POSIX IPC code  
> > against 7.2 last night and it works just fine.
>
> > SysV IPC works under FreeBSD 6 (and perhaps earlier versions; 6 is the  
> > oldest I've tested). ISTR that by default each message queue is  
> > limited to 2048 bytes in total size. 'sysctl kern.ipc' can probably  
> > tell you that and may even let you change it. Other than that I can't  
> > think of any SysV limitations that might bite you.
>
> > HTH
> > Philip
>
> Hi Philip,
> Thanks for that information. I wish I could upgrade the machine to
> 7.2! alas, out of my power.  I get the following results from sysctl:
> % sysctl kern.ipc | grep msg
> kern.ipc.msgseg: 2048
> kern.ipc.msgssz: 8
> kern.ipc.msgtql: 40
> kern.ipc.msgmnb: 2048
> kern.ipc.msgmni: 40
> kern.ipc.msgmax: 16384
>
> I'll write some test programs using multiprocessing and see how they
> go before committing to rewrite my current code. I've also been
> looking at 'parallel python' although it may have the same issues.http://www.parallelpython.com/
>
> thanks again,
> --Tim

Well that didn't work out well. I can't import either Queue or Pool
from multiprocessing, so I'm back to the drawing board. I'll see now
how parallel python does on freebsd.

--Tim