[pypy-dev] Dead loop occurs when using python-daemon and multiprocessing together in PyPy 4.0.1

Maciej Fijalkowski fijall at gmail.com
Wed Dec 23 09:48:41 EST 2015 

I bet the difference is due to "we see more FDs in pypy". If you close the
correct FD in CPython, it would break too I presume? Or is there special
code to handle that?

On Wed, Dec 23, 2015 at 3:57 PM, Carl Friedrich Bolz <cfbolz at gmx.de> wrote:

> Hi Maciek,
>
> Of course it's a difference between pypy and cpython. if you close all
> fds, cpython's random module still works, but PyPy's doesn't.
>
> Cheers,
>
> Carl Friedrich
>
>
> On December 23, 2015 2:54:23 PM GMT+01:00, Maciej Fijalkowski <
> fijall at gmail.com> wrote:
>>
>> well, ok, but that does not sound like a pypy bug then - "close all
>> existing fds and complain that some of them are closed" is a bit not good -
>> maybe it's a bug in python-daemon and the PEP?
>>
>> On Wed, Dec 23, 2015 at 3:36 PM, hubo <hubo at jiedaibao.com> wrote:
>>
>>> No, the python-daemon module is critical in this problem, because it is
>>> the python-daemon module who closed the fd to /dev/urandom. When process
>>> swith to daemon, it forks itself, and then close all open fds (including
>>> stdin, stdout and stderr), so it also closes the fd for /dev/urandom which
>>> is used by PyPy library. It is the standard behavior defined by
>>> https://www.python.org/dev/peps/pep-3143/#daemoncontext-objects and
>>> also the standard behavior for unix daemons. And unfortunately there is not
>>> a way to prevent the fd to be closed without knowing exactly what number it
>>> is on.
>>>
>>> Without python-daemon (or similar libraries), it is only possible to
>>> reproduce the problem by closing the fd (usually 4) forcely, but it does
>>> not make much sense.
>>>
>>>
>>> 2015-12-23
>>> ------------------------------
>>> hubo
>>> ------------------------------
>>>
>>> *发件人：*Maciej Fijalkowski <fijall at gmail.com>
>>> *发送时间：*2015-12-23 21:22
>>> *主题：*Re: Re: Re: [pypy-dev] Dead loop occurs when using python-daemon
>>> and multiprocessing together in PyPy 4.0.1
>>> *收件人：*"hubo"<hubo at jiedaibao.com>
>>> *抄送：*"pypy-dev"<pypy-dev at python.org>
>>>
>>> can you reproduce the OSError problem without having the daemon module
>>> involved either?
>>>
>>> On Wed, Dec 23, 2015 at 3:14 PM, hubo <hubo at jiedaibao.com> wrote:
>>>
>>>> I can only reproduce the *OSError* problem. Maybe the CPU 100% is not
>>>> really a dead lock, but rather some kind of automatic crash report?
>>>> Although it is quite easy to crash the program with os.urandom, it
>>>> only stops responding when the crash happens in system libraries like
>>>> multiprocessing or email.
>>>>
>>>> The posix.urandom problem is quite easy to reproduce:
>>>>
>>>> #!/usr/bin/pypy
>>>> import os
>>>> os.urandom(16)
>>>> def test():
>>>>     print repr(os.urandom(16))
>>>> import daemon
>>>> import sys
>>>> if __name__ == '__main__':
>>>>     with daemon.DaemonContext(initgroups=False,
>>>> stderr=sys.stderr,stdout=sys.stdout):
>>>>         test()
>>>>
>>>> (stderr and stdout is kept open to show console messages in the daemon.
>>>> initgroups=False is a workaround on python-daemon not working in Python2.6)
>>>>
>>>> Or, with module random:
>>>>
>>>> #!/usr/bin/pypy
>>>> import random
>>>> def test():
>>>>     random.Random()
>>>> import daemon
>>>> import sys
>>>> if __name__ == '__main__':
>>>>     with daemon.DaemonContext(initgroups=False,
>>>> stderr=sys.stderr,stdout=sys.stdout):
>>>>         test()
>>>> And when run scripts with pypy:
>>>>
>>>> pypy test3.py
>>>>
>>>> it crashes with OSError:
>>>> Traceback (most recent call last):
>>>>   File "test2.py", line 13, in <module>
>>>>     test()
>>>>   File "test2.py", line 6, in test
>>>>     random.Random()
>>>>   File
>>>> "/opt/pypy-4.0.1-linux_x86_64-portable/lib-python/2.7/random.py", line 95,
>>>> in __init__
>>>>     self.seed(x)
>>>>   File
>>>> "/opt/pypy-4.0.1-linux_x86_64-portable/lib-python/2.7/random.py", line 111,
>>>> in seed
>>>>     a = long(_hexlify(_urandom(2500)), 16)
>>>> OSError: [Errno 9] Bad file descriptor
>>>>
>>>> It is still not clear why it causes dead loop (or long-time no
>>>> responding) in multiprocessing (should have thrown an ImportError) and the
>>>> exact condition for the file descriptor of /dev/urandom appears (just call
>>>> os.urandom and import random does not reproduce the result), but I believe
>>>> it is definitely linked to the problem.
>>>>
>>>> 2015-12-23
>>>> ------------------------------
>>>> hubo
>>>> ------------------------------
>>>>
>>>> *发件人：*Maciej Fijalkowski <fijall at gmail.com>
>>>> *发送时间：*2015-12-23 20:07
>>>> *主题：*Re: Re: [pypy-dev] Dead loop occurs when using python-daemon and
>>>> multiprocessing together in PyPy 4.0.1
>>>> *收件人：*"hubo"<hubo at jiedaibao.com>
>>>> *抄送：*"pypy-dev"<pypy-dev at python.org>
>>>>
>>>> That's very interesting, can you produce a standalone example that does
>>>> not use multiprocessing? That would make it much easier to fix the bug
>>>> (e.g. os.fork followed by os.urandom failing)
>>>>
>>>> On Wed, Dec 23, 2015 at 1:54 PM, hubo <hubo at jiedaibao.com> wrote:
>>>>
>>>>> Thanks for the response. Should I put it directly in the bug tracker?
>>>>>
>>>>> FYI, I've located the reason to be the incompatibility with
>>>>> python-daemon (or rather the standard unix-daemon behavior) and PyPy
>>>>> *posix.urandom* implementation.
>>>>>
>>>>> It seems that in PyPy 4.0.1, when module *random* loaded, a file
>>>>> descriptor is created on /dev/urandom. I think PyPy implementation use the
>>>>> shared descriptor to read from /dev/urandom. Sadly when python-daemon fork
>>>>> the process and turns it into an unix daemon, it closes all the currently
>>>>> open file descriptors. After that all os.urandom calls failed with OSError.
>>>>> I think maybe the other functions of Random class is also using the file
>>>>> descriptor in C code and just never detects if the return value is 0, and
>>>>> causes the dead loop.
>>>>>
>>>>> I think the problem will be solved if the implementation re-open the
>>>>> handle when it is closed somehow.
>>>>>
>>>>> multiprocessing is using random internally. Also there are lots of
>>>>> other modules using random, like email etc. The dead loop occurs when you
>>>>> use any of the libraries in a daemon.
>>>>>
>>>>>
>>>>>
>>>>> 2015-12-23
>>>>> ------------------------------
>>>>> hubo
>>>>> ------------------------------
>>>>>
>>>>> *发件人：*Maciej Fijalkowski <fijall at gmail.com>
>>>>> *发送时间：*2015-12-23 19:35
>>>>> *主题：*Re: [pypy-dev] Dead loop occurs when using python-daemon and
>>>>> multiprocessing together in PyPy 4.0.1
>>>>> *收件人：*"hubo"<hubo at jiedaibao.com>
>>>>> *抄送：*"pypy-dev"<pypy-dev at python.org>
>>>>>
>>>>> Hi hubo
>>>>>
>>>>> Can you put it as a bug report? Those things get easily lost on the
>>>>> mailing list (and sadly I won't look at it right now, multiprocessing
>>>>> scares me)
>>>>>
>>>>> On Wed, Dec 23, 2015 at 12:03 PM, hubo <hubo at jiedaibao.com> wrote:
>>>>>
>>>>>> Hello devs,
>>>>>>
>>>>>> A (possible) dead loop is found when I use python-daemon and
>>>>>> multiprocessing together in PyPy 4.0.1, which does not appear in Python(2.6
>>>>>> or 2.7). Also it does not appear in earlier PyPy versions (2.0.2)
>>>>>>
>>>>>> *Reproduce*:
>>>>>>
>>>>>> First install python-daemon:
>>>>>> pypy_pip install python-daemon
>>>>>>
>>>>>> Use the following test script (also available in attachment):
>>>>>>
>>>>>> #!/usr/bin/pypy
>>>>>> import daemon
>>>>>> import multiprocessing
>>>>>> def test():
>>>>>>     q = multiprocessing.Queue(64)
>>>>>> if __name__ == '__main__':
>>>>>>     with daemon.DaemonContext():
>>>>>>         test()
>>>>>>
>>>>>> When executing the script with pypy:
>>>>>> pypy test.py
>>>>>>
>>>>>> The background service does not exit, and is consuming 100% CPU:
>>>>>> ps aux | grep pypy
>>>>>> root      7769 99.1  0.5 235332 46812 ?        R    17:52   2:09 pypy
>>>>>> test.py
>>>>>> root      7775  0.0  0.0 103252   804 pts/1    S+   17:54   0:00 grep
>>>>>> pypy
>>>>>>
>>>>>>
>>>>>>
>>>>>> Executing the script with python:
>>>>>> python2.7 test.py
>>>>>> And the background service normally exits.
>>>>>>
>>>>>> *Environment:*
>>>>>> I'm using CentOS 6.5, with portable PyPy distribution for linux (
>>>>>> https://bitbucket.org/squeaky/portable-pypy/downloads/pypy-4.0.1-linux_x86_64-portable.tar.bz2
>>>>>> )
>>>>>> I run the script on system built-in python (python 2.6.6), a compiled
>>>>>> CPython (2.7.11), and pypy from epel-release(pypy 2.0.2, python 2.7.2), and
>>>>>> the problem does not appear. Though the compiled CPython is 2.7.11 and PyPy
>>>>>> 4.0.4 is python 2.7.10, I think that does not matter much.
>>>>>>
>>>>>> Please contact if you have any questions or ideas.
>>>>>>
>>>>>>
>>>>>> 2015-12-23
>>>>>> ------------------------------
>>>>>> hubo
>>>>>>
>>>>>> _______________________________________________
>>>>>> pypy-dev mailing list
>>>>>> pypy-dev at python.org
>>>>>> https://mail.python.org/mailman/listinfo/pypy-dev
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>> ------------------------------
>>
>> pypy-dev mailing list
>> pypy-dev at python.org
>> https://mail.python.org/mailman/listinfo/pypy-dev
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20151223/ccba429c/attachment-0001.html>

More information about the pypy-dev mailing list