2013/8/30 Charles-François Natali

Hello,

This has been bothering me for years: why don't we properly handle EINTR, by running registered signal handlers and restarting the interrupted syscall (or eventually returning early e.g. for sleep)?

EINTR is really a nuisance, and exposing it to Python code is just pointless.

I agree. Is there a way to see in C code where EINTR is not handled? Or a method to handle this systematically?

Now some people might argue that some code relies on EINTR to interrupt a syscall on purpose, but I don't really buy it: it's highly non-portable (depends on the syscall, SA_RESTART flag...) and subject to race conditions (it it comes before the syscall or if you get a partial read/write you'll deadlock).

Furthermore, the stdlib code base is not consistent: some code paths handle EINTR, e.g. subprocess, multiprocessing, sock_sendall() does but not sock_send()... Just grep for EINTR and InterruptedError and you'll be amazed.

GHC, the JVM and probably other platforms handle EINTR, maybe it's time for us too?

Just for reference, here are some issues due to EINTR popping up: http://bugs.python.org/issue17097 http://bugs.python.org/issue12268 http://bugs.python.org/issue9867 http://bugs.python.org/issue7978 http://bugs.python.org/issue12493 http://bugs.pythoto see n.org/issue3771 http://bugs.python.org/issue3771

cf _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/amauryfa%40gmail.com

-- Amaury Forgeot d'Arc

2013/8/30 Amaury Forgeot d'Arc :

I agree. Is there a way to see in C code where EINTR is not handled?

EINTR can be returned on slow syscalls, so a good heuristic would be to start with code that releases the GIL. But I don't see a generic way apart from grepping for syscalls that are documented to return EINTR.

Or a method to handle this systematically?

The glibc defines this macro: # define TEMP_FAILURE_RETRY(expression) \ (__extension__ \ ({ long int __result; \ do __result = (long int) (expression); \ while (__result == -1L && errno == EINTR); \ __result; })) #endif which you can then use as: pid = TEMP_FAILURE_RETRY(waitpid(pid, &status, options)); Unfortunately, it's not as easy for us, since we must release the GIL around the syscall, try again if it failed with EINTR, only after having called PyErr_CheckSignals() to run signal handlers. e.g. waitpid(): """ Py_BEGIN_ALLOW_THREADS pid = waitpid(pid, &status, options); Py_END_ALLOW_THREADS """ should become (conceptually): """ begin_handle_eintr: Py_BEGIN_ALLOW_THREADS pid = waitpid(pid, &status, options); Py_END_ALLOW_THREADS if (pid < 0 && errno == EINTR) { if (PyErr_CheckSignals()) return NULL; goto begin_handle_eintr; } """ We might want to go for a clever macro (like BEGIN_SELECT_LOOP in socketmodule.c). 2013/8/30 Nick Coghlan :

Sounds good to me. I don't believe there's been a conscious decision that we *shouldn't* handle it, it just hasn't annoyed anyone enough for them to propose a systematic fix in CPython. If that latter part is no longer true, great ;)

Great, I'll open a bug report then :) cf

I don't have a strong opinion on this either. The distinction between send() and send_all() makes sense to me though (send_all() works hard to get all your data out, send() only does what it can quickly). Personally for calls like select() I think returning early on EINTR makes sense, it's usually part of a select loop anyway. The only thing I care deeply about is that you shouldn't restart anything before letting a Python-level signal handler run. A program might well have a Python signal handler that must run before the I/O is restarted, and the signal handler might raise an exception (like the default SIGINT handler, which raises KeyboardInterrupt). On Fri, Aug 30, 2013 at 10:02 AM, Antoine Pitrou wrote:

On Fri, 30 Aug 2013 12:29:12 +0200 Charles-François Natali wrote:

...

Furthermore, the stdlib code base is not consistent: some code paths handle EINTR, e.g. subprocess, multiprocessing, sock_sendall() does but not sock_send()... Just grep for EINTR and InterruptedError and you'll be amazed.

GHC, the JVM and probably other platforms handle EINTR, maybe it's time for us too?

I don't have any precise opinion on this. It's true that we should have a systematic approach, I just don't know if all interfaces should handler EINTR automatically, or only the high-level ones. (for the sake of clarity, I'm fine with either :-))

Regards

Antoine.

_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org

-- --Guido van Rossum (python.org/~guido)