[Python-checkins] peps: PEP 446: add an "Overlapped I/O" alternative

victor.stinner python-checkins at python.org
Thu Jul 4 12:58:38 CEST 2013 

http://hg.python.org/peps/rev/20d12c860d89
changeset:   4975:20d12c860d89
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Thu Jul 04 12:58:03 2013 +0200
summary:
  PEP 446: add an "Overlapped I/O" alternative

files:
  pep-0446.txt |  108 +++++++++++++++++++++++++++-----------
  1 files changed, 76 insertions(+), 32 deletions(-)


diff --git a/pep-0446.txt b/pep-0446.txt
--- a/pep-0446.txt
+++ b/pep-0446.txt
@@ -23,23 +23,31 @@
 Inherance of file descriptors
 -----------------------------
 
-The inherance of a file descriptor in a child process at the execution
-of a new program can be configured on each file descriptor using a
-*close-on-exec* flag. By default, the close-on-exec flag is not set. On
-Windows, file descriptors are not inherited if the ``bInheritHandles``
-parameter of the ``CreateProcess()`` function is ``FALSE``, even if the
-close-on-exec flag is not set. On UNIX, file descriptors are inherited
-by default.
+The inherance of file descriptors in child processes can be configured
+on each file descriptor using a *close-on-exec* flag. By default, the
+close-on-exec flag is not set.
+
+On Windows, file descriptors are not inherited if the
+``bInheritHandles`` parameter of the ``CreateProcess()`` function is
+``FALSE``, even if the close-on-exec flag is not set.
+
+On UNIX, file descriptors with the close-and-exec flag set are closed at
+the execution of a new program (ex: when calling ``execv()``). The flag
+has no effect on ``fork()``, all file descriptors are inherited by the
+child process.
+
+Issues of the inherance of file descriptors
+-------------------------------------------
 
 Inherance of file descriptors causes issues. For example, closing a file
 descriptor in the parent process does not release the resource (file,
 socket, ...), because the file descriptor is still open in the child
 process.
 
-Leaking file descriptors is a major security vulnerability. An
-untrusted child process can read sensitive data like passwords and
-take control of the parent process though leaked file descriptors. It
-is for example a known vulnerability to escape from a chroot.
+Leaking file descriptors is also a major security vulnerability. An
+untrusted child process can read sensitive data like passwords and take
+control of the parent process though leaked file descriptors. It is for
+example a known vulnerability to escape from a chroot.
 
 
 Non-blocking sockets
@@ -47,21 +55,24 @@
 
 To handle multiple network clients in a single thread, a multiplexing
 function like ``select()`` can be used. For best performances, sockets
-must be configured as non-blocking.
+must be configured as non-blocking. Operations like ``send()`` and
+``recv()`` return an ``EAGAIN`` or ``EWOULDBLOCK`` error if the
+operation would block.
 
 By default, newly created sockets are blocking. Setting the non-blocking
-mode requires extra system calls.
+mode requires additional system calls.
 
 
 Setting flags at the creation of the file descriptor
 ----------------------------------------------------
 
 Windows and recent versions of other operating systems like Linux
-support setting close-on-exec and blocking flags directly at the
-creation of file descriptors and sockets.
+support setting the close-on-exec flag directly at the creation of file
+descriptors, and close-on-exec and blocking flags at the creation of
+sockets.
 
-Setting these flags at the creation is atomic and avoids extra system
-calls.
+Setting these flags at the creation is atomic and avoids additional
+system calls.
 
 
 Proposal
@@ -95,12 +106,12 @@
 * ``socket.socket.fromfd``
 * ``socket.socketpair()``
 
-The default value of *cloexec* is ``False``, and the default value of
+The default value of *cloexec* is ``False`` and the default value of
 *blocking* is ``True``.
 
 The atomicity is not guaranteed. If the platform does not support
 setting close-on-exec and blocking flags at the creation of the file
-descriptor, the flags are set using extra system calls.
+descriptor or socket, the flags are set using additional system calls.
 
 New Functions
 -------------
@@ -120,21 +131,54 @@
 
 The close-on-exec flag must also be set on private file descriptors and
 sockets in the Python standard library. For example, on UNIX,
-os.urandom() opens ``/dev/urandom`` to read some random bytes, the file
-descriptor is closed at function exit.  The file descriptor is not
-expected to be inherited on execution of a new program in a child
-process.
+os.urandom() opens ``/dev/urandom`` to read some random bytes and the
+file descriptor is closed at function exit. The file descriptor is not
+expected to be inherited by child processes.
 
 
-Alternatives
-============
+Rejected Alternatives
+=====================
 
-The PEP 433 is a previous attempt proposing various other alternatives,
-but no consensus could be reached.
+PEP 433
+-------
 
-This PEP is much simpler, more conservative (no backward compatibility
-issue) and has a well defined behaviour (the default value of the new
-*cloexec* parameter is not configurable).
+The PEP 433 entitled "Easier suppression of file descriptor inheritance"
+is a previous attempt proposing various other alternatives, but no
+consensus could be reached.
+
+This PEP has a well defined behaviour (the default value of the new
+*cloexec* parameter is not configurable), is more conservative (no
+backward compatibility issue), and is much simpler.
+
+
+Add blocking parameter for file descriptors and Windows overlapped I/O
+----------------------------------------------------------------------
+
+Windows supports non-blocking operations on files using an extension of
+the Windows API called "Overlapped I/O". Using this extension requires
+to modify the Python standard library and applications to pass a
+``OVERLAPPED`` structure and an event loop to wait for the completion of
+operations.
+
+This PEP only tries to expose portable flags on file descriptors and
+sockets. Supporting overlapped I/O requires an abstraction providing a
+high-level and portable API for asynchronous operations on files and
+sockets. Overlapped I/O are out of the scope of this PEP.
+
+UNIX supports non-blocking files, moreover recent versions of operating
+systems support setting the non-blocking flag at the creation of a file
+descriptor. It would be possible to add a new optional *blocking*
+parameter to Python functions creating file descriptors. On Windows,
+creating a file descriptor with ``blocking=False``  would raise a
+``NotImplementedError``. This behaviour is not acceptable for the ``os``
+module which is designed as a thin wrapper on the C functions of the
+operating system. If a platform does not support a function, the
+function should not be available on the platform. For example,
+the ``os.fork()`` function is not available on Windows.
+
+For all these reasons, this alternative was rejected. The PEP 3156
+proposes an abstraction for asynchronous I/O supporting non-blocking
+files on Windows.
 
 
 Links
@@ -155,8 +199,8 @@
 * `#16946: subprocess: _close_open_fd_range_safe() does not set
   close-on-exec flag on Linux < 2.6.23 if O_CLOEXEC is defined
   <http://bugs.python.org/issue16946>`_
-* `#17070: PEP 433: Use the new cloexec to improve security and avoid
-  bugs <http://bugs.python.org/issue17070>`_
+* `#17070: Use the new cloexec to improve security and avoid bugs
+  <http://bugs.python.org/issue17070>`_
 
 Other links:
 

-- 
Repository URL: http://hg.python.org/peps

More information about the Python-checkins mailing list