[Python-Dev] Help preventing SIGPIPE/SIG_DFL anti-pattern.

[Alfred Perlstein]

(sorry for the double post, looks like maybe attachments are dropped, 
inlined the attachment this time.)

Hello,

I'm looking for someone in the python community to help with a problem 
of anti-patterns showing up dealing with SIGPIPE.

Specifically I've noticed an anti-pattern developing where folks will 
try to suppress broken pipe errors written to stdout by setting 
SIGPIPE's disposition to SIG_DFL.  This is actually very common, and 
also rather broken due to the fact that for all but the most simple text 
filters this opens up a problem where the process can exit unexpectedly 
due to SIGPIPE being generated from a remote connection the program makes.

I have attached a test program which shows the problem.

to use this program it takes several args.

# 1. Illustrate the 'ugly output to stderr' that folks want to avoid:

% python3 t0.py nocatch | head -1


# 2. Illustrate the anti-pattern, the program exits on about line 47 
which most folks to not understand

% python3 t0.py dfl | head -1

# 3. Show a better solution where we catch the pipe error and cleanup to 
avoid the message:

% python3 t0.py | head -1


I did a recent audit of a few code bases and saw this pattern pop often 
often enough that I am asking if there's a way we can discourage the use 
of "signal(SIGPIPE, SIG_DFL)" unless the user really understands what 
they are doing.

I do have a pull req here: https://github.com/python/cpython/pull/6773 
where I am trying to document this on the signal page, but I can't sort 
out how to land this doc change.

thank you,

-Alfred


=== CUT HERE ===

#
# Program showing the dangers of setting the SIG_PIPE handler to the 
default handler (SIG_DFL).
#
# To illustrate the problem run with:
# ./foo.py dfl
#
# The program will exit in do_network_stuff() even though there is a an 
"except" clause.
# The do_network_stuff() simulates a remote connection that closes 
before it can be written to
# which happens often enough to be a hazard in practice.
#
#
#

import signal
import sys
import socket
import os

def sigpipe_handler(sig, frame):
     sys.stderr.write("Got sigpipe \n\n\n")
     sys.stderr.flush()

def get_server_connection():
     # simulate making a connection to a remote service that closes the 
connection
     # before we can write to it.  (In practice a host rebooting, or 
otherwise exiting while we are
     # trying to interact with it will be the true source of such behavior.)
     s1, s2 = socket.socketpair()
     s2.close()
     return s1


def do_network_stuff():
     # simulate interacting with a remote service that closes its connection
     # before we can write to it.  Example: connecting to an http 
service and
     # issuing a GET request, but the remote server is shutting down between
     # when our connection finishes the 3-way handshake and when we are able
     # to write our "GET /" request to it.
     # In theory this function should be resilient to this, however if 
SIGPIPE is set
     # to SIGDFL then this code will cause termination of the program.
     if 'dfl' in sys.argv[1:]:
         signal.signal(signal.SIGPIPE, signal.SIG_DFL)

     for x in range(5):
         server_conn = get_server_connection()
         sys.stderr.write("about to write to server socket...\n")
         try:
             server_conn.send(b"GET /")
         except BrokenPipeError as bpe:
             sys.stderr.write("caught broken pipe on talking to server, 
retrying...")

def work():
     do_network_stuff()
     for x in range(10000):
         print("y")
     sys.stdout.flush()


def main():
     if 'nocatch' in sys.argv[1:]:
         work()
     else:
         try:
             work()
         except BrokenPipeError as bpe:
             signal.signal(signal.SIGPIPE, signal.SIG_DFL)
             os.kill(os.getpid(), signal.SIGPIPE)


if __name__ == '__main__':
     main()