[New-bugs-announce] [issue10202] ftplib doesn't check close status after sending file

John Nagle report at bugs.python.org
Tue Oct 26 21:04:50 CEST 2010

New submission from John Nagle <nagle at users.sourceforge.net>:

"ftplib" doesn't check the status on socket close after writing.  This can lead to silently truncated files when sending files with "ftplib".

A report of truncated files on comp.lang.python led me to check the source code. 

The "ftplib" module does sending by calling sock_sendall in "socketmodule.c". That does an OS-level "send", and once everything has been sent, returns.

But an OS-level socket send returns when the data is queued for sending, not when it is delivered.  Only when the socket is closed, and the close status checked, do you know if the data was delivered. There's a final TCP close handshake that occurs when close has been called at both ends, and only when it completes successfully do you know that the data has been delivered.

At the socket level, this is performed by "shutdown" (which closes the connection and returns the proper network status information), or by "close".

Look at sock_close in "socketmodule.c".  Note that it ignores the return status on close, always returns None, and never raises an exception.  As the Linux manual page for "close" says:
"Not checking the return value of close() is a common but nevertheless serious programming error. It is quite possible that errors on a previous write(2) operation are first reported at the final close(). Not checking the return value when closing the file may lead to silent loss of data."

"ftplib", in "storlines" and "storbinary", calls "close" without checking the status or calling "shutdown" first.  So if the other end disconnects after all data has been queued locally but not sent, received and acknowledged, the sender will never know.

components: Library (Lib)
messages: 119638
nosy: nagle
priority: normal
severity: normal
status: open
title: ftplib doesn't check close status after sending file
type: behavior
versions: Python 2.5, Python 2.6, Python 2.7, Python 3.1, Python 3.2, Python 3.3

Python tracker <report at bugs.python.org>

More information about the New-bugs-announce mailing list