[issue6333] logging: ValueError: I/O operation on closed file

Vinay Sajip report at bugs.python.org
Fri Jul 10 01:43:47 CEST 2009

Vinay Sajip <vinay_sajip at yahoo.co.uk> added the comment:

> However, sys.stdout|err can be assigned to some thing else
> (eg: py.test assigning it to an open file object) .. in which case it 
> is legitimate to handle (close) such objects (handles) by who
> created it (py.test).

@Sridhar: I disagree. Creation is not ownership. Clearly ownership is
*initially* with the creator, but when a stream is used to initialise a
StreamHandler, ownership of the stream passes to the StreamHandler.

@Holger: I am not asking that py.test be specially modified to cater for
use of a logging module it knows nothing about. But your example showed
logging calls; I indicated what changes were needed to that sample to
interoperate with logging. So if that was a part of the test cases, then
the design of test cases which use logging (and not py.test itself, if
it doesn't invoke logging directly) needs to consider the interaction
between each test case, logging and py.test. And it is not solely the
responsibility of py.test to minimise resource usage during test runs;
part of this is also the responsibility of the test case design, which
surely is why test frameworks have the concepts of setUp and tearDown.

Your example calls logging.basicConfig passing a stream, and the stream
is then immediately closed. If the test case had involved threads which
did logging, for example, how could you guarantee that a call on the
stream (other than flush) would never happen after the call to close the
stream? Would I then have to ensure not only that flush was defensively
programmed, but write as well? It seems like a can of worms to me,
because of the propensity of this kind of approach to mask logic errors.
I'm not saying you never have to resort to defensive programming - just
that errors shouldn't pass silently where this is avoidable. The design
of StreamHandler is predicated on it owning the stream (for example, it
takes care to serialise access to the stream with locks, to allow
thread-safe operation). This would be pointless if other code could
arbitrarily operate on the stream without worrying about thread-safety.

I also pointed out numerous flaws in Sridhar's reasoning both in this
issue and in issue6435 (not py.test related) and in my view, he needs to
look more closely at the logging package and the included functionality,
and if necessary the core's regression test module, test_logging.py, to
see what sorts of issues crop up. For example, I pointed out in msg90148
that file-like objects are not guaranteed to have a "closed" attribute,
to which he has not responded. I would appeal for more attention to detail.

And just in case it may seem otherwise, I'll state explicitly that I am
not trying to be difficult. Similar issues have arisen during the
initial development of test_logging.py. 

I will update the documentation for StreamHandler, FileHandler etc. to
highlight the issue of ownership of the stream, as perhaps it's not
clear enough. Sorry if this post seems a little long-winded.


Python tracker <report at bugs.python.org>

More information about the Python-bugs-list mailing list