[Web-SIG] Use cases for file-like objects (was Re: Bill's
comments on WSGI draft 1.4)
Phillip J. Eby
pje at telecommunity.com
Wed Sep 8 00:55:08 CEST 2004
At 08:12 PM 9/7/04 +0100, Alan Kennedy wrote:
>[Phillip J. Eby]
> > Instead of using 'fileno' as an extension attribute on the iterable,
> > we'll add a 'wsgi.file_wrapper' key, usable as follows by an
> > application:
> > return environ['wsgi.file_wrapper'](something,blksize)
> > The 'file_wrapper' may introspect "something" in order to do a
> > fileno() check, or other "I know how to send this kind of object
> > quickly" optimizations. It must return an iterable, that the
> > application may return back to the server.
>[tony at lownds.com]
> > Here's the tail end of the CGI example.
> > result = application(environ, start_response)
> > try:
> > if hasattr(result, 'read'):
> > result = iter(lambda: result.read(BLOCKSIZE), '')
> > for data in result:
> > write(data)
> > finally:
> > if hasattr(result,'close'):
> > result.close()
>Since I am just about to implement "wsgi.file_wrapper", I just wanted to
>check that my understanding of it is correct.
Before you implement it, I should warn you that I'm thinking 'file_wrapper'
was a bad idea, and that there's a better way to do all this.
As I understand them, the current use cases for file-like objects are:
1. sendfile(fileno()) for fast file-descriptor copying (Unix-like OSes
only, and only single-thread synchronous servers like Apache 1.x or CGI)
2. Convenience in returning an open file or pipe
3. Convenience in returning a StringIO or other "file-like" object
By the way, as far as I know, none of these use cases are especially common
in today's existing web frameworks. Anyway, use cases 2 and 3 can be
grouped into cases where the object is "large", "small", or "pipe-like":
return iter(lambda: filelike.read(SIZE), '')
return iter(filelike.read, '')
These are all very simple, one-line solutions (at least for 2.2+) and have
the advantage of being explicit, and refusing the temptation to guess. The
application is in total control of how the resource will be transmitted.
That leaves only use case 1, which is a fairly limited use case and isn't
even applicable to most web servers written in Python, as most such servers
are asynchronous and can't take advantage of the 'sendfile()' system call
(which Python doesn't expose as an 'os' facility anyway).
Therefore, my current thinking is to relegate use case 1 to a WSGI
extension, 'wsgi.fd_wrapper', which can used like this (if the application
is returning an object with a working 'fileno()' method):
if 'wsgi.fd_wrapper' in environ:
# return a normal iterable
In other words, 'wsgi.fd_wrapper' would be sort of like my earlier
'wsgi.file_wrapper', but it would be *optional* to implement and
use. (Meaning it can be relegated to an application note, instead of
having to be introduced in-line.)
For Alan's attempt to support Jython 2.1, he could write an 'iter' function
or class and put it in __builtin__, so that programs written to this idiom
would still work.
After thinking about the 'file_wrapper' idea some more, I'm thinking that
this way works better for everything but the issue of closing
files. However, my example 'file_wrapper' class should maybe be included
in the PEP under an application note about sending files and file-like objects.
More information about the Web-SIG