[Web-SIG] Returned application object and fileno.

[Alan Kennedy]

Phillip,

I'm fairly sure I understand your position now. But I think I don't 
agree with it ;-)

[Phillip J. Eby]
 > "File-like" is a complete red herring: the spec has never supported them
 > (and IMO never will).
 >
 > What the spec calls for is an *iterable*: an object that can be used in
 > a "for" loop.  In Python 2.2 and up, file objects are iterable.  In
 > older versions of Python, they are not.
 >
 > Thus, an application that returns a file object implicitly requires
 > Python 2.2 or up.  (This issue is mentioned in the spec, where it warns
 > that if you are using an older version of Python, you may *not* return a
 > file object.)
 >
 > WSGI does not support returning files or file-like objects: it is simply
 > an artifact of Python 2.2 and up that returning a file works at all!

My position here is that the iterator-ness of a returned file object is 
secondary when the returned object has a fileno() method: most cpython 
framework code is going to do this

if (hasattr(app_object, 'fileno') and callable(app_object.fileno):
   send_file(app_object)
else:
   treat_app_object_as_iterable(app_object)

I would summarise the position of the current spec as "you must return 
an iterable, except when you want to return a file object, which will 
work fine under cpython 2.2+, because files are iterable under cpython 
2.2+, even though they don't need to be iterable when they have fileno()".

 > The fact that you would like such code to run in a Jython 2.1 server
 > doesn't mean that the spec should expand its scope to cover even *file
 > objects*, let alone "file-like" objects.  It simply means that you'll
 > have to deal with the special cases that entails, until Jython 2.2 is
 > ready for prime-time.

Call me old-fashioned, but I'm a great believer in "practicality beats 
purity". I think we should be seeking to be as inclusive as possible, 
which means supporting as wide a software base as possible.

I'm just afraid that people will steam ahead writing WSGI middleware 
applications which return file-objects, which will fail on jython simply 
because putting the following lines in my code is a violation of the spec

if type(app_return) is types.FileType:
   do_file_stuff(app_return)

[Alan Kennedy]
 >> 2. The spec, as is, explicitly permits authors of cpython applications
 >> to return file-like objects,

[Phillip J. Eby]
 > Only if they are *iterable*, which is only true of the 'file' object in
 > 2.2 and up.

Which seems to me an arbitrary criterion, especially in the light that 
the iterator nature of the file object will possibly (likely) not be 
actually used, as described in the snippet above.

 > You're effectively arguing for removing the 'fileno()' special case
 > altogether, or else adding language to require the server to *first*
 > check for iterability and raise an error if the return isn't iterable,
 > so that running a 2.2 app in a 2.1 server won't "accidentally" succeed
 > when the 2.1 server supports 'fileno()'.

Not at all.

I'm arguing for us to be practical about applications returning file 
objects.

1. It's a very common use case
2. It's trivial to deal with
3. There are no python version dependency issues

In cpython frameworks, the code would look like this

if hasattr(app_object, 'fileno'):
   do_file_stuff(app_object.fileno())
else:
   do_iterator_stuff(app_object)

On jython

if type(app_object) is types.FileType:
   do_file_stuff(app_object)
else:
   do_iterator_stuff(app_object)

Is that so difficult to accept?

[Phillip J. Eby]
 > I applaud your bravery in trying to make it work for Jython, but
 > changing the spec to allow other kinds of objects isn't going to
 > decrease the amount of work you have to do, only increase it for other
 > people who *aren't* trying to support 2.2 apps in a server running under
 > Python 2.1.

It's not really about bravery, it's about wanting to maximize 
portability between available python platforms. I hope to achieve that 
through the application of a little pythonic simplicity.

After all, we're just trying to move byte streams from one place to 
another: do we have to be this complex about it?

 > It's not about "file-like" objects, only *actual* file objects.
 > Returning a "file-like" object offers no meaningful performance boost,
 > and it is *not* supported -- and never was.

Except when it is supported, for whatever complicated reasons, e.g. 
iterable objects with fileno()s.

[Alan Kennedy]
 >> I think we should explicitly allow return of a file-like object, and
 >> thus freedom to use the read() method, etc.

[Phillip J. Eby]
 > You are trying to let 2.2 code run in a 2.1 Python.

Well, I see it as WSGI forcing me to jump through hoops in order to 
support the notion of iterability, even when that notion is NOT 
universally applicable, as the fileno() exception proves.

 > That is why this is a Python versioning issue, and specific to your
 > attempt to run 2.2 code in a 2.1 Python.  It has absolutely nothing to
 > do with accepting "file-like" objects in the spec, which never accepted
 > them, nor is it intended to ever do so.
 >
 > Is this getting any clearer?

Crystal.

However, I think the absolute insistence on return objects being 
iterable is slightly arbitrary and unnecessarily constraining.

I understand your desire to keep the spec clean and simple, and also 
your desire to use modern python facilities to do it. But those modern 
python facilities are not universally available, and, strictly speaking, 
not absolutely required. I'm suppose I'm just pleading for a little 
pythonic practicality.

Maybe I'm just wasting my time? Maybe I'm the only one who is interested 
in seeing a jython WSGI server into which users can drop universal WSGI 
components and have them just work? Is anyone else interested in such a 
jython WSGI container? Or should I just toddle off back to J2EE servlets?

Lastly, since the spec is still potentially a moving target, I've 
translated as much of my java as possible into jython, which will 
greatly speed up the prototyping process. Once the spec is finalized, I 
may translate it back to java, if there is a sufficient 
performance/other requirement for that. (I should have prototyped it in 
jython from the start, and saved myself a load of time).

Kind regards,

Alan.