[Web-SIG] Reviewing WSGI open issues, again...
Alan Kennedy
py-web-sig at xhaus.com
Thu Sep 9 21:05:06 CEST 2004
[Phillip J. Eby]
> Java doesn't have file descriptors, so
> you're not going to be able to use 'sendfile()' in Java. So, what's
> the point of having 'fd_wrapper' available there?
and
> An object that works with 'fd' isn't going to work with 'nio', or vice
> versa is it? Or am I missing something about how nio works?
>
> I suppose the alternative is to specify 'wsgi.file_wrapper' such that
> it's required to always return *something* usable, even if it can't
> figure out any way to optimize it. Objects passed to 'file_wrapper'
> would have to have a 'read', optionally a 'close', and optionally
> 'fileno'. (A Jython WSGI server would ignore fileno, of course.)
Ah, I think I see where the confusion lies. Perhaps I should have taken
more time to explain a certain issue earlier than this.
Jython files *may* have the local analogue of a file descriptor, i.e. a
channel, but only when the jython code is running on a jvm that supports
java.nio, which means 1.4 or greater.
I could define the fileno method of jython files like this
class file:
def fileno(self):
if hasattr(self.java_file, 'getChannel'):
# java >= 1.4 behaviour
return self.java_file.getChannel()
else:
# java < 1.4 behaviour
raise UnimplementedException()
The current jython 2.1 library only raises the exception, because
java.nio didn't exist when it was written.
Now, I could suggest a patch to the jython runtime to redefine fileno as
above, but that's not a safe thing to do: existing python code that is
expecting a cpython file descriptor will almost certainly break if it
gets passed a java.nio.channels.FileChannel instead.
Unless the entire I/O subsystem has been rewritten, as I am doing for
jynio sockets, which *do* have a useful fileno() method which each of
the new modules knows how to use properly. The returned object confers
identical semantics to cpython file descriptors, when passed to jynio
socket modules. For example, when jynio is finished, this code will run
identically on cpython and jython
s = socket.socket(AF_INET, SOCK_STREAM)
fd = s.fileno()
po = select.poll()
po.register(fd)
http://www.xhaus.com/alan/python/jynio/socket.html#socketvschannel
A very similar set of operations can be carried out on both file
descriptors and channels: i.e. selectability/event notification, bulk
transfers, etc.
http://java.sun.com/j2se/1.4.2/docs/api/java/nio/channels/SelectableChannel.html
http://java.sun.com/j2se/1.4.2/docs/api/java/nio/channels/InterruptibleChannel.html
http://java.sun.com/j2se/1.4.2/docs/api/java/nio/channels/WritableByteChannel.html
http://java.sun.com/j2se/1.4.2/docs/api/java/nio/channels/GatheringByteChannel.html
So, when running on java 1.4+, I *can* get the local equivalent of a
file descriptor, and do meaningful things with it, in terms of bulk
transfer, etc.
But that doesn't work on older JVMs where only java.io is available:
There is no other way (without examining private file object data, i.e.
the java.io.FileInputStream encapsulated in the jython file) that I can
determine if something is file-like, other than to do this
if type(app_object) is types.FileType:
do_high_performance_file_stuff(app_object)
That's why I pushed for permitting return of file-likes from
applications: because it's the only "safe" way to recognise files on
pre-1.4 jvms. And it's also portable to all other python platforms.
It would still definitely be useful to recognise the optimisation on
older VMs, because I could still have a fast native-java
loop-while-sending-blocks implementation of "sendfile", which would be
substantially faster than a jython one, because it would avoid the
unnecessary transformation of the file data into jython data structures
(i.e. binary strings) and then back again.
But your solution of a single server-provided file_wrapper class solves
the problem nicely. Because the application has hinted that the
application object is a file, I now have a simple way of checking, that
works across all jvms. So I can now very simply provide the bulk
transfer optimisation, and implement it differently, depending on the
availability of the java.nio classes, e.g.
try:
import java.nio
class file_wrapper:
def send_file(self, dest)
use_nio_transfer_to(dest)
except ImportError:
import java.io
class file_wrapper:
def send_file(self, dest)
use_looping_sendfile(dest)
Also, the 'file_wrapper' solution alleviates the need for me look at
private data inside jython file objects, to see if the underlying
java.io.FileInputStream has a getChannel method. So it's definitely the
cleanest solution.
As for the 'file_wrapper' class name across platforms, as you can see
from the above, having different class names for each platform would not
change the above considerations one bit: it would just make the
application authors life more difficult.
I hope that makes the situation clearer!
Regards,
Alan.
More information about the Web-SIG
mailing list