On Feb 6, 2006, at 5:52 AM, Henrik Thostrup Jensen wrote:

Hi

I'm trying to use web2 to stream uploading of data, but I ran into some problems. First I tried building from the example in  http://twistedmatrix.com/projects/web2/documentation/examples/demo.html, however args and files objects in the request did not show anything.  However the length of stream was correct, and using tcpdump, showed the correct value. So I tried building my own uploader which look like this:

What are you actually trying to do? There's two different kinds of "streaming upload". The simpler is just a raw data stream. If you just want raw data, forget POST and associated form processing, and just use request.stream.read() in with a PUT method. That will work with curl -T. 

However, if you want it to be the result of a form post from a web browser, you'll need the MIME multipart support. The default POST support (with PostableResource) parses the multipart stream, and places all the results in memory or temporary files (depending on size), in the args & files dicts. It parses the data incrementally, but doesn't give control to your "render" method until all the data has been uploaded. If all you're looking for from a streaming upload is to not buffer everything in memory, this will do it for you.

If that's not good enough for your application, if you really need access to the data as it's being uploaded, you'll need to use the lower level machinery in fileupload: the MultipartMimeStream class. You give this a raw data stream (request.stream), and the multipart boundary from the content-type, then call its read() method repeatedly to get back tuples of (fieldname, filename, ctype, dataStream). Then you use dataStream.read() to get back data blocks and can do whatever you like with them.

I can help explain any of the alternatives in more detail if you let me know which you're really interested in.

James