Hey
I have a script to be able to upload logs to a twisted web server.
I have the following requirements:
1) to be able to limit the size of the upload to 10MB. 2) to NOT store file in memory during the request.
Does twisted.web provide these functionality?
Any inputs on these would be welcome!
Here is the *server part* os the script:
try: oStream = open(filename, 'wb') oStream.write(request.args['fname'][0]) outputStream.close()
except: # handle exception
a test *client script:*
<!DOCTYPE html> <html> <form action="https://serverUrl" enctype="multipart/form-data" method="post"> <div class="row"> <label for="fileUploadToServer">Select file</label><br /> fname : <input type="file" name="fname" /> <input type="submit" value="submit"> </div> </form> </html>
Pranav
I believe, assuming that this hasn't been worked on last time I looked at the code, that this is not currently possible. The code is about half way there.
The request processing function stores the request on the disk if it is over a certain size. However, once the request has been uploaded, a function from Python's cgi module is called to parse the data into request.params. Not only does this lose some information about file uploads (which can only be recovered by re-parsing the request), but it also loads all data into memory.
There is also not currently a way to limit either the size of the request, or the size of individual files in the request.
Assuming that you need a solution and don't want to work on implementing this in Twisted, I would suggest using nginx as a reverse proxy in front of Twisted, with the "Upload" module that can be found on the Addons page on nginx's wiki. This module will place any uploaded files into files on the disk, and include regular request parameters to tell your Twisted server where these files are. It is also capable of putting limits in place on the size of uploaded files.
Cameron
On 17 July 2013 01:58, Pranav Bhardwaj pranavb85@gmail.com wrote:
Hey
I have a script to be able to upload logs to a twisted web server.
I have the following requirements:
- to be able to limit the size of the upload to 10MB.
- to NOT store file in memory during the request.
Does twisted.web provide these functionality?
Any inputs on these would be welcome!
Here is the *server part* os the script:
try: oStream = open(filename, 'wb') oStream.write(request.args['fname'][0]) outputStream.close() except: # handle exceptiona test *client script:*
<!DOCTYPE html>
<html> <form action="https://serverUrl" enctype="multipart/form-data" method="post"> <div class="row"> <label for="fileUploadToServer">Select file</label><br /> fname : <input type="file" name="fname" /> <input type="submit" value="submit"> </div> </form> </html>
Twisted-web mailing list Twisted-web@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web
On Jul 16, 2013, at 6:55 PM, Shell cam.turn@gmail.com wrote:
I believe, assuming that this hasn't been worked on last time I looked at the code, that this is not currently possible. The code is about half way there.
The request processing function stores the request on the disk if it is over a certain size. However, once the request has been uploaded, a function from Python's cgi module is called to parse the data into request.params. Not only does this lose some information about file uploads (which can only be recovered by re-parsing the request), but it also loads all data into memory.
The trick to doing this with present twisted.web is to override the 'handleContentChunk' method of Request, rather than trying to catch the streaming-upload behavior in your Resource class. Of course, the right thing to do is to help get https://tm.tl/#288 fixed :).
-glyph
-
Glyph -
Pranav Bhardwaj -
Shell