On Sun, 07 Jan 2007 17:37:44 +0000, Ross Burton ross@burtonini.com wrote:

Hi,

I'm writing a Flickr upload tool[1] and would like to be able to show the progress of the image upload. Images are uploaded via HTTP POST:

from twisted.web import client ... return client.getPage("http://api.flickr.com/services/upload/", method="POST", headers=headers, postdata=form)

where form is the encoded title/tags/image data. Is there a way to get progress information from twisted.web.client? If I'm uploading 4M images the upload takes some time to complete, so I'd like to be able to have a progress bar.

Take a look at twisted.web.client.HTTPPageGetter's implementation. Note the last thing it does in connectionMade:

if data is not None: self.transport.write(data)

If you want progress reporting, then this is the code to fix in order to get it. Rather than writing all of the data at once, this code could use a producer to write it approximately only as quickly as the peer reads it. This will let you generate a progress report, since you can keep track of how many bytes you have written and how many bytes are yet to be written.

Due to the factoring of this code, I don't really see any way to provide this feature without at least copying all of the code in the existing connectionMade method and then only changing the last two lines, unless you refactor it to be more friendly to this use case and contribute a patch. Such a patch would be welcome, though, if you choose to do this.

Some producer documentation is available here:

http://twistedmatrix.com/projects/core/documentation/howto/producers.html

Also, should I be using twisted.web2.client now, or is it still under development?

It is still under development (like much of Twisted - but more likely to change incompatibly than much of Twisted).

Jean-Paul