pycurl and MAX_RECV_SPEED_LARGE
Emanuel dos Reis Rodrigues
emanueldosreis at gmail.com
Wed Jun 15 22:28:22 EDT 2011
Hello Folks,
I have a problem with pycurl. I need to do a download with a lower rate,
+ or - 1 Kb / 128bytes.
I use the MAX_RECV_SPEED_LARGE setting with 128 as value.
My problem is: The download take a long time to be finished.
File: test.jpg 92 KB, with 128 rate, take 2.38 Minutes.
The strange behavior is that: The connection keep ESTABLISHED only 1/5
from all time ( 2.38 minutes )
Analyzing with wireshark, I see that: After pass 1/5 of all time, do not
change packages anymore until reach 2.38 minutes, and client send a RST ACK.
Have a high delay between the transfer and the finished connection.
I need to solution that I can download with a lower rate, include using
Multi-Threading with many simultaneous downloads.
Follow my test code:
import pycurl
import random
import os
def download1(t_url):
headers = [
'Accept:
text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
'Accept-Language: en-us,pt-br;q=0.8,pt;q=0.5,en;q=0.3',
'Accept-Encoding: gzip,deflate',
'Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7',
'Cache-control: no-cache',
'Pragma: no-cache',
'Connection: Keep-Alive',
'Keep-Alive: 300']
c = pycurl.Curl()
f = "/tmp/teste.tmp"
c.setopt(c.URL, t_url)
c.setopt(c.FAILONERROR, 1)
# c.setopt(c.VERBOSE, 1)
c.setopt(c.MAX_RECV_SPEED_LARGE, 128)
# c.setopt(c.TIMEOUT, timeout)
c.setopt(c.NOSIGNAL, 1)
c.setopt(pycurl.HTTPHEADER, headers)
c.setopt(c.CONNECTTIMEOUT,10)
c.setopt(c.WRITEDATA, file(f,"w"))
c.setopt(c.USERAGENT,"XADASSDASDASD")
try:
c.perform()
except pycurl.error:
print "Connection Problem"
c.close()
return 2
else:
print "Download Complete"
print f
c.close()
return 0
download1("http://192.168.111.128/teste.jpg")
Thank You.
Emanuel
More information about the Python-list
mailing list