[New-bugs-announce] [issue4932] Little improvement on urlparse module, urlparse function.

[Andres Moreira]

New submission from Andres Moreira <elkpichico at gmail.com>:

Hi, 
 I make a little change in the urlsplit function of urlparse module. And
the performance when I'm parsing a lot of urls was improved a bit.
In the best case was improved around a 20% percent.

Python version:
2.5.2 (r252:60911, Oct  5 2008, 19:29:17) 
[GCC 4.3.2]

Here are the benchmarks:
#:~/tests$ python profile_urlparse.py 
timing urlparse.urlparse():
    [0.28006601333618164, 0.27513313293457031, 0.20408511161804199]
timing myurlparse.urlparse():
    [0.11000704765319824, 0.10729002952575684, 0.10677695274353027]
#:~/tests$ python profile_urlparse2.py 
timing urlparse.urlparse():
    [0.28334403038024902, 0.27912592887878418, 0.15959692001342773]
timing myurlparse.urlparse():
    [0.11277103424072266, 0.11163187026977539, 0.11175107955932617]
#:~/tests$ python profile_urlparse2.py 
timing urlparse.urlparse():
    [0.28750920295715332, 0.2779538631439209, 0.27816200256347656]
timing myurlparse.urlparse():
    [0.25010085105895996, 0.11236691474914551, 0.11198592185974121]


#-- Paste here the profiling code -----------
#-- Please rename the file as : profile_urlparse.py

urls = [
   
"http://www.notonthehighstreet.com/boxwood/product/dotty_picture_frames",
    "http://www.fancylighting.com/acatalog/Petrushka_Bronze.html",
   
"http://cgi.ebay.co.uk/3-LITRE-SUNNEX-STAINLESS-STEEL-TEAPOT-COFFEE-POT_W0QQitemZ160230173283QQcategoryZ122942QQcmdZViewItem",
   
"http://retail.ictc.co.uk/acatalog/Online_Catalogue__Homegrown_184.html",
   
"http://www.amazon.co.uk/Big-Mog-Tape-Judith-Kerr/dp/0001025252?SubscriptionId=0QE3E4F7T4Q5DCSKG202&tag=ws&linkCode=xm2&camp=2025&creative=165953&creativeASIN=0001025252",
   
"http://www.dinerstore.co.uk/acatalog/copy_of_Bedford_Oak_Table_and_Six.html",
   
"http://www.panik-design.co.uk/acatalog/Iittala_-_A_Citterio_-_Citterio_98_Cutlery_24pcs_.html",
    "http://www.johnlewis.com/230544027/Product.aspx",
   
"http://cgi.ebay.co.uk/Damask-Black-Pink-Cream-Large-Modern-Rugs-120x170cm_W0QQitemZ400021540458QQcategoryZ57237QQcmdZViewItem",
   
"http://www.amazon.co.uk/Nikon-50Mm-F1-2-Nikkor-Lens/dp/B00009R95Y?SubscriptionId=0QE3E4F7T4Q5DCSKG202&tag=ws&linkCode=xm2&camp=2025&creative=165953&creativeASIN=B00009R95Y",
   
"http://www.amazon.co.uk/Storeys-Guide-Raising-Llamas-Birutta/dp/1580173284?SubscriptionId=0QE3E4F7T4Q5DCSKG202&tag=ws&linkCode=xm2&camp=2025&creative=165953&creativeASIN=1580173284",
    "http://www.24electric.com/detail.php?ProdID=46519663",
   
"http://cgi.ebay.co.uk/Antique-Pine-Midi-Sleeper-Childrens-Bed-VGC_W0QQitemZ280285887713QQcategoryZ122763QQcmdZViewItem",
    "http://www.johnlewis.com/230421907/Product.aspx",
   
"http://cgi.ebay.co.uk/WICKER-PLACE-DINNER-MATS-X6-IN-A-WICKER-BASKET-WITH-LID_W0QQitemZ350141392941QQcategoryZ20660QQcmdZViewItem",
   
"http://www.trueshopping.co.uk/product/Draper_1_4_Square_Drive_Reversible_Ratchet/3495/43235.html",
   
"http://www.amazon.co.uk/Transcend-TS128MIB6986-128MB-Module/dp/B000HCO61K?SubscriptionId=0QE3E4F7T4Q5DCSKG202&tag=ws&linkCode=xm2&camp=2025&creative=165953&creativeASIN=B000HCO61K",
   
"http://www.unitedinteriors.co.uk/regency-pine-hi-fi-cabinet-4741-p.asp",
   
"http://www.amazon.co.uk/BATTERY-CAMCORDER-DCR-DVD602-DCR-DVD602E-DCRDVD602/dp/B0017UM1OU?SubscriptionId=0QE3E4F7T4Q5DCSKG202&tag=ws&linkCode=xm2&camp=2025&creative=165953&creativeASIN=B0017UM1OU",
    "http://www.tooled-up.com/Product.asp?PID=145138",
   
"http://www.dinerstore.co.uk/acatalog/copy_of_Bedford_Oak_Table_and_Six.html",
   
"http://www.panik-design.co.uk/acatalog/Iittala_-_A_Citterio_-_Citterio_98_Cutlery_24pcs_.html",
    "http://www.johnlewis.com/230544027/Product.aspx",
   
"http://cgi.ebay.co.uk/Damask-Black-Pink-Cream-Large-Modern-Rugs-120x170cm_W0QQitemZ400021540458QQcategoryZ57237QQcmdZViewItem",
   
"http://www.amazon.co.uk/Nikon-50Mm-F1-2-Nikkor-Lens/dp/B00009R95Y?SubscriptionId=0QE3E4F7T4Q5DCSKG202&tag=ws&linkCode=xm2&camp=2025&creative=165953&creativeASIN=B00009R95Y",
   
"http://www.amazon.co.uk/Storeys-Guide-Raising-Llamas-Birutta/dp/1580173284?SubscriptionId=0QE3E4F7T4Q5DCSKG202&tag=ws&linkCode=xm2&camp=2025&creative=165953&creativeASIN=1580173284",
    "http://www.24electric.com/detail.php?ProdID=46519663",
   
"http://cgi.ebay.co.uk/Antique-Pine-Midi-Sleeper-Childrens-Bed-VGC_W0QQitemZ280285887713QQcategoryZ122763QQcmdZViewItem",
    "http://www.johnlewis.com/230421907/Product.aspx",
   
"http://cgi.ebay.co.uk/WICKER-PLACE-DINNER-MATS-X6-IN-A-WICKER-BASKET-WITH-LID_W0QQitemZ350141392941QQcategoryZ20660QQcmdZViewItem",
   
"http://www.trueshopping.co.uk/product/Draper_1_4_Square_Drive_Reversible_Ratchet/3495/43235.html",
]

if __name__ == '__main__':
    import sys
    import timeit

    if len(sys.argv) > 1:
        times = int(sys.argv[1])
    else:
        times = 1000

    t = timeit.Timer("[urlparse.urlparse(u) for u in urls]", 
                     "from profile_urlparse import urls; import urlparse")
    print "timing urlparse.urlparse():"
    print "   ", t.repeat(3, times)


    t = timeit.Timer("[myurlparse.urlparse(u) for u in urls]", 
                     "from profile_urlparse import urls; import myurlparse")
    print "timing myurlparse.urlparse():"
    print "   ", t.repeat(3, times)

#--- End of profile code ----------------------------------------

----------
components: Library (Lib)
files: urlparse.py
messages: 79752
nosy: andrix
severity: normal
status: open
title: Little improvement on urlparse module, urlparse function.
type: performance
versions: Python 2.5
Added file: http://bugs.python.org/file12720/urlparse.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue4932>
_______________________________________