Visit All URLs with selenium python

Deborah Swanson python at deborahswanson.net
Thu Apr 13 02:56:15 EDT 2017


Nicole wrote, on Wednesday, April 12, 2017 11:30 PM
> 
> Here you can see now
> 
> from selenium.webdriver.firefox.firefox_profile import FirefoxProfile 
> import random 
> from selenium import webdriver 
> from selenium.webdriver.common.keys import Keys 
> 
> browser.get('https://www.google.co.uk/search?q=Rashmi&oq=Rashm
> i&aqs=chrome..69i57j69i60l3.6857j0j1&sourceid=chrome&ie=UTF-8#
> q=Mobiles+in+london')
> time.sleep(5)
> 
> try:        
>         p_links = browser.find_elements_by_css_selector(' div 
> > h3 > a')
>         url_list = []
>         for urls in p_links:
>             if "London" in urls.text:
> 
>                 urls.get_attribute("href")
>                 url_list.append(urls)
>                 for links in url_list:
>                     browser.get(links)
>                     time.sleep(4)
> except:
>     browser.close()

Ok, I'm sure you changed the search terms, don't know if you changed the
google url, but the code looks the same.

I'm headed away from the computer for the night, but try using a shorter
sleep time, like

time.sleep(1)



More information about the Python-list mailing list