Visit All URLs with selenium python
Deborah Swanson
python at deborahswanson.net
Thu Apr 13 02:56:15 EDT 2017
Nicole wrote, on Wednesday, April 12, 2017 11:30 PM
>
> Here you can see now
>
> from selenium.webdriver.firefox.firefox_profile import FirefoxProfile
> import random
> from selenium import webdriver
> from selenium.webdriver.common.keys import Keys
>
> browser.get('https://www.google.co.uk/search?q=Rashmi&oq=Rashm
> i&aqs=chrome..69i57j69i60l3.6857j0j1&sourceid=chrome&ie=UTF-8#
> q=Mobiles+in+london')
> time.sleep(5)
>
> try:
> p_links = browser.find_elements_by_css_selector(' div
> > h3 > a')
> url_list = []
> for urls in p_links:
> if "London" in urls.text:
>
> urls.get_attribute("href")
> url_list.append(urls)
> for links in url_list:
> browser.get(links)
> time.sleep(4)
> except:
> browser.close()
Ok, I'm sure you changed the search terms, don't know if you changed the
google url, but the code looks the same.
I'm headed away from the computer for the night, but try using a shorter
sleep time, like
time.sleep(1)
More information about the Python-list
mailing list