时间戳

首页 > 解决方案 > 为什么在 python 烧瓶中使用 Selenium 会出现 No such element 异常？

python - 为什么在 python 烧瓶中使用 Selenium 会出现 No such element 异常？

问题描述

我在 python 烧瓶中使用 selenium 编写了一个程序，它对产品进行网络抓取。我的意图是首先输入产品名称（这是通过编程完成的）->显示产品后->它应该显示我将在终端中显示的产品价格。然而，我的问题是它不会抓取网站并且会抛出Selenium NoSuchElementException. 这是我的代码。

def scrape_johnLewis(product_name):
    website_address = 'https://www.johnlewis.com/'
    options = webdriver.ChromeOptions()
    options.add_argument('start-maximized')
    options.add_experimental_option("excludeSwitches", ["enable-automation"])
    options.add_experimental_option('useAutomationExtension', False)

    browser = webdriver.Chrome(ChromeDriverManager().install(), options=options)

    browser.get(website_address)
    time.sleep(10)
    browser.implicitly_wait(20)
    browser.find_element_by_css_selector('button.c-button-yMKB7 c-button--primary-39fbj c-button--inverted-UZv88 c-button--primary-3tLoH').click()
    browser.find_element_by_id('mobileSearch').send_keys(product_name)
    browser.find_element_by_css_selector('div.input-with-submit.module-inputWrapper--63f9e > button.button.module-c-button--fe2f1').click()
    time.sleep(5)
    # browser.find_elements_by_class_name('button.module-c-button--fe2f1')[0].submit()
    product_price_raw_list = browser.find_elements_by_xpath('//div[@class="info-section_c-product-card__section__2D2D- price_c-product-card__price__3NI9k"]/span')
    product_price_list = [elem.text for elem in product_price_raw_list]
    print(product_price_list)


if __name__ == "__main__":
    scrape_johnLewis('Canon EOS 90D Digital SLR Body')

我得到的错误在这里browser.find_element_by_css_selector('button.c-button-yMKB7 c-button--primary-39fbj c-button--inverted-UZv88 c-button--primary-3tLoH').click()

这是堆栈跟踪：

Traceback (most recent call last):
  File "scrapejohnLewis.py", line 32, in <module>
    scrape_johnLewis('Canon EOS 90D Digital SLR Body')
  File "scrapejohnLewis.py", line 20, in scrape_johnLewis
    browser.find_element_by_css_selector('button.c-button-yMKB7 c-button--primary-39fbj c-button--inverted-UZv88 c-button--primary-3tLoH').click()
  File "/home/mayureshk/.local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 598, in find_element_by_css_selector
    return self.find_element(by=By.CSS_SELECTOR, value=css_selector)
  File "/home/mayureshk/.local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 978, in find_element
    'value': value})['value']
  File "/home/mayureshk/.local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/home/mayureshk/.local/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"button.c-button-yMKB7 c-button--primary-39fbj c-button--inverted-UZv88 c-button--primary-3tLoH"}
  (Session info: chrome=74.0.3729.108)
  (Driver info: chromedriver=74.0.3729.6 (255758eccf3d244491b8a1317aa76e1ce10d57e9-refs/branch-heads/3729@{#29}),platform=Linux 5.0.0-1034-oem-osp1 x86_64)

我试图用 find_element_by_tag_name 替换它，但这也没有完成工作。通过检查网站，我找到了确切的元素，但令我惊讶的是，错误表明没有这样的元素。究竟可能是什么情况？请帮忙。

标签： pythonpython-3.xseleniumselenium-webdriverselenium-chromedriver

解决方案

cookie 提示没有正确管理，它没有消失，因此它阻止了下面的代码，从而停止Selenium查找元素。我还对代码进行了一些调整。

from selenium.webdriver.common.keys import Keys

def scrape_johnLewis(product_name):
    website_address = 'https://www.johnlewis.com/'
    options = webdriver.ChromeOptions()
    options.add_argument('start-maximized')
    options.add_experimental_option("excludeSwitches", ["enable-automation"])
    options.add_experimental_option('useAutomationExtension', False)

    browser = webdriver.Chrome(ChromeDriverManager(log_level='0').install(), options=options)

    browser.get(website_address)
    time.sleep(3)
    # browser.implicitly_wait(20)
    browser.find_element_by_xpath('//*[@id="pecr-cookie-banner-wrapper"]/div/div[1]/div/div[2]/button[1]').click()
    browser.find_element_by_id('desktopSearch').send_keys(product_name + Keys.ENTER)
    time.sleep(5)
    product_price_raw_list = browser.find_elements_by_xpath('//div[@class="info-section_c-product-card__section__2D2D- price_c-product-card__price__3NI9k"]/span')
    product_price_list = [elem.text for elem in product_price_raw_list]
    print(product_price_list)

# ouptut
['£1,249.00', '£1,629.99', '£1,349.99']

css_selector如果您找不到更好的元素定位器，例如id, xpath, ，也请尝试使用作为最后的手段tag_name。

推荐阅读