时间戳

首页 > 解决方案 > 即使在 python 中使用 .click()、implicitly_wait() 和 selenium webdriver 后也无法提取数据

python-3.x - 即使在 python 中使用 .click()、implicitly_wait() 和 selenium webdriver 后也无法提取数据

问题描述

无法从以下列表中抓取数据：

# Links
links = ['https://www.energymadeeasy.gov.au/plan?id=1ST82937MRE&postcode=2000', 'https://www.energymadeeasy.gov.au/plan?id=1ST82959MRE&postcode=2000', 'https://www.energymadeeasy.gov.au/plan?id=1ST82961MRE&postcode=2000']

我正在尝试在“其他详细信息 > 合同详细信息 > 分销商”下找到分销商名称“Ausgrid”。

在下面找到我的代码并建议需要做什么：

# Importing the library
from selenium import webdriver

driver = webdriver.Chrome(executable_path=r"...chromedriver_win32\chromedriver.exe")

# Links
links = ['https://www.energymadeeasy.gov.au/plan?id=1ST82937MRE&postcode=2000', 'https://www.energymadeeasy.gov.au/plan?id=1ST82959MRE&postcode=2000', 'https://www.energymadeeasy.gov.au/plan?id=1ST82961MRE&postcode=2000']

# Empty List
distributor = []

# Loop to extract the data and append to a list
for ind_lnk in links:
    driver.get(ind_lnk)
    driver.implicitly_wait(30)
    
    clk = driver.find_elements_by_xpath('//*[@id="content"]/div/section/div/div[1]/div[6]/div/button')
    clk[0].click()
    driver.implicitly_wait(20)
    
    try:
        distributor.append(driver.find_element_by_xpath("//*[@id='content']/div/section/div/div[1]/div[6]/div/div/div[1]/section/div[5]/span").text)
    except:
        distributor.append("")

期望输出为：['Ausgrid','Ausgrid','Ausgrid']但变空['','','']

标签： python-3.xselenium-webdriver

解决方案

它对我的工作WebDriverWait用于详细信息url

import time
from selenium import webdriver
from selenium.webdriver.support.wait import WebDriverWait
# Links
links = ['https://www.energymadeeasy.gov.au/plan?id=1ST82937MRE&postcode=2000', 'https://www.energymadeeasy.gov.au/plan?id=1ST82959MRE&postcode=2000', 'https://www.energymadeeasy.gov.au/plan?id=1ST82961MRE&postcode=2000']

# Empty List
distributor = []
browser = webdriver.Firefox(executable_path='./geckodriver')
browser.maximize_window()

for ind_lnk in links:
    browser.get(ind_lnk)
    WebDriverWait(browser, 20).until(EC.presence_of_element_located((By.XPATH, '//*[@id="content"]/div/section/div/div[1]/div[6]/div/button'))).click()
    time.sleep(3)
    try:
        distributor.append(browser.find_element_by_xpath("//*[@id='content']/div/section/div/div[1]/div[6]/div/div/div[1]/section/div[5]/span").text)
    except:
        distributor.append("")
    browser.close()

print(distributor)

推荐阅读