时间戳

首页 > 解决方案 > 脚本无法继续单击加载更多按钮

python - 脚本无法继续单击加载更多按钮

问题描述

我在 Python 中编写了一个与 selenium 相关的脚本，以继续单击MORE按钮以加载更多项目，直到网页中没有新项目可供加载。MORE但是，我下面的脚本可以在该页面底部的可用按钮上单击一次。

链接到那个网站

到目前为止，这是我的尝试：

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

link = "https://angel.co/companies?company_types[]=Startup&company_types[]=Private+Company&company_types[]=Mobile+App&locations[]=1688-United+States"

driver = webdriver.Chrome()
wait = WebDriverWait(driver, 10)
driver.get(link)
while True:
    for elems in wait.until(EC.presence_of_all_elements_located((By.CSS_SELECTOR,".results .name a.startup-link"))):
        print(elems.get_attribute("href"))

    try:
        loadmore = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR,"[class='more']")))
        driver.execute_script("arguments[0].scrollIntoView();", loadmore)
        loadmore.click()
    except Exception:break

driver.quit()

我怎样才能继续单击该MORE按钮，直到没有这样的按钮可以单击和解析链接，因为我已经尝试过使用for loop.

标签： pythonpython-3.xseleniumselenium-webdriverweb-scraping

解决方案

我已经设法在我现有的脚本中解决了追求安德森爵士逻辑的问题。这就是修改后的脚本的样子。

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

link = "https://angel.co/companies?company_types[]=Startup&company_types[]=Private+Company&company_types[]=Mobile+App&locations[]=1688-United+States"

driver = webdriver.Chrome()
wait = WebDriverWait(driver, 10)
driver.get(link)
while True:
    try:
        loadmore = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR,"[class='more']")))
        driver.execute_script("arguments[0].click();", loadmore)
        wait.until(EC.staleness_of(loadmore))
    except Exception:break

for elems in wait.until(EC.presence_of_all_elements_located((By.CSS_SELECTOR,".results .name a.startup-link"))):
    print(elems.get_attribute("href"))

driver.quit()

推荐阅读