python - 使用按钮操作后通过 API 抓取数据

问题描述

我想从 https 响应中获取数据，但它们的数据只有在单击打开对话框窗口的按钮后才会出现（查看下面的屏幕），所以我必须打开所有必要的窗口，然后运行 request.get。主要困难是我不明白如何为我运行 driver.get 命令的同一窗口制作 request.get 。

import requests
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("start-maximized")
driver = webdriver.Chrome(options=chrome_options)
driver.get(url)
XPATH = "//*[name()='svg' and contains(@class, 'jss350')]"
XPATH2 = '/html/body/div[3]/div[3]/div/div[3]/button/span[1]'

# Wait for all svg elements to be clickable
WebDriverWait(driver, 20).until(EC.presence_of_all_elements_located((By.XPATH, XPATH)))

# find list of svg elements
svg_elements = driver.find_elements_by_xpath(XPATH)

for element in svg_elements[3:4]:
    element.click() #open windows
    WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, XPATH2))).click() #close

#here is I want to run the request.get function to grab data from the same window

driver.quit()

params = {
    'products': 'RU',
}


url = 'my url'

r = requests.get(url, params=params, headers={
    'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.132 Safari/537.36'})
json_data = r.json()
print(r.url)
print(r.text)

标签： pythonapiselenium

python - 使用按钮操作后通过 API 抓取数据

问题描述

解决方案

推荐阅读