python - 当我尝试在 Amazon 中使用请求模块/python 进行抓取时的交付和位置问题

Question

我想使用请求库在 amazon.com 中抓取订单的价格

问题是，亚马逊没有给我相关数据，因为我无法选择送货地址。如何在 cookie 中添加收货地址？

例子; 当我在 chrome 上选择交付时

当我没有选择

我的代码：

from bs4 import BeautifulSoup
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
import pandas as pd
import random
import requests
from datetime import datetime
import re
from lxml import etree
from random import choice
import json
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.chrome.options import Options

random1 = random.randrange(7, 10, 1)
random2 = random.randrange(70, 79, 1)
random3 = random.randrange(100, 130, 1)
random4 = random.randrange(5000, 9999, 1)
headers = {
        'User-Agent': "Mozilla/5.0 (Windows NT "+str(random1)+".0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/"+str(random2)+".0."+str(random4)+"."+str(random3)+" Safari/537.36",
        'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
        'Accept-Language': 'en-US,en;q=0.5',
        'Accept-Encoding': 'gzip, deflate, br',
        'Connection': 'keep-alive',
        'Upgrade-Insecure-Requests': '1',
        'TE': 'Trailers'}

url = "https://www.amazon.com/gp/offer-listing/B07PXGQC1Q/ref=olp_f_new?ie=UTF8&f_new=true"
req = requests.get(url, headers=headers)
soup = BeautifulSoup(req.text, 'lxml')


pricedata=soup.select(".olpOfferPrice")
print(pricedata)

标签： pythonseleniumweb-scrapingpython-requests