post - 无法使用 python-request 模块登录百思买

Question

我正在尝试使用 python 请求模块登录百思买 (ca) 网站：https : //www.bestbuy.ca/profile/signin.aspx，但我无法这样做。我的代码如下：

import requests

userInfo = {'username': '[e-mail]', 'password': 'anonymous'}
loginUrl = 'https://www.bestbuy.ca/profile/signin.aspx'
s = requests.Session()
s.headers['User-Agent'] = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.82 Safari/537.36'
r = s.get(loginUrl) #Redirects me to the actual login page
tid = r.url[r.url.find('tid=')+4:] #tid found in final redirected link itself
userInfo['tid'] = tid
r = s.post('https://www.bestbuy.ca/identity/login', data=userInfo)

有了这个，我得到一个 403 access denied 错误。我检查了 chrome 中的页面，发现以下登录请求包含常规、标头和有效负载字段：

*General*

Request URL: https://www.bestbuy.ca/identity/login
Request Method: POST
Status Code: 401 
Remote Address: 23.36.176.78:443
Referrer Policy: strict-origin-when-cross-origin

*Request Headers*

:authority: www.bestbuy.ca
:method: POST
:path: /identity/login
:scheme: https
accept: application/vnd.bestbuy+json
accept-encoding: gzip, deflate, br
accept-language: en-US,en;q=0.9
cache-control: no-cache
content-length: 866
content-type: text/plain;charset=UTF-8
cookie: ai_user=wKE6/1DR2TjH+loc7XJJNk|2021-03-10T22:02:00.409Z; clientId=o4YNH8yEUqslZFPx; criteoVisitorId=60951d2e-b5fd-4a23-bc2e-b945143d3ff9; s_ecid=MCMID%7C31251505297131553713839324795245895414; __gads=ID=9885b8fd218feb95:T=1615495544:S=ALNI_MaOAgs7R24M8UvcOatHQ42DjV8_Ig; BVBRANDID=c0321986-a245-431e-a1ef-96070be0fe6f; _gcl_au=1.1.1987592661.1615495545; _ga=GA1.2.562987812.1615495544; _pin_unauth=dWlkPVlXVXhNVFZtWW1RdFkyUXdNeTAwTWpVNExXRTFaakV0TjJFMFlUUTRNR1pqWkRCaA; _fbp=fb.1.1615495546617.1959428176; rxVisitor=1615497402570V3P2T6HTDC3S8D8QBM456ATFMAKBEA32; surveyOptOut=1; ReturnUrl=https://www.bestbuy.ca/; enabled=1; BVImplmain_site=18193; at_check=true; AMCVS_D6E638125859683E0A495D2D%40AdobeOrg=1; s_cc=true; s_fid=33796110583641A8-32BC6499EAD6F939; _gid=GA1.2.4050501.1615818482; fdb7491a5cc3d693edd0926b3a48659f=1caf4b56d46d54b9dc2e28357e6c6cd4; tia=88dfb9d44ddfb42be4f9d11be0e2be0159d619f81c4a8311ddd5b0ea5162691f; d610aaaf3438bb52dd98fd19edd17a8c=0af4110ed7c9ed471d9e30a7d165e968; e13d5bc268331a905ff9fe23a1820225=718e925ec6f4f66725c4533501ebe0de; 4b92e78b2a2c24b9f5aaf8ff1d99b6ef=71a9ab6fd77c3138eed618f97fd5d8ad; cid=%7B%22email%22%3A%22randyk98%40hotmail.com%22%2C%22id%22%3A%22%7B883b3f7e-ca8d-4dd2-acc7-b42cbf885a64%7D%22%2C%22firstName%22%3A%22Randy%22%2C%22lastName%22%3A%22Khoshaba%22%2C%22authenticationState%22%3A%22UNAUTHENTICATED%22%7D; dtm_mSession=055d12f70a1b41feb3cd100d41e698c9; bm_sz=5E67298E306C07C853104960DC6800DE~YAAQhZo7F3jcIxJ4AQAAwlqAOwspnWJIhJWvgA8u/o5IBY7AESdFQcaP4u5Y9XSsTnIqEHoAJaxjYLS63irc50oLztsAqghlppyUPtCG2AyFw0N0hbdVD8VpWVbAuDAaObBkEKfY74qTUrjhIPLduiezD2He3V+JLdLhF8xzFFDu3LI2d6iKsRxrOV4W2xr8Xbc2etkKc0fwtwPN6QbVZOL0mIKtU7rWr588wdJJKY7y8V9/i0xg6oomvluB602Fq1wQxqmdM9CZUJTTbxcoHhy/Rtird87oYlS3nQ4=; _abck=41FAA19E59C2F2D1C65AB3DD13E9A2F9~0~YAAQhZo7F3ncIxJ4AQAAwlqAOwXctkoIjQPhot25/ki0W06KVRCI9OFl338+jnaVwiV3Qp+AjCAfLbkgHmfNp6T28yH0I2jNH1tD+IG8UO/4GxaTJO1BSnq3qnchMBcrYjp7CQ3vYER3M0DfxU/DPVFLCBqEgmkH4+HwYOpXyVlijOTT17blmR9boBnDfbWrfTW40WI6I0Mg5d1F14gWWhzlrjPTGMFZuiIeADocorDoybDYY6M8GDv1JCCUHvjiTUx5nkrA8uspMFKUZpS7GROecdqumla1RW+/mrQ1IHCAeN04oA6W0EacYf18Pg7lCnzqtQYHB2R4yl23l4HVA+WmxeNX/iAITBQtV3QZnhjfqan6d2n5p9RCDYdajw7eCZsoElHotgPaJfvp0XXwu6HhTzwSn2aSwgxSwNzSp8SpvnvLmrXZquGWd/m9QfI=~-1~-1~-1; dtLatC=42; bm_mi=0D3DA468DA0B9502B9828F6339298277~fks6JM8WlmWRLVYNRRvXWkbbIyUls6FHpWnhJFqsqcH4TZD38CV7DkeXD9QRnFDIpbx3B3s9J6+aIul6LhlrqL5lQPqXrdCFyqimvvjrDw1hGTbAoUdeJmutnrAZ0spq+oYyZqKqDWvP23cZhuu8SWasXFe7hymx+jIg40jS/KzdxsIaaLb+rmx8cLg/OEnXfhI2DNxrYFFsP3dEZZEAMBMRjjAAs4w2D58MyQdf2ZAq8ldgrNUuNG7N2HPepRF2; mbox=PC#a417f07fe9f24779900d3ebc76c00940.35_0#1679092041|session#539592b00c0840f7932cdd170ed78b2c#1615907834; nps={"currentUrlPath":"/en-ca","hasSurveyBeenDisplayed":false,"heartBeat":1615905974,"isInSampling":false,"pageViewCount":0,"surveyLastDisplayed":1647031544}; ak_bmsc=82E42670BDBD54960A0B59CD122BBEA1173B9A8512750000B2C450604691AD70~plkbMeXTXCL7hMIiM2RRACeuAthhljfcXOA1j0gw2ZhVi8OdJEiZvS3EGUTBPeJ5fh9Ntz8+A58b562CwjhQs949AQ1i//OtJESKrtONkWg7G7KHL+8GxGAy/RJii9gfBY5lK0TIKlQ9iCL6VDvDrdhgJgZYKabYOw9xGT1urz1nSl4nbDkj0dTxYO0UDkBZnuFTzwgSL1rWeRpe6Nl7VpS4CBAtvlejwIs8HVDjJN3FBbgnwvXA7gbPqhq1ZW9XnH; dtSa=false%7CC%7C11%7CAccount%7Cx%7C1615905974970%7C105971014_211%7Chttps%3A%2F%2Fwww.bestbuy.ca%2Fen-ca%7CBest%20Buy%3A%20Shop%20Online%20For%20Deals%20%26%20Save%20%5Ep%20Best%20Buy%20Canada%7C1615905975305%7C%7C; dtPC=1$105971014_211h-vUKEHJFUKIIMEQQWKCTOQFWFMQCCGCCAO-0e2; dtCookie=1$I9HCMHVKHVE5OTMV9B191SGV3R03RCPN|ea7c4b59f27d43eb|0; rxvt=1615907775848|1615905971037; s_lv_s=Less%20than%201%20day; s_vnum=1617260400784%26vn%3D14; s_invisit=true; AA_previousCategory=sign%20in; AA_previousPageName=my%20account%20|sign%20in; AA_previousPageType=; AA_previousTA=; AMCV_D6E638125859683E0A495D2D%40AdobeOrg=-1303530583%7CMCIDTS%7C18702%7CMCMID%7C31251505297131553713839324795245895414%7CMCAAMLH-1616510779%7C9%7CMCAAMB-1616510779%7CRKhpRz8krg2tLO6pguXWp5olkAcUniQYPHaMWWgdJ3xzPWQmdj0y%7CMCOPTOUT-1615913179s%7CNONE%7CMCAID%7CNONE%7CvVersion%7C3.3.0%7CMCCIDH%7C-1328349937; s_sq=%5B%5BB%5D%5D; tx=eyJhbGciOiJIUzI1NiJ9.eyJqdGkiOiIzNDBiOGJmMy0xYmY1LTQxY2ItYmEyZS0xYzdjMzA1OGUwNDkifQ.DDY5A_18QcKw02ryjztHflwEXOh_MBGHCsGMhBa8-b8; bm_sv=7598B2F60D9A4BA534D52B5C21A5EC4A~IZZXDCqlvQFlAUqgCJ5qlEzQBU+Vw1M9zJKWyOTqFn+Vu/z9MJGHMVotkR+9ZX6JoU45iAZemWMPYwhYNw6ccEs7qag0wOPgztf+8A10DUNKLjrmL0aF8aJwrvHHcTu10tvgNBQNsnt/kLiIpIMrVCu7qU4L+dhxsmoO8nc7a/c=; ai_session=0Vfaww8IrNRbkV3rz5JMD2|1615905975357|1615907628042.87; AA_pagePathingCount=2; s_lv=1615907630577; s_getNewRepeat=1615907630587-Repeat
origin: https://www.bestbuy.ca
pragma: no-cache
referer: https://www.bestbuy.ca/identity/en-ca/signin?tid=le%252Fp0RI4kno2n1H8JBxYKoeiTo4DkxZodt8oaDlpK%252FZdTL1t7%252FK04QDgj0yzlO3NHJxUq%252F0PzIFHf62XWuBhRm%252BYTDop2BL94ouDWEMcafUVZLtiuu1zlNlVgrgg9NqTfNOzs3SToo7QI3yEPmEeNEh7h7LplK4pE1FI2o4VOLfqQproLazcDIoLNyNVxV4fjFoAL6168SiHMKvqclZNsz%252B88FIb5D90LfoJu3SvHK19QfRyubN8VS04tjwiXc6kxh3vmEOsXIZoRs9g3n0bQmxSNnbPTKO9maxOzqMf%252FXo%253D
sec-ch-ua: "Google Chrome";v="89", "Chromium";v="89", ";Not A Brand";v="99"
sec-ch-ua-mobile: ?0
sec-fetch-dest: empty
sec-fetch-mode: cors
sec-fetch-site: same-origin
user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.82 Safari/537.36
{,…}

*Request Payload*

captchaResponse: "03AGdBq27ZYIQFLlKtkRexvMdHLAsXmgSVLwOcaQtbEqTUyw3Dw19TOzyTLJJPUMJTHShjDQBiR0OxUlwroIflwZvz6V2NtvgM70LjCNbAyaMFq46J_y-sj1iX7oFk9Ka9SDMJutTy6hNORKfGsCvgtPQ5mYDDIAlRpm75NYtuTWx_S6rpaMdjsvWY2hXPgQ2pLdBF1_Xj1KOBmsx3a6RIAeg_32WkSOUa-UdvK-2Nlqdybw_dV5cmy6-DIyc_3aWamjpHQ0WMRdfNurWdhGFbtn9s3nDV7NC8tzx8MJSfK_3PrKun1WjpNY2ZdlUQitvlpijsDZNo3SwtEGjVtJUzSi_0qZeYm8F1XsHDYoDI2wgWnrnqm8MJ9hqEHYove34l5G61pXt3mR62d1IWZXlwJbR3aARV0rotZSnTJ3cedAeZUA_qz62qNLslyofryWRvMJjsl0z3B4Id"
password: "anonymous"
tid: "le%2Fp0RI4kno2n1H8JBxYKoeiTo4DkxZodt8oaDlpK%2FZdTL1t7%2FK04QDgj0yzlO3NHJxUq%2F0PzIFHf62XWuBhRm%2BYTDop2BL94ouDWEMcafUVZLtiuu1zlNlVgrgg9NqTfNOzs3SToo7QI3yEPmEeNEh7h7LplK4pE1FI2o4VOLfqQproLazcDIoLNyNVxV4fjFoAL6168SiHMKvqclZNsz%2B88FIb5D90LfoJu3SvHK19QfRyubN8VS04tjwiXc6kxh3vmEOsXIZoRs9g3n0bQmxSNnbPTKO9maxOzqMf%2FXo%3D"
username: "[e-mail]"

请注意，我在上面故意使用了错误的凭据，因为发布请求会（似乎）在发布请求成功后从网络选项卡中消失。使用正确的电子邮件和密码，浏览器请求在 python 请求失败的地方成功。

在此登录请求之前，有一个 userVerify POST 请求到https://www.google.com/recaptcha/api2/userverify?k=6LdLhXEUAAAAACZo2QdzqcnoSN7FvEZT8pMykY2s，但是当我搜索该请求时，似乎没有匹配的验证码响应字段上面的登录请求。这个 captchaResponse 字段是未授权错误的原因吗？如果是这样，我怎样才能获得该值？

我不想使用硒。

标签： postweb-scrapingpython-requests