python-3.x - 如何向网站发出请求并下载搜索数据

Question

我正在尝试为个人列表下载一些汽车 VIN 数据。这包括通过 python3.7 中的 requests.get向网站“ http://vin.place/search.php ”发出请求。

我的代码如下所示：

import requests
import pandas as pd
from bs4 import BeautifulSoup

payload = {'first name':'JOHN','last name':'DOE'}
webpage_response = requests.get("http://vin.place/search.php",data = payload)

webpage = webpage_response.content
soup = BeautifulSoup(webpage,"html.parser")

results = soup.select(".search-content")

for x in results:
  print(x.get_text())

不幸的是，我发现 results = [] 相反。我怀疑我编写requests.get命令的方式有问题。我不确定网站的正确键应该是多少。有人可以帮忙吗？谢谢。

标签： python-3.xbeautifulsouppython-requests

Answer 1

您的有效载荷是错误的，应该是first, last。以及小写的名称：

import requests
from bs4 import BeautifulSoup

payload = {'first':'john','last':'doe'}
webpage_response = requests.post("http://vin.place/search.php", data=payload)

webpage = webpage_response.content
soup = BeautifulSoup(webpage,"html.parser")

results = soup.select(".search-content")

for x in results:
    print(x.text)

印刷：

JOHN DOE  Purchase of a  2007 HONDA ACCORD
JOHN DOE  Purchase of a  2007 CHRYSLER TOWN AND COUNTRY
JOHN DOE  Purchase of a  2007 DODGE RAM PICKUP 2500
JOHN DOE  Purchase of a  2007 DODGE RAM PICKUP 1500
JOHN DOE  Purchase of a  2007 VOLKSWAGEN PASSAT
JOHN DOE  Purchase of a  2007 VOLKSWAGEN JETTA
JOHN DOE  Purchase of a  2007 CHRYSLER SEBRING
JOHN DOE  Purchase of a  2007 DODGE RAM PICKUP 1500
JOHN DOE  Purchase of a  2007 JEEP PATRIOT
JOHN DOE  Purchase of a  2007 Chevrolet TrailBlazer
JOHN DOE  Purchase of a  2007 FORD EDGE
JOHN DOE  Purchase of a  2007 JEEP WRANGLER UNLIMITED
JOHN DOE  Purchase of a  2007 DODGE RAM PICKUP 2500
JOHN DOE  Purchase of a  2007 FORD MUSTANG
JOHN DOE  Purchase of a  2007 JEEP WRANGLER
JOHN DOE  Purchase of a  2007 CHRYSLER 300
JOHN DOE  Purchase of a  2007 JEEP WRANGLER UNLIMITED
JOHN DOE  Purchase of a  2007 DODGE MAGNUM
JOHN DOE  Purchase of a  2007 CADILLAC ESCALADE ESV
JOHN DOE  Purchase of a  2007 FORD F-150
JOHN DOE  Purchase of a  2007 HONDA ACCORD
JOHN DOE  Purchase of a  2007 JEEP LIBERTY
JOHN DOE  Purchase of a  2007 CADILLAC ESCALADE
JOHN DOE  Purchase of a  2007 DODGE RAM PICKUP 1500
JOHN DOE  Purchase of a  2007 JEEP WRANGLER