python - 如何修复代码以抓取 Zomato 网站?
问题描述
我写了这段代码,但在运行最后一行后得到了错误“IndexError:list index out of range”。请问,我该如何解决这个问题?
import requests
from bs4 import BeautifulSoup
headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML,
like Gecko) Chrome/61.0.3163.100 Safari/537.36'}
response = requests.get("https://www.zomato.com/bangalore/top-restaurants",headers=headers)
content = response.content
soup = BeautifulSoup(content,"html.parser")
top_rest = soup.find_all("div",attrs={"class": "sc-bblaLu dOXFUL"})
list_tr = top_rest[0].find_all("div",attrs={"class": "sc-gTAwTn cKXlHE"})
list_rest =[]
for tr in list_tr:
dataframe ={}
dataframe["rest_name"] = (tr.find("div",attrs={"class": "res_title zblack bold nowrap"})).text.replace('\n', ' ')
dataframe["rest_address"] = (tr.find("div",attrs={"class": "nowrap grey-text fontsize5 ttupper"})).text.replace('\n', ' ')
dataframe["cuisine_type"] = (tr.find("div",attrs={"class":"nowrap grey-text"})).text.replace('\n', ' ')
list_rest.append(dataframe)
list_rest
解决方案
我最近做了一个项目,让我研究了在菲律宾马尼拉的 Zomato 网站。我使用 Geolibrary 获取马尼拉市的经度和纬度值,然后使用这些信息抓取餐厅的详细信息。ADD:您可以在zomato网站上获取自己的API密钥,一天最多可以进行1000次调用。
# Use geopy library to get the latitude and longitude values of Manila City.
from geopy.geocoders import Nominatim
address = 'Manila City, Philippines'
geolocator = Nominatim(user_agent = 'Makati_explorer')
location = geolocator.geocode(address)
latitude = location.lenter code hereatitude
longitude = location.longitude
print('The geographical coordinate of Makati City are {}, {}.'.format(latitude, longitude))
# Use Zomato's API to make call
headers = {'user-key': '617e6e315c6ec2ad5234e884957bfa4d'}
venues_information = []
for index, row in foursquare_venues.iterrows():
print("Fetching data for venue: {}".format(index + 1))
venue = []
url = ('https://developers.zomato.com/api/v2.1/search?q={}' +
'&start=0&count=1&lat={}&lon={}&sort=real_distance').format(row['name'], row['lat'], row['lng'])
try:
result = requests.get(url, headers = headers).json()
except:
print("There was an error...")
try:
if (len(result['restaurants']) > 0):
venue.append(result['restaurants'][0]['restaurant']['name'])
venue.append(result['restaurants'][0]['restaurant']['location']['latitude'])
venue.append(result['restaurants'][0]['restaurant']['location']['longitude'])
venue.append(result['restaurants'][0]['restaurant']['average_cost_for_two'])
venue.append(result['restaurants'][0]['restaurant']['price_range'])
venue.append(result['restaurants'][0]['restaurant']['user_rating']['aggregate_rating'])
venue.append(result['restaurants'][0]['restaurant']['location']['address'])
venues_information.append(venue)
else:
venues_information.append(np.zeros(6))
except:
pass
ZomatoVenues = pd.DataFrame(venues_information,
columns = ['venue', 'latitude',
'longitude', 'price_for_two',
'price_range', 'rating', 'address'])
推荐阅读
- oracle-sqldeveloper - 无法在 SQL Developer for Oracle 的单独目录中导出数据
- javascript - 如何批量提交()用于firestore大量文档?
- python - 在 ListView django 中进行分页
- java - 设置用于验证从服务器接收到的证书的最小 RSA 密钥长度
- ruby - 授权错误,错误 400:redirect_uri_mismatch Gmail API
- c# - 如何在 ASP.NET Core 5 Blazor 服务器模式下要求身份验证并重定向到登录
- reactjs - 反应路由器 dom:
不适用于无效的嵌套路径,例如 localhost:9090/{invalidaa}/{invalidb}/{invalidc} - ruby-on-rails - 在 Rails 中广播消息时未调用 ActionCable 频道
- r - RStudio - RMarkdown - Knit to PDF 将创建一个显示使用的代码但不显示代码结果的 pdf
- c# - 如何获取 Xamarin Forms 中 ObservableCollection 的第一个元素的数据?