时间戳

首页 > 解决方案 > BeautifulSoup python的网页抓取

python - BeautifulSoup python的网页抓取

问题描述

我正在尝试通过 BeautifulSoup抓取在线商店的产品，但我有一个问题，我只能抓取一种产品！，但我想刮掉所有产品。

import requests
from bs4 import BeautifulSoup
res = requests.get('https://www.digikala.com/search/category-wearable-gadget/')
soup = BeautifulSoup(res.text, 'html.parser')
result = soup.find_all('div', class_='c-product-box__content')
first_watch = result[0]
first_name = first_watch.a.text
first_rate = first_watch.find(class_='c-product-box__rate-comparision--rate-people')
first_price = first_watch.find(class_='c-price__value-wrapper')
first_rate = first_rate.text
first_price = first_price.text

一旦我使用first_watch = result并使用find_all 显示此错误：

回溯（最后一次调用）：文件“C:/Users/Oogway/PycharmProjects/web_scraping1/web_s.py”，第 8 行，在 first_name = first_watch.a.text 文件“C:\Users\Oogway.virtualenvs\web_scraping1\ lib\site-packages\bs4\element.py"，第 2160 行，in__getattr__ raise AttributeError( AttributeError: ResultSet object has no attribute 'a'。您可能将元素列表视为单个元素。您调用 find_all( ) 当你打算调用 find() 时？

import requests
from bs4 import BeautifulSoup
res = requests.get('https://www.digikala.com/search/category-wearable-gadget/')
soup = BeautifulSoup(res.text, 'html.parser')
result = soup.find_all('div', class_='c-product-box__content')
first_watch = result
first_name = first_watch.a.text
first_rate = first_watch.find_all(class_='c-product-box__rate-comparision--rate-people')
first_price = first_watch.find_all(class_='c-price__value-wrapper')
first_rate = first_rate.text
first_price = first_price.text

标签： pythonweb-scrapingbeautifulsoup

解决方案

您需要迭代result对象。

import requests
from bs4 import BeautifulSoup
res = requests.get('https://www.digikala.com/search/category-wearable-gadget/')
soup = BeautifulSoup(res.text, 'html.parser')
result = soup.find_all('div', class_='c-product-box__content')
for itm in result:
    print(itm.a.text)

find_all方法将返回所有div带类c-product-box__content。

推荐阅读