时间戳

首页 > 解决方案 > 使用 BeautifulSoup 抓取问题

python - 使用 BeautifulSoup 抓取问题

问题描述

我是使用 python 抓取数据的新手，它给了我和错误以及如何解决它。

这是我的代码。

from bs4 import BeautifulSoup
import requests

html_text = requests.get('https://www.olx.com.pk/mobile-phones_c1453/q-iphone-x')
# print(html_text)
soup = BeautifulSoup(html_text, 'lxml')

info = soup.findAll('ul', class_ = 'de20bb96 _021e1d41')

for one_info in info:
    single_phone = soup.find('li', class_ = 'Listing').text

    print(single_phone)

这是错误信息！！

E:\Softwares\Anaconda\python.exe "D:/Courses/Webscraping using python/olxweb.py" Traceback（最近一次调用最后）：文件“D:/Courses/Webscraping using python/olxweb.py”，第 7 行，在 soup = BeautifulSoup(html_text, 'lxml') 文件“E:\Softwares\Anaconda\lib\site-packages\bs4_ init _.py”中，第 310 行，在init elif len(markup) <= 256 和 (TypeError ：“响应”类型的对象没有 len() 进程以退出代码 1 结束

标签： pythonweb-scraping

解决方案

.content在html_text变量上使用：

import requests
from bs4 import BeautifulSoup

html_text = requests.get(
    "https://www.olx.com.pk/mobile-phones_c1453/q-iphone-x"
)
soup = BeautifulSoup(html_text.content, "lxml")  # <-- use .content

for one_info in soup.select('li[aria-label="Listing"]'): # <-- select by aria-label= atribute
    print(one_info.get_text(strip=True, separator=" "))

印刷：

Featured Rs 64,000 Iphone X for sale Chauburji Park, Lahore 5 days ago
Featured Rs 88,000 Iphone x 256gb (PTA Approved) 10/10 Anarkali, Lahore 2 weeks ago
Featured Rs 48,000 iphone X DHA Defence, Islamabad 3 days ago
Featured Rs 76,000 iphone x 256gb pta approved Shalamar Town, Lahore 1 week ago
Featured Rs 68,000 IPHONE X 64GB PTA APPROVED Dharampura, Lahore 2 weeks ago
Featured Rs 69,500 Iphone X Kot Khawaja Saeed, Lahore 1 week ago
Rs 85,000 Iphone x Multan 3 minutes ago
Rs 72,000 iphone X 64Gb Pta approve Gujrat 9 minutes ago
Rs 110,000 iphone x  s max for sale condition 10/9 pta aproved Wazirabad 14 minutes ago
Rs 3,000 iphone x orignal panel touch not working Askari, Lahore 15 minutes ago
Rs 40,000 IPhone x 64 gb Bahadurabad, Karachi 20 minutes ago
Rs 82,000 Iphone X 256 GB Ghazi Road, Lahore 22 minutes ago
Rs 93,000 iPhone X 64 GB Peshawar 26 minutes ago
Rs 30,000 iphone x bypass Model Town Extension, Lahore 29 minutes ago
Rs 75,000 iphone x few month used battery heath 94% Mandi Bahauddin 29 minutes ago
Rs 67,000 iphone x for sale Attock 30 minutes ago
Rs 65,000 iphone x Haidery, Karachi 33 minutes ago
Rs 65,000 iphone x 64gb white colour 7th Avenue, Islamabad 38 minutes ago
Rs 95,000 iphone x white Gulzar Colony, Lahore 53 minutes ago
Rs 80,000 iphone x white colour 64gb Multan 55 minutes ago

推荐阅读