python - python web-crawler 猜测解析器警告

Question

我正在尝试使用 python (3.8) 制作一个网络爬虫，我主要认为我已经完成了，但是我遇到了这个错误，任何人都可以帮助我并提前感谢。

蟒蛇代码：

import requests
from bs4 import BeautifulSoup


def aliexpress_spider (max_pages):
    page = 1
    while page <= max_pages:
        url = "https://www.aliexpress.com/af/ps4.html?trafficChannel=af&d=y&CatId=0&SearchText=ps4&ltype=affiliate&SortType=default&page=" + str(page)
        sourcecode = requests.get(url)
        plaintext = sourcecode.text
        soup = BeautifulSoup(plaintext)
        for link in soup.findAll('a' , {'class' : 'item-title'}):
            href = "https://www.aliexpress.com" + link.get("href")
            title  = link.string
            print(href)
            print(title)
        page += 1


aliexpress_spider(1)

错误按摩：

  GuessedAtParserWarning: No parser was explicitly specified, so I'm using the best available HTML parser for this system ("html.parser"). This usually isn't a problem, but if you run this code on another system, or in a different virtual environment, it may use a different parser and behave differently.

The code that caused this warning is on line 11 of the file C:/Users/moham/PycharmProjects/moh/test.py. To get rid of this warning, pass the additional argument 'features="html.parser"' to the BeautifulSoup constructor.

  soup = BeautifulSoup(plaintext)

标签： pythonbeautifulsouppython-requestsweb-crawler

Answer 1

import requests
from bs4 import BeautifulSoup


def aliexpress_spider (max_pages):
    page = 1
    while page <= max_pages:
        url = "https://www.aliexpress.com/af/ps4.html?trafficChannel=af&d=y&CatId=0&SearchText=ps4&ltype=affiliate&SortType=default&page=" + str(page)
        sourcecode = requests.get(url)
        
        soup = BeautifulSoup(sourcecode.text ,"html.parser")
        for link in soup.findAll('a' , {'class' : 'item-title'}):
            href = "https://www.aliexpress.com" + link.get("href")
            title  = link.string
            print(href)
            print(title)
        print(soup.title)
        page += 1



aliexpress_spider(1)