时间戳

首页 > 解决方案 > BeautifulSoup 找不到给定类的列表元素

python - BeautifulSoup 找不到给定类的列表元素

问题描述

我正在尝试访问以下网站的成分列表中的元素：https ://www.jamieoliver.com/recipes/pasta-recipes/gennaro-s-classic-spaghetti-carbonara/

<div class="col-md-12 ingredient-wrapper">
  <ul class="ingred-list ">                                                                                                                                           
    <li> 
      3 large free-range egg yolks                                                                                                                                                                                                                                                                                                           
    </li>                                                                                                                                          
    <li> 
      40 g Parmesan cheese, plus extra to serve                                                                                                                                                                                                                                                                                                                                       
    </li>                                                                                                                                          
    <li> 
      1 x 150 g piece of higher-welfare pancetta                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    </li>                                                                                                                                          
    <li> 
      200g dried spaghetti                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   
    </li>                                                                                                                                           
    <li> 
      1 clove of garlic                                                                                                                                                                                                                                                                                                                                                                                                                           
    </li>                                                                                                                                         
    <li> 
      extra virgin olive oil                                                                                                                                                                                                                                                                                                                                                                    
    </li>                                                                                  
  </ul>
</div

我第一次尝试只使用请求和漂亮的汤，但我的代码没有找到列表元素。然后我尝试使用 Selenium，但它仍然没有用。我的代码如下：

from selenium import webdriver
from bs4 import BeautifulSoup
import pandas as pd 

url = "https://www.jamieoliver.com/recipes/pasta-recipes/cracker-ravioli/"

driver = webdriver.Chrome()
driver.get(url)
html = driver.page_source

soup = BeautifulSoup(html, 'html.parser')

for ultag in soup.findAll('div', {'class': "col-md-12 ingredient-wrapper"}):
# for ultag in soup.findAll('ul', {'class': 'ingred_list '}):
    for litag in ultag.findALL('li'):
        print(litag.text)

标签： pythonseleniumweb-scrapingbeautifulsoup

解决方案

要获取成分列表，您可以使用以下示例：

import requests
from bs4 import BeautifulSoup


url = 'https://www.jamieoliver.com/recipes/pasta-recipes/gennaro-s-classic-spaghetti-carbonara/'
headers = {'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:80.0) Gecko/20100101 Firefox/80.0'}
soup = BeautifulSoup(requests.get(url, headers=headers).content, 'html.parser')

for li in soup.select('.ingred-list li'):
    print(' '.join(li.text.split()))

印刷：

3 large free-range egg yolks
40 g Parmesan cheese , plus extra to serve
1 x 150 g piece of higher-welfare pancetta
200 g dried spaghetti
1 clove of garlic
extra virgin olive oil

推荐阅读