python - 无法同时抓取字符串和列表

Question

我正在尝试使用 python 脚本从网页获取name,address和。key contacts我可以以正确的方式单独获取它们。但是，我想要做的是获取name和address作为字符串和列表中的关键联系人，以便我可以将它们写入 6 列的 csv 文件中。我找不到任何方法将 的值包含data-cfemail在联系人列表中。

网站地址

我试过：

import requests
from bs4 import BeautifulSoup

link = "https://www.fis.com/fis/companies/details.asp?l=e&filterby=species&specie_id=615&page=1&company_id=160574&country_id="

res = requests.get(link,headers={"User-Agent":"Mozilla/5.0"})
soup = BeautifulSoup(res.text,'lxml')
name = soup.select_one("#name").text.strip()
address = soup.select("#description_details tr:contains('Address:') td")[1].text
contacts = [' '.join(item.get_text(strip=True).split()) for item in soup.select("#contacts table tr td")]
print(name,address,contacts)

电流输出：

Bahia Grande S.A. - BG Group 
Maipú 1252 Piso 8°
 ['Founder & PresidentMr Guillermo Jacob', 'VP FinanceMr Andres Jacob[email protected]', 'ControllerMr Juan Carlos Peralta[email protected]', 'VP AdmnistrationMs Veronica Vinuela[email protected]', '']

预期输出（因为电子邮件受到保护，所以值data-cfemail会做）：

Bahia Grande S.A. - BG Group 
Maipú 1252 Piso 8°

[Founder & President, Mr Guillermo Jacob]
[VP Finance, Mr Andres Jacob,bbdad1dad8d4d9fbd9dad3d2dadcc9dad5dfde95d8d4d695dac9]
[Controller,Mr Juan Carlos Peralta,0b61687b6e796a677f6a4b696a63626a6c796a656f6e25686466256a79]
[VP Admnistration,Ms Veronica Vinuela,87f1f1eee9f2e2ebe6c7e5e6efeee6e0f5e6e9e3e2a9e4e8eaa9e6f5]

标签： pythonpython-3.xweb-scraping

Answer 1

您可以通过以下方式限制到适当的 tds #contacts td[height]，然后是适当的 ids td.select('#contacts_title, #contacts_name, #contacts_email')，然后在列表理解中进行测试，如果 current 具有 cfemail 并相应地采取行动。

from bs4 import BeautifulSoup as bs
import requests

r = requests.get('https://www.fis.com/fis/companies/details.asp?l=e&filterby=species&specie_id=615&page=1&company_id=160574&country_id=')
soup = bs(r.content, 'lxml')
name = soup.select_one('#name').text.strip()
address = soup.select_one('#description_details td:contains("Address:") + td div').text

print(name)
print(address)

for td in soup.select('#contacts td[height]'):
    print([i.text.strip().replace('\xa0',' ') if i.select_one('.__cf_email__') is None else i.select_one('.__cf_email__')['data-cfemail']
          for i in td.select('#contacts_title, #contacts_name, #contacts_email')])

---

OP的实现：

contacts = [', '.join([i.text.strip().replace('\xa0',' ') if i.select_one('.__cf_email__') is None else i.select_one('.__cf_email__')['data-cfemail'] for i in td.select('#contacts_title, #contacts_name, #contacts_email')]) for td in soup.select('#contacts td[height]')]