时间戳

首页 > 解决方案 > 在 Python CSV Writer 循环中写入标题一次

python - 在 Python CSV Writer 循环中写入标题一次

问题描述

下面是一个爬虫，它遍历两个网站，爬取一个团队的花名册信息，将信息放入一个数组中，然后将数组导出到一个 CSV 文件中。一切都很好，但唯一的问题是每次刮板移动到第二个网站时，csv 文件中的 writerow 标题都会重复。是否可以调整代码的 CSV 部分，以便当刮板循环通过多个网站时，标题只出现一次？提前致谢！

import requests
import csv
from bs4 import BeautifulSoup

team_list={'yankees','redsox'}

for team in team_list:
    page = requests.get('http://m.{}.mlb.com/roster/'.format(team))
    soup = BeautifulSoup(page.text, 'html.parser')

    soup.find(class_='nav-tabset-container').decompose()
    soup.find(class_='column secondary span-5 right').decompose()

    roster = soup.find(class_='layout layout-roster')
    names = [n.contents[0] for n in roster.find_all('a')]
    ids = [n['href'].split('/')[2] for n in roster.find_all('a')]
    number = [n.contents[0] for n in roster.find_all('td', index='0')]
    handedness = [n.contents[0] for n in roster.find_all('td', index='3')]
    height = [n.contents[0] for n in roster.find_all('td', index='4')]
    weight = [n.contents[0] for n in roster.find_all('td', index='5')]
    DOB = [n.contents[0] for n in roster.find_all('td', index='6')]
    team = [soup.find('meta',property='og:site_name')['content']] * len(names)

    with open('MLB_Active_Roster.csv', 'a', newline='') as fp:
        f = csv.writer(fp)
        f.writerow(['Name','ID','Number','Hand','Height','Weight','DOB','Team'])
        f.writerows(zip(names, ids, number, handedness, height, weight, DOB, team))

标签： pythoncsvweb-scrapingbeautifulsoup

解决方案

使用变量检查是否添加了标头可能会有所帮助。如果添加了标题，它将不会添加第二次

header_added = False
for team in team_list:
    do_some stuff

    with open('MLB_Active_Roster.csv', 'a', newline='') as fp:
        f = csv.writer(fp)
        if not header_added:
            f.writerow(['Name','ID','Number','Hand','Height','Weight','DOB','Team'])
            header_added = True
        f.writerows(zip(names, ids, number, handedness, height, weight, DOB, team))

推荐阅读