时间戳

首页 > 解决方案 > 使用 BeautifulSoup 3 将数据从 json 格式提取到 csv 的问题

python-3.x - 使用 BeautifulSoup 3 将数据从 json 格式提取到 csv 的问题

问题描述

我正在尝试以 CSV 格式从 json 格式导出数据，但没有得到任何结果。下面是代码

import requests
from bs4 import BeautifulSoup
import json
import re

url = "https://www.daraz.pk/catalog/?q=dell&_keyori=ss&from=input&spm=a2a0e.home.search.go.35e34937qjElRf"



page = requests.get(url)

print(page.status_code)
print(page.text)
soup = BeautifulSoup(page.text, 'html.parser')
print(soup.prettify())

alpha = soup.find_all('script',{'type':'application/ld+json'})
jsonObj =`json.loads(alpha[1].text)`

for item in jsonObj['itemListElement']:
    name = item['name']
    price = item['offers']['price']
    currency = item['offers']['priceCurrency']
    availability = item['offers']['availability'].split('/')[-1]
    availability = [s for s in re.split("([A-Z][^A-Z]*)", availability) if s]
    availability = ' '.join(availability)

    print('Availability: %s  Price: %0.2f %s   Name: %s' %(availability,float(price), currency,name))

这是我尝试以 CSV 格式导出数据但未以 CSV 格式获取结果的代码

创建要写入的文件，添加标题行

outfile = open('products.csv','w', newline='')
writer = csv.writer(outfile)
writer.writerow(["name", "offers", "price", "priceCurrency", "availability" ])
outfile.close()
alpha = soup.find_all('script',{'type':'application/ld+json'})

jsonObj = json.loads(alpha[1].text)

for item in jsonObj['itemListElement']:
    name = item['name']
    price = item['offers']['price']
    currency = item['offers']['priceCurrency']
    availability = item['offers']['availability'].split('/')[-1]
    availability = [s for s in re.split("([A-Z][^A-Z]*)", availability) if s]
    availability = ' '.join(availability)

标签： python-3.xweb-scrapingbeautifulsoupexport-to-csv

解决方案

你没有得到结果，因为没有在循环中写入 CSV

outfile = open('products.csv','w', newline='')
writer = csv.writer(outfile)
writer.writerow(["name", "type", "price", "priceCurrency", "availability" ])

alpha = soup.find_all('script',{'type':'application/ld+json'})

jsonObj = json.loads(alpha[1].text)

for item in jsonObj['itemListElement']:
    name = item['name']
    type = item['@type']
    price = item['offers']['price']
    currency = item['offers']['priceCurrency']
    availability = item['offers']['availability'].split('/')[-1]
    # forgot this?
    writer.writerow([name, type, price, currency, availability ])

# and close the CSV here
outfile.close()

推荐阅读