时间戳

首页 > 解决方案 > 如何递归地抓取网页以检查python中是否有新的pdf文件？

python - 如何递归地抓取网页以检查python中是否有新的pdf文件？

问题描述

有一个网站每个月都会发布一份 pdf 报告。我想每小时监控一次，并将新的 pdf 通过电子邮件发送到我的电子邮件，每当新的 pdf 上传时我都会收到。我想为此使用python。我也熟悉美丽的汤和scrapy，但我不知道如何检查新的pdf文件并且只获取新的pdf文件。

标签： pythonscreen-scraping

解决方案

import requests
from time import strftime, sleep

# Here I Will Get The Current Month And Year And Assign It To Variables.

month = strftime("%B").lower()
year = strftime("%Y")

# I've noticed that the website publishing the file by the Month name in lower-case and year.
# Now we will loop on the url each one hour and download the file once released and exit.
# You Have to set cron-job to run every month.


while True:
    r = requests.get(
        f"http://www.pbs.gov.pk/sites/default/files//price_statistics/monthly_price_indices/nb/2019/cpi_review_nb_{month}_{year}.pdf")
    if r.status_code == 200:
        with open(f"{month}.pdf", 'wb') as f:
            f.write(r.content)
            print(f"File Already Saved As {month}.pdf")
            break
    else:
        print("File Not Raised Yet, We Will Check Back After One Hour.")
        sleep(3600)
        continue

# you can also put the current href links in a file.
# then loop over source and check if new href is released so download and append the href to file.
# but if the href in file so keep checking each hour.

推荐阅读