首页 > 解决方案 > 有没有办法实时显示不断变化的数据框?

问题描述

我正在从https://www.bseindia.com/markets/PublicIssues/BSEBidDetails_ofs.aspx?flag=NR&Scripcode=541729抓取数据。数据每 10 秒更改一次。所以我为脚本运行做了一个计时器。在每次迭代开始时,它会重新初始化 pandas 数据帧并抓取网站并将数据添加到数据帧中。有没有办法可以在静态窗口中显示这些数据?

from urllib.request import Request, urlopen
from bs4 import BeautifulSoup as soup
import pandas as pd
import time
url = 'https://www.bseindia.com/markets/PublicIssues/BSEBidDetails_ofs.aspx?flag=NRScripcode=541729'
req = Request(url , headers={'User-Agent': 'Mozilla/5.0'})
bseobj=pd.DataFrame(columns=['Price Interval','qNumber of bids','qConfirmed','qYet to be confirmed','qTotal','Confirmed','Yet to be confirmed','Total'])
while True:
   i=0
   r=[]
   lr=[]    
   webpage = urlopen(req).read()
   page_soup = soup(webpage, "html.parser")
   table=page_soup.find_all("table")

   for row in table[1]:
       for column in row.find_all("td",{"class":"body_text_table"},{"bgcolor":"#FFFFFF"}):
            val=column.get_text()
            val=val.replace(',','')
            chk=val.replace('.','')
            if chk.isdigit():
                r.append(float(val))
            elif val=="":
                r.append(0)
            else:
                r.append(val)
            
        lr.append(r)
        r=[]
    for i in range(2,len(lr)-2):
        bseobj=bseobj.append({'Price Interval':lr[i][0],'qNumber of bids':lr[i][1],'qConfirmed':lr[i][2],'qYet to be confirmed':lr[i][3],'qTotal':lr[i][4],'Confirmed':lr[i][5],'Yet to be confirmed':lr[i][6],'Total':lr[i][7]},ignore_index=True)
    print(bseobj)
    bseobj=bseobj.iloc[0:0]
    time.sleep(10)

标签: python-3.xpandasuser-interfaceweb-scraping

解决方案


推荐阅读