python - 如何从 Python 中的网络选项卡中读取？

Question

使用 Selenium、Requests 和 Beautiful Soup，我希望能够找到并打印此 .m3u8 链接（或此处显示的任何链接），但我不知道在 Python 中显示请求。

def locator(url):
    driver = sp.driver # just geckodriver with profile
    driver.get(url)
    sleep(4)
    # from here needs to somehow access the network tab & locate GET requests with Host == "cfvod.kaltura.com"

标签： pythonnetworkingweb-scrapingfirefox-developer-tools

Answer 1

不得不使用 ChromeDriver、PyChrome 和 DevTools 协议，但这有效：

def outputstart(**kwargs):
    print("START ", kwargs)

driver = sp.driver # my chromedriver profile with an argument added for port 8000

dev_tools = pychrome.Browser(url="http://localhost:8000")
tab = dev_tools.list_tab()[0]
tab.start()

url = 'https://google.com'

start = time.time()
driver.get(url)
tab.call_method("Network.emulateNetworkConditions",
            offline=False,
            latency=100,
            downloadThroughput=93750,
            uploadThroughput=31250,
            connectionType="wifi")

def outputstart(**kwargs):
    print("START ", kwargs)

tab.call_method("Network.enable", _timeout=20)
tab.set_listener("Network.requestWillBeSent", outputstart)