时间戳

首页 > 解决方案 > pyppeteer.errors.TimeoutError: Navigation Timeout Exceeded: 超过 8000 ms

python - pyppeteer.errors.TimeoutError: Navigation Timeout Exceeded: 超过 8000 ms

问题描述

运行以下代码时出现错误。这是我用于网络抓取的。

from requests_html import HTMLSession

url = 'https://www.beerwulf.com/en-gb/c/beers?segment=Beers&catalogCode=Beer_1'

s = HTMLSession()
r = s.get(url)

r.html.render(sleep=1) 
print(r.status_code)

我收到以下错误

Traceback (most recent call last):
File "c:/Python/Python385/web_scraping/task1.py", line 8, in <module>
  r.html.render(sleep=1) 
File "C:\Python\Python385\lib\site-packages\requests_html.py", line 598, in render
  content, result, page = self.session.loop.run_until_complete(self._async_render(url=self.url, 
  script=script, sleep=sleep, wait=wait, content=self.html, reload=reload, scrolldown=scrolldown, 
  timeout=timeout, keep_page=keep_page))  
File "C:\Python\Python385\lib\asyncio\base_events.py", line 616, in run_until_complete
  return future.result()
File "C:\Python\Python385\lib\site-packages\requests_html.py", line 512, in _async_render
  await page.goto(url, options={'timeout': int(timeout * 1000)})
File "C:\Python\Python385\lib\site-packages\pyppeteer\page.py", line 885, in goto
  raise error
pyppeteer.errors.TimeoutError: Navigation Timeout Exceeded: 8000 ms exceeded.

标签： pythonweb-scraping

解决方案

默认超时为r.html.render()8 秒。如果在 8 秒内未加载所有 JavaScript 代码，则会出现此错误。您可以通过更改默认超时（例如 20 秒）来解决此问题。使用以下代码：

r.html.render(timeout=20)

推荐阅读