python - unable to requests.get() a website, 'Remote end closed connection without response'

Question

when I try to send a request to this website:

import requests
requests.get('https://www.ldoceonline.com/')

An exception returned

requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))

The weird part is, if you access the website through normal approach(via a browser), they are fully functional and respond very well. Only when you try to retrieve information via web-scraping technique do you encounter this response.

Any idea on how to successfully scraping it?

标签： pythonweb-scraping

Answer 1

尝试使用标头来获得所需的响应。

import requests

res = requests.get('https://www.ldoceonline.com/',headers={"User-Agent":"Mozilla/5.0"})
print(res.status_code)

输出：

200