python-3.x - 请求模块未收到源代码

Question

我正在使用请求模块向服务器发送请求。作为回应，我从页面源中得到的只是几行而不是整个页面。

from urllib.error import HTTPError
from urllib.error import URLError
from bs4 import BeautifulSoup
import requests
url = "https://www.yellowpages.ae/c/advs/uae/web.html"

header = {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8',
          'Accept-Encoding': 'gzip, deflate, br',
          'Accept-Language': 'en-GB,en;q=0.9,en-US;q=0.8,ml;q=0.7',
          'Cache-Control': 'max-age=0',
          'Connection': 'keep-alive',
          'Host': 'www.yellowpages.com',
          'Upgrade-Insecure-Requests': '1',
          'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.140 Safari/537.36'
          }
try:
    print("Present Link: ", url)
    response = requests.get(url, headers=header)
except HTTPError as e:
    print(e)
except URLError:
    print("URL Error")
else:
    print(response.status_code)
    print("response is:", response.text)

如果我没有在请求中发送标头， 则页面输出只是这个数据，而页面源包含很多文本，请检查它并建议我做错了什么。

Web at Yellowpages.ae
window.onload = markiconsLarge;
}
$create(Sys.Extended.UI.AutoCompleteBehavior, {"completionSetCount":12,"delimiterCharacters":"","id":"advs_autokey","minimumPrefixLength":2,"serviceMethod":"Getkeyword","servicePath":"../../../YpWebService.asmx","useContextKey":true}, null, null, $get("advs_txtkey"));
});

. . . . .

类似的东西无法将整个输出粘贴为代码。因此，这只是阐明我的观点的一部分。

将不胜感激。谢谢

标签： python-3.xpython-requests