python - django 'set' 对象没有属性 'items' 错误
问题描述
我正在尝试在 AWS 上部署我的 django 应用程序。我使用 bs4 和请求在我的应用程序中创建了一个网络爬虫。我用它来从电子商务网站获取数据。它在亚马逊上完美运行,但是当我尝试从 newegg 抓取时它会抛出这个(参见图 1)。但相同的代码适用于本地主机。首先我认为这是用户代理问题,我尝试使用其他用户代理但没有用。
我真的不知道是什么问题,所以如果您知道导致此错误的原因,那么请告诉它会非常有帮助。
下面的代码和图像:
headers = {"User-agent":"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.132 Safari/537.36"}
url = "https://www.newegg.com/p/pl?d="+query
data = re.get(url,headers=headers)
soup = bs4.BeautifulSoup(data.content, 'lxml')
items_container = soup.find("div", {"class":"list-wrap"})
items = items_container.find("div", {"class":"items-view"})
prdList = []
for item in items.find_all("div", {"class":"item-container"}):
header = item.find("a", {"class":"item-title"})
title = header.text
link = header["href"]
img = item.find("a", {"class":"item-img"})
imgsrc = img.find("img")["src"]
price = item.find("li", {"class":"price-current"})
if price == None:
price = '$--'
else:
price = price.find("strong").text
prdList.append(Scraper.prdTmplt(imgsrc, title, price, link, site))
allProduct = prdList
return allProduct
错误:
Environment:
Request Method: GET
Request URL: http://ec2-54-179-183-56.ap-southeast-1.compute.amazonaws.com:8000/search?sq=gaming+laptop&site_selected=newegg&sq-submit=Search
Django Version: 2.2.6
Python Version: 3.6.8
Installed Applications:
['groups.apps.GroupsConfig',
'product_details.apps.ProductDetailsConfig',
'scraper.apps.ScraperConfig',
'pages.apps.PagesConfig',
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles']
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware']
Traceback:
File "/home/ubuntu/django/lib/python3.6/site-packages/django/core/handlers/exception.py" in inner
34. response = get_response(request)
File "/home/ubuntu/django/lib/python3.6/site-packages/django/core/handlers/base.py" in _get_response
115. response = self.process_exception_by_middleware(e, request)
File "/home/ubuntu/django/lib/python3.6/site-packages/django/core/handlers/base.py" in _get_response
113. response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/home/ubuntu/best-price-py/pages/views.py" in search
29. context['bb'] = Scraper.neweggList(search_query, site)
File "/home/ubuntu/best-price-py/scraper/scrapers.py" in neweggList
120. data = re.get(url)
File "/home/ubuntu/django/lib/python3.6/site-packages/requests/api.py" in get
75. return request('get', url, params=params, **kwargs)
File "/home/ubuntu/django/lib/python3.6/site-packages/requests/api.py" in request
60. return session.request(method=method, url=url, **kwargs)
File "/home/ubuntu/django/lib/python3.6/site-packages/requests/sessions.py" in request
519. prep = self.prepare_request(req)
File "/home/ubuntu/django/lib/python3.6/site-packages/requests/sessions.py" in prepare_request
462. hooks=merge_hooks(request.hooks, self.hooks),
File "/home/ubuntu/django/lib/python3.6/site-packages/requests/models.py" in prepare
314. self.prepare_headers(headers)
File "/home/ubuntu/django/lib/python3.6/site-packages/requests/models.py" in prepare_headers
446. for header in headers.items():
Exception Type: AttributeError at /search
Exception Value: 'set' object has no attribute 'items'
解决方案
您传递的标题字典被呈现为一组而不是字典。查看图像中的转储。
推荐阅读
- linux - Linux 变量未在整个循环中重置的问题
- sql - VBA - 使用递归从 JSON 创建 MSAccess 关系表
- r - 如何在 Shiny 应用程序中判断导航栏是否折叠?
- powerbi - Power BI - 从 1..* 关联中获取第一行和第二行
- python - 对于每个 Pandas 组,根据与另一个数据框匹配的列删除行
- file - 在 svelte 中过滤文件
- bioinformatics - Snakemake 工作流程卡住 - 等待在 SLURM 集群上执行的作业
- flutter - 验证时 Flutter TextForm 字段填充
- performance - 为什么安装库时 conda 比 pip 慢很多?
- json - 为什么 VS Code 允许 .vscode 内的 JSON 文件中的注释,但其他地方却不允许?