python - WebDriverException:未知错误:Chrome 无法启动:使用 make docker-run 命令异常退出
问题描述
目标:
使用此页面作为指南https://robertorocha.info/setting-up-a-selenium-web-scraper-on-aws-lambda-with-python/构建 Docker 容器以将 selenium web scraper 上传到 AWS Lambda 。
make docker-run
当前使用输出以下错误的命令测试本地 Lambda 环境:
{"errorType":"WebDriverException","errorMessage":"Message: unknown error:
Chrome failed to start: exited abnormally\n (chrome not reachable)\n (The
process started from chrome location /var/task/bin/headless-chromium is no
longer running, so ChromeDriver is assuming that Chrome has
crashed.)\n","stackTrace":[" File \"/var/lang/lib/python3.7/imp.py\", line
234, in load_module\n return load_source(name, filename, file)\n","
File \"/var/lang/lib/python3.7/imp.py\", line 171, in load_source\n
module = _load(spec)\n"," File \"\u003cfrozen
importlib._bootstrap\u003e\", line 696, in _load\n"," File \"\u003cfrozen
importlib._bootstrap\u003e\", line 677, in _load_unlocked\n"," File
\"\u003cfrozen importlib._bootstrap_external\u003e\", line 728, in
exec_module\n"," File \"\u003cfrozen importlib._bootstrap\u003e\", line
219, in _call_with_frames_removed\n"," File
\"/var/task/src/lambda_function.py\", line 150, in \u003cmodule\u003e\n
lambda_handler()\n"," File \"/var/task/src/lambda_function.py\", line 33,
in lambda_handler\n driver =
webdriver.Chrome(executable_path=os.getcwd() + '/bin/chromedriver',
chrome_options=chrome_options)\n"," File
\"/var/task/lib/selenium/webdriver/chrome/webdriver.py\", line 81, in
__init__\n desired_capabilities=desired_capabilities)\n"," File
\"/var/task/lib/selenium/webdriver/remote/webdriver.py\", line 157, in
__init__\n self.start_session(capabilities, browser_profile)\n"," File
\"/var/task/lib/selenium/webdriver/remote/webdriver.py\", line 252, in
start_session\n response = self.execute(Command.NEW_SESSION,
parameters)\n"," File \"/var/task/lib/selenium/webdriver/remote/webdriver.py\", line 321, in
execute\n self.error_handler.check_response(response)\n"," File
\"/var/task/lib/selenium/webdriver/remote/errorhandler.py\", line 242, in
check_response\n raise exception_class(message, screen, stacktrace)\n"]}
版本:
铬 79.0.3945.88
无头铬 1.0.0-55
蟒蛇 3.7.1
来自requirements.txt:
boto3==1.9.24
botocore==1.12.24
硒==3.141
chromedriver-binary==79.0.3945.36 ( https://chromedriver.storage.googleapis.com/79.0.3945.36/chromedriver_linux64.zip )
美丽的汤4==4.6.3
刮板脚本中的 Chromedriver 选项:
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--headless')
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--disable-gpu')
chrome_options.add_argument('--window-size=1280x1696')
chrome_options.add_argument('--user-data-dir=/tmp/user-data')
chrome_options.add_argument('--hide-scrollbars')
chrome_options.add_argument('--enable-logging')
chrome_options.add_argument('--log-level=0')
chrome_options.add_argument('--v=99')
chrome_options.add_argument('--single-process')
chrome_options.add_argument('--data-path=/tmp/data-path')
chrome_options.add_argument('--ignore-certificate-errors')
chrome_options.add_argument('--homedir=/tmp')
chrome_options.add_argument('--disk-cache-dir=/tmp/cache-dir')
chrome_options.add_argument('user-agent=Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36')
chrome_options.binary_location = os.getcwd() + '/bin/headless-chromium'
driver = webdriver.Chrome(executable_path=os.getcwd() + '/bin/chromedriver', chrome_options=chrome_options)
相关 Makefile 代码:
# Get chromedriver
curl -SL https://chromedriver.storage.googleapis.com/79.0.3945.36/chromedriver_linux64.zip > chromedriver.zip
unzip chromedriver.zip -d bin/
# Get Headless-chrome
curl -SL https://github.com/adieuadieu/serverless-chrome/releases/download/v1.0.0-55/stable-headless-chromium-amazonlinux-2017-03.zip > headless-chromium.zip
unzip headless-chromium.zip -d bin/
相关 docker-compose.yml 代码(环境变量):
# scraper path
PYTHONPATH=/var/task/src:/var/task/lib
# chromedriver and headless-chromium path
PATH=/var/task/bin
尝试:
降级 chrome 驱动程序 (v2.43) 和 selenium (3.14)
降级的 chrome 驱动程序 (v2.41) 和 selenium (3.141)
解决方案
推荐阅读
- c# - 从另一个表单访问按钮
- entity-framework-core - 包 Microsoft.EntityFrameworkCore.Sqlite 5.0.0-rc.1.20451.13 不兼容 net50 (.NETFramework,Version=v5.0) / win7-x86
- javascript - Jquery & Ajax -> 无法上传文件
- reactjs - 如何在集成测试中防止未安装组件的 React 状态更新?
- mysql - 您可以在 MySQLl 数据库中使用数据类型:varchar() 吗?
- java - 如何创建实时检查代码质量的 Intellij idea maven/gradle 插件?
- python - 使用opencv查找图像中的所有轮廓
- javascript - TypeScript 和 typescript-eslint 有重叠的目的吗?
- sql - SQL Sever 2016 - 行为不一致 - “JSON_VALUE 或 JSON_QUERY”的参数 2 必须是字符串文字
- laravel - 在 Laravel 中使用 HyN Multitenant 重置密码不起作用