时间戳

首页 > 解决方案 > ElasticBeanstalk worker cron 每分钟运行多次

laravel - ElasticBeanstalk worker cron 每分钟运行多次

问题描述

我有一个运行 Laravel 的 EB 工作环境（单实例），用于运行计划和分派的作业。

我面临的问题是每分钟多次调用 cron，导致计划的作业加倍。这可以在worker的访问日志中看到；

-------------------------------------
/var/log/httpd/access_log
-------------------------------------
127.0.0.1 (-) - - [27/Feb/2019:00:07:01 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:07:01 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:07:01 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:07:01 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:07:01 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:07:07 +0000] "POST /scheduled.php HTTP/1.1" 302 215 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:07:07 +0000] "POST /scheduled.php HTTP/1.1" 302 215 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:07:08 +0000] "POST /scheduled.php HTTP/1.1" 302 215 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:07:09 +0000] "POST /scheduled.php HTTP/1.1" 302 215 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:07:09 +0000] "POST /scheduled.php HTTP/1.1" 302 215 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:07:09 +0000] "POST /scheduled.php HTTP/1.1" 302 215 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:07:10 +0000] "POST /scheduled.php HTTP/1.1" 302 215 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:07:11 +0000] "POST /scheduled.php HTTP/1.1" 302 215 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:07:11 +0000] "POST /scheduled.php HTTP/1.1" 302 215 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:08:00 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:08:00 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:08:00 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:08:00 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:08:00 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:08:00 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:08:00 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:08:00 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:08:00 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:08:00 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:08:00 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:08:00 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:08:00 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:08:15 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:08:15 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:08:15 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:08:15 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:08:15 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:08:15 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:08:15 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:08:15 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:08:15 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:08:15 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:08:15 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:08:15 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:08:15 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:08:15 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:08:16 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:08:16 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:08:16 +0000] "POST /scheduled.php HTTP/1.1" 302 215 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:08:16 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:08:16 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:08:16 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:08:16 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:08:16 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:08:16 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:08:16 +0000] "POST /scheduled.php HTTP/1.1" 302 215 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:08:16 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:08:16 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:08:16 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:08:16 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:08:16 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:08:48 +0000] "POST /scheduled.php HTTP/1.1" 302 215 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:09:00 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:09:00 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:09:00 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:09:00 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:09:00 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:09:00 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:09:00 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:09:00 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:09:00 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"
127.0.0.1 (-) - - [27/Feb/2019:00:09:00 +0000] "POST /worker/schedule HTTP/1.1" 302 217 "-" "aws-sqsd/2.4"

导致我的 CloudWatch 日志中出现类似的重复；

20:40:00  INFO: Processing /2019/20190225.csv [] []
20:40:00  INFO: Processing /2019/20190225.csv [] []
20:40:00  INFO: Processing /2019/20190225.csv [] []
20:40:01  INFO: Processing /2019/20190225.csv [] []
20:40:01  INFO: Processing /2019/20190225.csv [] []
20:40:02  INFO: Processing /2019/20190225.csv [] []
20:40:02  INFO: Processing /2019/20190225.csv [] []
20:40:03  INFO: Processing /2019/20190225.csv [] []
20:40:03  INFO: Processing /2019/20190225.csv [] []
20:40:04  INFO: Processing /2019/20190225.csv [] []
20:40:04  INFO: Processing /2019/20190225.csv [] []
20:40:06  INFO: Processing /2019/20190225.csv [] []
20:40:06  INFO: Processing /2019/20190225.csv [] []
20:40:06  INFO: Processing /2019/20190225.csv [] []
20:40:06  INFO: Processing /2019/20190225.csv [] []
20:40:06  INFO: Processing /2019/20190225.csv [] []
20:40:06  INFO: Processing /2019/20190225.csv [] []

我正在使用dusterio/laravel-aws-worker与SQS 队列交互以处理分派的作业。

工人有以下情况cron.yaml；

version: 1
cron:
    - name: "schedule"
      url: "/worker/schedule"
      schedule: "* * * * *"

我的工作人员的控制台内核正在安排这样的每小时工作；

protected function schedule(Schedule $schedule)
{
    $schedule->job(new ImportData())
        ->hourly()
        ->timezone(config('app.timezone')); // Pacific/Auckland
}

我在这里缺少什么吗？

标签： laravelamazon-web-servicesamazon-sqsamazon-elastic-beanstalk

解决方案

对于那些偶然发现这个问题的人来说，问题是我的应用程序上发生了 HTTPS 重定向，并且同一个应用程序正在部署到 Web 层和工作层。302 响应会导致 SQS 重试到您的最大重试限制，但每个调用仍在执行作业。

一种解决方案是从该重定向中排除 SQS 代理；

files:
    /etc/httpd/conf.d/http-redirect.conf:
        mode: "000644"
        owner: root
        group: root
        content: |
            RewriteEngine On
            RewriteCond %{HTTP:X-Forwarded-Proto} !https
            RewriteCond %{HTTP_USER_AGENT} !ELB-HealthChecker
            RewriteCond %{HTTP_USER_AGENT} !aws-sqsd
            RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}

另一种方法是防止将此脚本部署到工作环境中。

推荐阅读