时间戳

首页 > 解决方案 > 谷歌机器人不断发送无效网址请求

wordpress - 谷歌机器人不断发送无效网址请求

问题描述

我最近注意到 AWS 上记录了一些不寻常的流量进入我的 Bitnami WordPress 网站。我检查了 Apache 访问日志，然后注意到，Google Bot 正在向无效 URL 发送请求。这些 URL 或图像甚至不是我的站点或站点地图的一部分。

bitnami@ip-xx-xx-xx-x:~$ tail -f /opt/bitnami/apache2/logs/access_log
157.55.39.4 - - [12/Jun/2021:16:14:01 +0000] "GET /14965bGbiiniimebuHe497756 HTTP/1.1" 404 137190
207.46.13.161 - - [12/Jun/2021:16:14:01 +0000] "GET /1995WVsutoraipuof600785 HTTP/1.1" 404 137189
157.55.39.4 - - [12/Jun/2021:16:14:04 +0000] "GET /21126EIyutakameikuci970336 HTTP/1.1" 404 137190
54.36.148.222 - - [12/Jun/2021:16:14:08 +0000] "GET /product-category/mens/?per_page=15&filter_colour=black%2Cspecial&orderby=date HTTP/1.1" 200 29307
66.249.66.159 - - [12/Jun/2021:16:14:11 +0000] "GET /202112135 HTTP/1.1" 302 210
54.36.149.46 - - [12/Jun/2021:16:14:19 +0000] "GET /product-category/mens/?per_page=15&filter_colour=black%2Cspecial&orderby=popularity HTTP/1.1" 200 29308
66.249.66.157 - - [12/Jun/2021:16:14:29 +0000] "GET /pk5820/ HTTP/1.1" 302 208
54.36.148.205 - - [12/Jun/2021:16:14:30 +0000] "GET /product-category/mens/?per_page=15&filter_colour=black%2Cspecial&orderby=price HTTP/1.1" 200 29308
66.249.66.195 - - [12/Jun/2021:16:14:33 +0000] "GET /Shin-Guards/822575-Exceptional-Flexible-Soft-Light-Weight-for-Adults-SHINESTONE-Football/img/news2016-1.jpg HTTP/1.1" 302 308
54.36.148.205 - - [12/Jun/2021:16:14:42 +0000] "GET /product-category/mens/?per_page=15&filter_colour=black%2Cspecial&orderby=price-desc HTTP/1.1" 200 29313
54.36.149.106 - - [12/Jun/2021:16:14:46 +0000] "GET /Engine-Mounts/127269-Mitsubishi-Outlander-AWD-4WD-20032006-Replace-MR961407-Unlimited-Rider-Left-Rear-Differential/ HTTP/1.1" 302 316
66.249.66.157 - - [12/Jun/2021:16:14:47 +0000] "GET /bv13551/KoRd385030 HTTP/1.1" 302 219
13.66.139.0 - - [12/Jun/2021:16:14:49 +0000] "GET /12547zuboreeXh964662 HTTP/1.1" 404 137190
54.36.148.163 - - [12/Jun/2021:16:14:49 +0000] "GET /dial-color/black/page/9/?add-to-cart=13962 HTTP/1.1" 302 243

bitnami@ip-xx-xx-xx-x:~$:~$ host 66.249.66.157
157.66.249.66.in-addr.arpa domain name pointer crawl-66-249-66-157.googlebot.com.
bitnami@ip-xx-xx-xx-x:~$:~$

我想知道造成这种情况的根本原因是什么，如果我能就如何解决这个问题获得一些建议，那将非常有帮助。

标签： wordpressamazon-ec2googlebot

解决方案

推荐阅读