php - What can lock google bot out

Question

Google bot crawls our website weird, some pages are blocked and others are not by robots.txt If you use the google mobile friendly test, then css and js files and images are randomly blocked with "other reason". Sometimes you have to retry the page self in the test.

The user doesn't see anything when visiting our site, I think only when you are a bot or use curl. And if it happens it doesn't always happen.

The site is made of a WordPress installation and some lonely PHP files and a coppermine gallery. But all pages have problems, not always but randomly I guess.

• Example lonely page: https://www.woefkesranch.be/puppies-te-koop/shiba-inu-puppies-te-koop.php
• Example wordpress page: https://www.woefkesranch.be/hondenrassen/
• Example coppermine gallery: https://www.woefkesranch.be/gallery/index.php?cat=4

What could be the reason?

Edit: my PHP version is 5.6

标签： phpcurlwebservergooglebot