Support

 I get a block from Admin Tools on Botsimulator and Google Bot (Fetch) on petrakersten.com.

How to circumvent this?

Please check .htaccess Maker, in the list of user agents to block to make sure your bots are not included. If they are, you need to delete them from the list and regenerate your .htaccess file. Note that some host cache the .htaccess files so it may take a while for your changes to be effective.

I disabled Admintools and all works just fine. So how can I resolve this?

This is htaccess no blocks!

If they are blocked by WAF there should be a record in the Security Exceptions Log that tells why they are blocked. Please post the reason here and we can figure out how to create the exception to allow they to work.

I did check that there are no entries in the Security Log for Googlebot.

WAF wouldn't block a bot because it's a bot, that only happens in .htaccess. It would be an IP address blocked because of some suspicious behavior. You can trigger your 403 error then immediately check the log. If the log is sorted by date, it should be the top entry.

Thanks for the hint on the log. I imagine the GEO blocked it!!! Thank you, sorry for the trouble

You're welcome!

But how come i did Geo block and it allows .googlebot.com? Does not that override?

It can be blocked by either .htaccess or WAF. But it has to be allowed by both in order to work. If something is allowed in one it does not override a block by the other.

So how can I do both, as some of my clients do not have a market in the US? So googlebot is save in WAF but in htaccess i have nothing for Googlebot to be ok, despite Geoblock?

If you GeoIP block the USA, you will block the googlebot. There is no way around that.

The GeoIP blocking is really not all that effective. It is trivial for a human attacker to use a different compromised computer, log in through a proxy server, etc.  The block will stop most scripts but that is about it. In addition, we are using the free version of MindMax's database, which is only about 90% accurate. Their paid version is better, but not cheap.

This ticket has been automatically closed. All tickets which have been inactive for a long time are automatically closed. If you believe that this ticket was closed in error, please contact us.