Support

Thank you for the very precise report — you are absolutely right, and this is a bug on our side.

The .htaccess Maker was wrapping each user agent pattern in double quotes, e.g. RewriteCond %{HTTP_USER_AGENT} "ahrefs" [NC]. Inside a mod_rewrite RewriteCond those quotes are not stripped by Apache; they become literal characters in the regular expression, so the pattern effectively looks for a user agent that literally contains "ahrefs" (quotes included) and therefore never matches. That is exactly why removing the quotes made your rule work. This affected the feature as a whole, not just AhrefsBot.

We have fixed this. The patterns are now written without the quotes (with spaces inside multi-word agents handled correctly, so entries like "mass downloader" keep working), and we have verified the corrected rules against Apache: the blocked bots now get a 403 while normal browser traffic is unaffected.

The fix is available right now in our latest developer release. You can download it (Admin Tools Professional, version 7.8.10) from the top of this page:

https://www.akeeba.com/download/developer-releases.html

Please note this is a pre-release build meant for evaluating the fix. After you install it, you need to re-open the .htaccess Maker and click "Save and create .htaccess" once more — installing the update does not rewrite your existing .htaccess file on its own. Once you have regenerated it, you can remove the manual rule you added at the top of the file (leaving it in place is harmless too).

One more thing regarding SemrushBot specifically: ahrefs is part of the default block list, so AhrefsBot is covered once the fix is applied, but semrush was not in the default list. We have now added it to the defaults for future installations, however your existing saved list will not be updated automatically. Please add semrush to your "User agents to block" list so SemrushBot is caught as well.

Finally, a quick note to set expectations: blocking by user agent works well against well-behaved bots such as Ahrefs and Semrush, which send an honest, identifiable user agent string. It is not a security measure against malicious clients, since those can freely spoof or omit their user agent — but for keeping bandwidth-hogging SEO crawlers off your site, it is exactly the right tool.