Everybody will be able to see its contents. Do not include usernames, passwords or any other sensitive information.
Latest post by on Wednesday, 02 September 2020 17:17 CDT
I have created the following commants in my robots.txt to block the following agents that created enormous harmful trafic on https://rakispilacourisltd.com/. Can I ask why these were not listed in the .htaccess maker bad agents list ?
User-agent: MJ12bot
Disallow: /
User-agent: SemrushBot
Disallow: /
User-agent: DotBot
Disallow: /
User-agent: SemrushBot
Disallow: /
If a bot respects the robots.txt file it's annoying, not malicious. The purpose of the robots.txt file is to tell different robots how to behave on your site.
The purpose of .htaccess Maker's user agent blocking is to defend against bad robots which do not respect the robots.txt file. Their number is practically infinite. We include the top ones but it's not an exhaustive list, it's not meant to be one and it's not practical to come up with one. That's why the list is user-editable.
Nicholas K. Dionysopoulos
Lead Developer and Director
🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!
Working hours: We are open Monday to Friday, 9am to 7pm Cyprus timezone (EET / EEST). Support is provided by the same developers writing the software, all of which live in Europe. You can still file tickets outside of our working hours, but we cannot respond to them until we're back at the office.
Support policy: We would like to kindly inform you that when using our support you have already agreed to the Support Policy which is part of our Terms of Service. Thank you for your understanding and for helping us help you!