Phew! Long time since my last blog post. Truth be told, I was very busy with producing JoomlaPack 2.4.b1, the video tutorials and some other projects I am currently working on. Since you waited so long to hear from me, let me give you a nice, short, useful piece of information. I'm going to discuss why many of you get HTTP 403 "Forbidden" error messages and how to work around them!
Important! If you want to find out how to dodge HTTP 403 Errors while backing up and spare the technical details, just skip ahead to "Solution".
Many servers out there employ security solutions to protect themselves against malicious users trying to perform nasty things like SQL injection or DDoS attacks. The most commonly employed countermeasures are mod_security and apparently mod_evasive. Without falsely claiming that I am an Apache server security expert, the rundown of how these two modules work is like this. First, they check the request's IP address against a permanent and a temporary black-list. If you're on the black-list, it's HTTP Error 403 for you. The next - and most important - check is request frequency, which I'm going to discuss right now.
Let's for a moment get in a wannabe attacker's shoes. How can we bring down foobar.com? The easiest (not very technically challenging) way is to launch a Denial of Service flooding attack. In other words, we use a beefy connection to bombard the foobar.com's server with hundreds of requests every second. Obviously, we won't get far with just 1 network connected machine if foobar.com is hosted on a server cluster pool or an equivalently flexible cloud service. To this end, we can group together many machines, probably on disparate physical networks, to launch a DDoS (Distributed Denial of Service) attack. Yes, that's what botnets do. Anyhow, the common denominator is that the same IP hits the same or similar server resources many times in succession. Both of these security tools will detect this behavior, return an HTTP 403 Error and put the IP in the temporary black-list. The more the attacker insists, the longer he'll be in this blacklist.
Now, both security modules have even more features, but it's this anti-flooding / anti-DoS feature which gets in JoomlaPack's way. Why? Let me explain to you how JoomlaPack works in AJAX mode. Every time a step (the smallest quantum of work JoomlaPack's "CUBE" backup engine can perform) is finished, a reply is sent to the browser. The browser will then send a new AJAX request to the browser, asking for an updated status page which then renders on the window. Finally, a new AJAX request is sent back to the browser to let the new step run.
Some JoomlaPack steps can take milliseconds to complete. They happen when the engine switches to a new domain (e.g. from preparing to database dumping, or from database dumping to file packing). The ultimate reason is that since we can't gauge how fast the preparation step will run, we prefer to run it inside a step of its own in order to avoid timeouts. Yeah, but this causes requests to be sent too fast to server. Bummer!
The solution (for now)
Quite a few versions ago, I coded a nifty feature called "Delay between backup steps". It's in the Advanced section of JoomlaPack's configuration page, as long as you have enabled the Expert Mode. With this feature you can force a delay to be issued between individual backup steps. Usually, a value between 750 and 2000 (it's in milliseconds, so we're talking 0.75 to 2 seconds) will do. However, this is not the ultimate solution! As I mentioned above, JoomlaPack in AJAX mode does two requests per step. This can still cause backup errors. So, the complete solution involves changing two parameters:
- Set "Backup method" to "Javascript Redirects"
- Set "Delay between backup steps" to a value between 750 and 2000.
And yes, I tested this on a local testing server (Ubuntu Server 8.04 LTS with Apache + mod_security + mod_evasive; an overkill of security features) and it works just fine. As always YMMV.
The future - and better - solution
Ah, you know me, I can't resist giving away my future plans on JoomlaPack in those blog posts! Well, simply put, I am going to deprecate the "Delay between backup steps" in favor of a cleaner solution called "Minimum time per step". This will make sure that NO step will last shorter than the specified amount of milliseconds. For example, setting it to 2000 will make each step last at least 2 seconds. Why is that? The current solution adds the delay even if it's not necessary, on steps lasting several seconds. This makes the backup slower without any good reason. Moreover, this new solution will be implemented server-side (in the PHP code) so it will work in AJAX mode, Javascript Redirects mode, as well as front-end backups and CRON jobs! One size fits all and goodbye backup worries due to security modules! Since you are wondering, the default value will be 2000, as I haven't run into a server which bans users issuing less than one request per second. We're talking about sane defaults here, folks :)
Whenever this is ready, you can be sure it will hit our development builds' server, the Bleeding Edge. Stay tuned.