Support

Akeeba Backup for Joomla!

#8973 Security issues running backup.php and altbackup.php

Posted in ‘Akeeba Backup for Joomla! 4 & 5’
This is a public ticket

Everybody will be able to see its contents. Do not include usernames, passwords or any other sensitive information.

Environment Information

Joomla! version
n/a
PHP version
n/a
Akeeba Backup version
n/a

Latest post by nicholas on Thursday, 04 August 2011 17:09 CDT

user8399
Mandatory information about my setup:

Have I read the related troubleshooter articles above before posting (which pages?)? Automated scheduling in manual
Have I searched the forum before posting? Yes
Have I read the documentation before posting (which pages?)? Yes
Joomla! version: 1.5.23
PHP version: 5.2.17
MySQL version: 5.1.56
Host: Heart Internet
Akeeba Backup version: 3.3.1

EXTREMELY IMPORTANT: Please attach your Akeeba Backup log file in order for us to help you with any backup or restoration issue.

Description of my issue:
Starting a backend backup through the browser works fine; so config settings must be OK. If I try to use backup.php through a CRON job, the backup cuts out part way through. This may be to do with the timing issue referenced in the manual. Running altbackup.php, I'm told that no message was received and a fatal php error has caused the backup to fail. The Akeeba log for the latter doesn't show anything.

I feel that this must be something to do with site's php.ini. I've attached a copy, which is restrictive for security purposes. To overcome this, I placed a copy of php.ini in the com_akeeba components directory with the view of lifting the security restrictions and increasing resources (see attached).

I don't think this has taken effect because if I look at the log of a "normal" backend backup, Akeeba isn't picking up on the configuration changes. If I place the akeeba php.ini into the /administrator directory and ask Joomla to give me the PHPInfo, it shows the updated settings. Obviously, I want to be careful which directories are exposed to lessened PHP security.

Any troubleshooting ideas on where to take this next? Anyone had a similar issue?

Dominic

user8399
Attachments didn't add.

nicholas
Akeeba Staff
Manager
All your problems spring from this:
INFO |110802 12:32:31|PHP Interface :cgi


You are using the PHP CGI (Common Gateway Interface, used for web scripts) binary, not the PHP CLI (Command Line Interface). You have to use the PHP CLI binary for either backup.php or altbackup.php. Otherwise they are bound by the timeout and other restrictions of the PHP CGI interface.

Moreover, the PHP CLI binary has a completely different php.ini configuration than the CGI interface, meaning that the less strict security settings set forth in the CLI's php.ini file will not have anything to do -and, therefore, unable to reduce- your web site's security (it's bound by the rules of the PHP CGI php.ini).

Finally, please note that the custom php.ini for your site must be placed on both the site's root and administrator directory in order to have any effect. Moreover, this file will not have any effect whatsoever for PHP scripts which do not go through Joomla!'s index files. Many extensions do use such scripts, therefore the php.ini settings will be completely irrelevant. If you are enforcing php.ini settings for security reasons do so in the web server's global php.ini file, not on your local php.ini files. In any other case it's trivial for an attacker to bypass those settings by creating a subdirectory (or using an existing subdirectory), upload the hacking script in there and exploit your site with his leisure, without being bound by your security settings.

Nicholas K. Dionysopoulos

Lead Developer and Director

πŸ‡¬πŸ‡·Greek: native πŸ‡¬πŸ‡§English: excellent πŸ‡«πŸ‡·French: basic β€’ πŸ• My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

user8399
Thank you, that information was helpful. Can I just highlight that the native cron documentation only references /php5-cli interpreter in the last example. All other examples just reference /php5, which is what Heart Internet give as a PHP interpreter example for their CRON jobs.

Making this small change (which Heart obviously supports), I have made some progress. Backup.php still fails but it's getting further than it did before. It doesn't write the third .j03 file or the .jpa file. (log attached).

altbackup.php gives the same error as last time, even with the correct interpreter and doesn't produce a log in Akeeba.

nicholas
Akeeba Staff
Manager
Hi!

I can see that in both cases the process dies 300 seconds into the backup process. This is not a coincidence; it's a hard time limit set by your host. What this means is that they are halting CRON scripts which try running over 300 seconds. You can ask them if they can make an exception for the backup script. Otherwise, you won't be able to take a backup using either script (backup.php or altbackup.php).

Nicholas K. Dionysopoulos

Lead Developer and Director

πŸ‡¬πŸ‡·Greek: native πŸ‡¬πŸ‡§English: excellent πŸ‡«πŸ‡·French: basic β€’ πŸ• My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

user8399
I've asked Heart and they say that the max execution time is set at 0 seconds for unlimited. When I looked at the akeeba command line log, I can indeed see that max execution time is recorded as 0.

I know from what you said before that it would make no difference but because Heart suggested it, I removed the execution time statement from my php.ini. Predictably, this made no difference at all.

Could there be a timeout statement somewhere other than the php cli php.ini that is affecting the cron job?

nicholas
Akeeba Staff
Manager
There are two time limits.

1. The PHP maximum execution time limit. This applies to PHP only and in your case it is, indeed, 0 which means "no limit". I already knew that, looking at the two log files.

2. The maximum time a process spawned by CRON can run. This is outside of the php.ini file. It is per user and applies to all CRON jobs launched on behalf of that particular user, no matter if it's a PHP script, a Bash script or a binary executable file. In your case, it appears to be set to 300 seconds and this is what I want your host to modify.

If the first level of support technicians can't help you, try escalating your issue. The first level of support technicians is not always proficient with this kind of advanced server settings. Depending on the host, their level may var from script-reading drones to junior engineers. The second level of support technicians -on all hosts I've seen to this day- are senior engineers, that is well-versed professionals who know the server setup and will understand what we're talking about.

Nicholas K. Dionysopoulos

Lead Developer and Director

πŸ‡¬πŸ‡·Greek: native πŸ‡¬πŸ‡§English: excellent πŸ‡«πŸ‡·French: basic β€’ πŸ• My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

user8399
I'm stuck between a rock and a hard place.

Heart have come back and swear blind that there is no time limit as to the length of time that a Cron process can take. They claimed that they could see some CRON jobs that had been running for some time (more than 300 seconds). What they did do, was run the same backup.php command from an actual command line shell (as a user with the same permissions level as me). The backup worked fine and took about 90 seconds. The akeeba logs showed that everything ran perfectly.

So I told Heart that there must be something different between CRON running the backup command, and doing it from the shell. It's the only variable! They say not. They say the lack of errors in the logs don't give them any indication of where to look for the problem but that it can't be a timeout setting. But I say it can't be the backup.php script either because it was run successfully from an actual command line.

Knowing what backup.php does, can you think of anything that could be done to highlight within the akeeba logs the reason for the script to fail (like a verbose output or something)? I don't get to see the crontab file that's produced. Do you know of anything that could go in there that might upset the execution?

user8399
Example of the successful backup.

nicholas
Akeeba Staff
Manager
The only differentiating factor between the successful and failed backup that the successful backup ran through an interactive shell instead of through the CRON daemon. Knowing what backup.php does (nothing special, it just loads the backup engine -like the back-end backup- and asks it to step through the backup process until it's done) I can't see what could be different between the two runs.

The things that can be different:
- Nice level (priority of the process execution). If the CRON runs with a very big nice level, it will run very slowly. However, I can't see how on Earth that could possibly have to do with anything, unless we have a time limit.
- CPU usage restrictions. When running a command from an interactive shell, there is no such thing. Your host claims that there is no such thing for CRON. Huh!
- Forced time limit for CRON jobs. In combination with the high nice level it could explain why the backup dies prematurely.

In short, what your host tells you doesn't stick with what we observe. If your host is right, the backup fails for no reason at all. Since there are no error reported in the backup log and no errors reported in the PHP error log it's not a code issue. All the other possibilities which could cause the CRON job to fail are disputed by your host. I don't know what to say. Either your host is right and there is a ghost in the machine killing the CRON job, or your host is wrong and one (or all) of the above limitations are in effect and kill your CRON job.

Nicholas K. Dionysopoulos

Lead Developer and Director

πŸ‡¬πŸ‡·Greek: native πŸ‡¬πŸ‡§English: excellent πŸ‡«πŸ‡·French: basic β€’ πŸ• My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

user8399
I think I can definitely say that Heart are using NICE levels on CRON jobs. From the command line, the backup takes about 1 minute 30 seconds to complete. From CRON, the job is already at 2 minutes by the time it stops on the 3rd file.

I am also slightly dubious of the fact that they haven't put restrictive controls in place for CRON jobs. It is, after all, a shared server.

I'm continuing to look for differences. I wondered if the user environment variables might be a cause but I'm specifying full paths anyway. I've tried starting the command from a shell script and placing the execution of the shell script into the CRON job. No luck.

Can you just help explain some differences in the akeeba log?

Here is the start of the log from the successful backup (command line) and then the unsuccessful backup (CRON). Note that the values for Safe Mode, Disabled Functions, Open Base Dir, Memory limit and Current Memory Usage are different. These values have come from my php.ini file but I thought php-cli should be using a different php.ini? Shouldn't both logs reflect the same starting conditions? Would any of these things have an affect on the outcome of the backup?

Successful
--- START OF RAW LOG --
INFO |110803 10:53:52|--------------------------------------------------------------------------------
INFO |110803 10:53:52|Akeeba Backup 3.3.1 (2011-07-22)
INFO |110803 10:53:52|Got backup?
INFO |110803 10:53:52|--------------------------------------------------------------------------------
INFO |110803 10:53:52|--- System Information ---
INFO |110803 10:53:52|PHP Version :5.2.17
INFO |110803 10:53:52|OS Version :Linux
INFO |110803 10:53:52|DB Version :5.1.56-community-log
INFO |110803 10:53:52|Web Server :n/a
INFO |110803 10:53:52|PHP Interface :cli
INFO |110803 10:53:52|Joomla! version :Joomla! 1.5.23 Stable [ senu takaa ama baji ] 04-March-2011 18:00 GMT
INFO |110803 10:53:52|Safe mode :
INFO |110803 10:53:52|Display errors :
INFO |110803 10:53:52|Error reporting :E_ERROR | E_WARNING | E_PARSE | E_CORE_ERROR | E_CORE_WARNING | E_COMPILE_ERROR | E_COMPILE_WARNING | E_USER_ERROR | E_USER_WARNING | E_USER_NOTICE
INFO |110803 10:53:52|Error display :off
INFO |110803 10:53:52|Disabled functions :
INFO |110803 10:53:52|open_basedir restr.:
INFO |110803 10:53:52|Max. exec. time :0
INFO |110803 10:53:52|Memory limit :128M
INFO |110803 10:53:52|Current mem. usage :1711952
INFO |110803 10:53:52|GZIP Compression : available (good)
INFO |110803 10:53:52|JPATH_BASE :/administrator
INFO |110803 10:53:52|JPATH_SITE :
INFO |110803 10:53:52|JPATH_ROOT :
INFO |110803 10:53:52|JPATH_CACHE :/administrator/cache
INFO |110803 10:53:52|Computed root :
INFO |110803 10:53:52|Output directory :/home/sites/datatrackplc.com/backup
INFO |110803 10:53:52|Temporary directory:/home/sites/datatrackplc.com/tmp


CRON
INFO |110804 08:13:06|--- System Information ---
INFO |110804 08:13:06|PHP Version :5.2.17
INFO |110804 08:13:06|OS Version :Linux
INFO |110804 08:13:06|DB Version :5.1.56-community-log
INFO |110804 08:13:06|Web Server :n/a
INFO |110804 08:13:06|PHP Interface :cli
INFO |110804 08:13:06|Joomla! version :Joomla! 1.5.23 Stable [ senu takaa ama baji ] 04-March-2011 18:00 GMT
INFO |110804 08:13:06|Safe mode :0
INFO |110804 08:13:06|Display errors :
INFO |110804 08:13:06|Error reporting :E_ERROR | E_WARNING | E_PARSE | E_CORE_ERROR | E_CORE_WARNING | E_COMPILE_ERROR | E_COMPILE_WARNING | E_USER_ERROR | E_USER_WARNING | E_USER_NOTICE
INFO |110804 08:13:06|Error display :off
INFO |110804 08:13:06|Disabled functions :show_source, system, shell_exec, passthru, exec, phpinfo, popen, proc_open
INFO |110804 08:13:06|open_basedir restr.:/home/sites/datatrackplc.com/
INFO |110804 08:13:06|Max. exec. time :0
INFO |110804 08:13:06|Memory limit :120M
INFO |110804 08:13:06|Current mem. usage :1710212
INFO |110804 08:13:06|GZIP Compression : available (good)
INFO |110804 08:13:06|JPATH_BASE :/administrator
INFO |110804 08:13:06|JPATH_SITE :
INFO |110804 08:13:06|JPATH_ROOT :
INFO |110804 08:13:06|JPATH_CACHE :/administrator/cache
INFO |110804 08:13:06|Computed root :
INFO |110804 08:13:06|Output directory :/home/sites/datatrackplc.com/backup
INFO |110804 08:13:06|Temporary directory:/home/sites/datatrackplc.com/tmp

user8399
I've done some tests. There's a timeout somewhere; I just have to convince Heart.

I copied my default profile and started testing some exclusions. The biggest contributors (for file size) to the backup are a blob table in the database and the /images directory. Exclude one of those and you have a big impact on the time the backup takes. The blob table takes 24 seconds to dump. About 16 seconds to scan the /images directory.

If I try and do the whole backup, it stops at 1 minute 25 seconds. If I exclude the blob table from the database, the backup successfully completes in 1 minute 07 seconds. If I switch and include the blob table and exclude the /images, the backup completes in 1 minute 18 seconds.

This proves that neither the blob table or the /images directory can be causing a problem on its own because independently they get backed up. The only difference is that it takes longer to back up both.

user8399
Forgot to mention, when run from the command line, the script completed the whole backup in 1 minute 17 seconds, which of course is under the 1 minute 25 when it stops with CRON.

user8399
[SOLVED}
Why does native CRON backup.php script suddenly stop when run on Heart Internet using php-cli.

In the Akeeba configuration settings, using the JPA Format for the Archiver Engine, there's a setting for Big File Threshold. For me, this was set to 3MB (and had been working fine for Remote and Backend backups). Changing this setting to 2MB to match the setting for Chunk Size For Large Files Processing was the difference between the backup stopping and completing successfully.

I literally tried setting it to 3, back to 2 and then back to 3. It was like an on/off switch for a successful backup.

I got Heart to run the backup command from the terminal with a NICE level to match cron. The process took 1 minute 30 seconds and still completed. When I made the setting change to 2MB, the task completed successfully via cron and took a similar amount of time.

I'd be interested to hear your thoughts on what must have changed in Akeeba's behaviour. There was obviously some conflict with Heart's host environment. Strange that it's worked ok via RPC and the backend.

nicholas
Akeeba Staff
Manager
Based on the two log excerpts, it seems that when you run PHP from the SSH terminal and when you call it through CRON, a different php.ini (or even a different PHP binary) is loaded. If the PHP binary which runs under CRON has a limitation of creating files up to 2Mb, this problem will happen. That's the only reasonable explanation. Of course that's something your host's support technicians ought to have known and tell us, but they seem to know less about their servers than me; and just think that I have never laid hands on any of their servers... Maybe you should consider migrating to a host which knows what they're doing?

Nicholas K. Dionysopoulos

Lead Developer and Director

πŸ‡¬πŸ‡·Greek: native πŸ‡¬πŸ‡§English: excellent πŸ‡«πŸ‡·French: basic β€’ πŸ• My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Support Information

Working hours: We are open Monday to Friday, 9am to 7pm Cyprus timezone (EET / EEST). Support is provided by the same developers writing the software, all of which live in Europe. You can still file tickets outside of our working hours, but we cannot respond to them until we're back at the office.

Support policy: We would like to kindly inform you that when using our support you have already agreed to the Support Policy which is part of our Terms of Service. Thank you for your understanding and for helping us help you!