Support

Akeeba Backup for Joomla!

#9124 CloudFiles post-processing upload failure

Posted in ‘Akeeba Backup for Joomla! 4 & 5’
This is a public ticket

Everybody will be able to see its contents. Do not include usernames, passwords or any other sensitive information.

Environment Information

Joomla! version
n/a
PHP version
n/a
Akeeba Backup version
n/a

Latest post by nicholas on Monday, 07 November 2011 10:09 CST

user9599
Mandatory information about my setup:

Have I read the related troubleshooter articles above before posting (which pages?)? Yes. 'My backup files are not being uploaded to Amazon S3', all other articles suggested but nothing else seems related to the subject.
Have I searched the forum before posting? Yes
Have I read the documentation before posting (which pages?)? Yes. Akeeba Backup Docs
Joomla! version: 1.5.23 Stable
PHP version: 5.3.8
MySQL version: 14.14 Distrib 5.1.59
Host: Rackspace Cloud Server
Akeeba Backup version: Akeeba Backup Professional 3.3.4 (2011-09-12)

EXTREMELY IMPORTANT: Please attach your Akeeba Backup log file in order for us to help you with any backup or restoration issue.

Description of my issue:

When I try to use the 'Upload to RackSpace CloudFiles' post-processing option in Akeeba Backup, it does not often work. I am thinking the following error from the attached log might indicate the issue:

Invalid response: (curl error: 7) couldn't connect to host: Failed to obtain valid HTTP response.

I did get a number of successful file transfers to cloudfiles with this when I first tested it, but never an entire archive. The last few tests I have tried I get no successful writes at all. The attached log file is from a single part archive backup, but I have also tried multipart archives with 20 and 49 gig parts with the same result.

I read in the forum that you were unable to personally test this on Rackspace because of bank negotiation issues. I will have to check and make sure, but we can possibly provide a temporary test cloud server to troubleshoot this with if it will help.

The attached log is password protected. I will pm the password.

Thanks,

Kirk

nicholas
Akeeba Staff
Manager
Hi Kirk,

Since then I have been able to purchase a UK RackSpace account which required a slightly different setup, hence the extra configuration option for UK accounts in the CloudFiles configuration pane :) I am now sure that this feature does work.

You said that smaller files worked, but bigger ones usually don't. This is very common, sadly. The problem is usually with the host and has to do with how they have configured their servers, their hardware firewalls and routers. Actually, the error message you get indicates that the connection was abruptly closed while the transfer was taking place, which falls into this category of problems I just mentioned. The only workaround is to try smaller values. On some hosts we have to get the part size for split archives all the way down to 500Kb for such transfers to work - or change hosts.

Nicholas K. Dionysopoulos

Lead Developer and Director

πŸ‡¬πŸ‡·Greek: native πŸ‡¬πŸ‡§English: excellent πŸ‡«πŸ‡·French: basic β€’ πŸ• My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

user9599
Hi Nicholas. It's true that the files that were successfully transferred to cloudfiles were smaller than some I tried, but except for the first attempts several days ago, I am not getting a single file transferred no matter what the part size. As far as I can see the files never even start to transfer. There are no incomplete files written to the cloudfiles container, but perhaps these are cleaned up automatically? At any rate, I've attached a log using a part size of 500k. You can see that it also fails before the first part is transferred.

There have been no issues with other transfers between our cloud server and cloud files. Both the cloud server and the cloud files are within the Rackspace internal data center network, which has performed great with all setups we have tested.

Thanks,

Kirk

nicholas
Akeeba Staff
Manager
Hi Kirk,

I did some more testing now, using a RackSpace CloudServer (based on Ubuntu 11.04 as that's the distro I'm also using for my local Linux testing server and I'm very familiar with it). It looks like if you want to use a directory with CloudFiles you need to have created it beforehand. Alternatively, you can leave the directory field blank and the backup archives will be stored in your container's root.

Nicholas K. Dionysopoulos

Lead Developer and Director

πŸ‡¬πŸ‡·Greek: native πŸ‡¬πŸ‡§English: excellent πŸ‡«πŸ‡·French: basic β€’ πŸ• My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

user9599
I left the directory at the default "/". The container was created beforehand.

nicholas
Akeeba Staff
Manager
Can you try removing everything from the directory, i.e. leaving the field blank?

Nicholas K. Dionysopoulos

Lead Developer and Director

πŸ‡¬πŸ‡·Greek: native πŸ‡¬πŸ‡§English: excellent πŸ‡«πŸ‡·French: basic β€’ πŸ• My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

user9599
Same result. By setting 'Process each part immediately' and doing the backup from Joomla, I can at least get the result sooner.

nicholas
Akeeba Staff
Manager
Given that a RackSpace CloudServer is actually a virtualised machine, equivalent to a dedicated server with root access, I think that your problem is, indeed, the firewall. Since there are a dozen of Linux distributions offered and each one of them offers at last half a dozen different firewall utilities I don't know how to help you :( I can simply tell you that you have to review your firewall settings and allow outbound connections over TCP ports 80 and 443 for all rackspace.com and rackspace.co.uk subnets for the upload to work.

Nicholas K. Dionysopoulos

Lead Developer and Director

πŸ‡¬πŸ‡·Greek: native πŸ‡¬πŸ‡§English: excellent πŸ‡«πŸ‡·French: basic β€’ πŸ• My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

user9599
Bingo! TCP out 80 and 443 are open, but I just turned off the firewall completely and the transfers work. Something else needs to be open in the firewall. I'm sure Rackspace can help me figure how what is needed.

Akeeba backup wrote most of the files successfully to cloudfiles on my first tests with the same firewall settings. JomCDN writes to cloudfiles with no problem. Nothing I can think of is blocking the outgoing connection. Yet I still should have thought to test with our firewall off the first time you brought it up. I was thinking of the firewall on the other end when you mentioned it, did not even consider our firewall could be blocking this.

Thanks!

Kirk

nicholas
Akeeba Staff
Manager
You're welcome!

Nicholas K. Dionysopoulos

Lead Developer and Director

πŸ‡¬πŸ‡·Greek: native πŸ‡¬πŸ‡§English: excellent πŸ‡«πŸ‡·French: basic β€’ πŸ• My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

user9599
The transfers to cloudfiles are working great now, but I can't seem to get remote files quotas to work. I see in the manual: 'This option actually works only for files stored on Amazon S3 or a remote FTP server'. Any chance quotas will ever work on Rackspace? It's a shame because this makes cloudfiles impractical for our backup storage. Even deleting container files manually from the control panel does not work reliably. Since the Joomla JomCDN plugin can delete cloudfiles, it seems this this must be possible.

Thanks,

Kirk

nicholas
Akeeba Staff
Manager
I'll be adding that to my to-do list. Nobody has asked before, so I hadn't really gotten into looking into it :)

Nicholas K. Dionysopoulos

Lead Developer and Director

πŸ‡¬πŸ‡·Greek: native πŸ‡¬πŸ‡§English: excellent πŸ‡«πŸ‡·French: basic β€’ πŸ• My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

user9599
Great! Thanks Nicholas.

Kirk

wbaccus
Hijacking a bit here, but add my vote to a request for Cloud files quota support.

nicholas
Akeeba Staff
Manager
Dully noted and is already on my to-do list for the next release :)

Nicholas K. Dionysopoulos

Lead Developer and Director

πŸ‡¬πŸ‡·Greek: native πŸ‡¬πŸ‡§English: excellent πŸ‡«πŸ‡·French: basic β€’ πŸ• My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Support Information

Working hours: We are open Monday to Friday, 9am to 7pm Cyprus timezone (EET / EEST). Support is provided by the same developers writing the software, all of which live in Europe. You can still file tickets outside of our working hours, but we cannot respond to them until we're back at the office.

Support policy: We would like to kindly inform you that when using our support you have already agreed to the Support Policy which is part of our Terms of Service. Thank you for your understanding and for helping us help you!