Akeeba Backup for Joomla!

Wednesday, 02 June 2010 01:00 CDT

peterleslie

firstly - love you product - bought the Pro version today ... 3.0.rc1 (2010-05-20)
I have configured S3 Upload with Access & Secret keys. copied direct from my amazon account.
I run a backup and it runs successfully - ZIP file created in backup directory. Then it begins the Post Backup processing to upload to S3 and i get the following error message and it stops. I've attached log file.
I tried doing this with ZIP part files and with Single file - same result.

Warnings
Failed to process file /home/nuvolaco/public_html/demo/backups/site-demo.nuvola.com.au-20100602-203950.zip
Post-processing interrupted -- no more files will be transferred

This happens immediately it attempts the S3 uploading.

cheers, peter

Wednesday, 02 June 2010 15:10 CDT

dlb

Peter,

Please zip and post the backup log. That may have more information about what is happening.

You will need to use split archives for uploading to S3. About 5 Mb per part should be enough to avoid a timeout during the transfer.

Dale L. Brackin
Support Specialist

English: native

Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

????
My time zone is EST (UTC -5) (click here to see my current time in Philadelphia, PA)

Wednesday, 02 June 2010 15:16 CDT

peterleslie

Hi Dale

understand regarding the breaking up of ZIP - however i was really saying the problem occurs when i break up the ZIP and when i dont.

log file attached - which i thought i did to the original message - but i might have missed that

Log File

Saturday, 05 June 2010 01:17 CDT

nicholas

Since this is a live host, please check with your host that their firewall is not blocking access to s3.amazonaws.com. If it doesn, they should be able to add an exception to their firewall for Akeeba Backup to be able to upload the backup archives to S3.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Tuesday, 15 June 2010 04:38 CDT

peterleslie

thanks Nicholas - i will follow up with the Web Host and find out.
are there any particular Ports that are used by Akeeba to S3 that should be open. If they are blocking and require port to unblock?

peter

Tuesday, 15 June 2010 14:06 CDT

peterleslie

OK, my webhost has come back and told me that they only allow external connections via a Proxy - is there a way to set a Proxy for Akeeba Pro?

cheers, peter

Tuesday, 15 June 2010 20:31 CDT

nicholas

No, using a proxy is not possible at this time. It will be implemented in the next release, though.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Tuesday, 15 June 2010 21:50 CDT

peterleslie

any idea when the next release will be available?

Wednesday, 16 June 2010 21:34 CDT

peterleslie

Nicholas

my web host will look at opening the port to Amazon S3

do you know what port numbers are required and what if any IP addresses.

peter

Thursday, 17 June 2010 00:05 CDT

nicholas

The port required in 80 (if you didn't tick encrypted connection checkbox) or 443 (if you do use encrypted connections). FYI, all communication with S3 is done using a RESTful API over standard HTTP or HTTPS communications. This information may be useful for your host when setting up their firewall.

The next version will take a while. There are some big changes planned for the backup administration page and the interaction of the engine with its clients (back-end backup, front-end backup, remote backup) so it'll take at least another month and a half before a beta is released.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Thursday, 17 June 2010 21:00 CDT

user11739

I have the exact same problem (aka: I get the same warning) but I have no clue about what I am doing wrong. You can see the log file attached to my post.
Kind regards,
Jeroen

Friday, 18 June 2010 03:54 CDT

nicholas

Did you try the solution suggested on this thread before posting?

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Sunday, 20 June 2010 21:39 CDT

peterleslie

Hi Nichola

my webhost have opened Port 80 & 443 to s3.amazonaws.com. However i have no real way to verify that this works other than running a Akeeba backup - which i have just done. However backup failed with the same problem

i have uploaded the latest log file from this backup - done 1/2 hr ago

your advice would be appreciated.

peter

Log File

Monday, 21 June 2010 08:45 CDT

nicholas

Still, there is no communication with S3. At this point I'd suggest taking a good look at the Amazon keys you have configured in this backup profile. Are both keys copied & pasted verbatim from Amazon's account page? Do note that they are not the same as your Amazon username and password, they are the two keys appearing on your account page (don't ask me why, I haven't figured out why Amazon is so complicated). Both keys are case sensitive.

If both keys are correct, there still is a problem at the firewall level. As you said, there is no other way to check it than run an Akeeba Backup backup attempt. Maybe you can tell your host at what time the backup attempt is scheduled and at what time the actual upload attempt happens (check for the post-processing message in the log and not down the timestamp) so that your host can cross-check it with their firewalls, identify and resolve the communication error.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Wednesday, 23 June 2010 16:27 CDT

peterleslie

Hi Nicholas

my webhoster has confirmed there is connectivity to the S3 from the webserver - see below. However i am still getting the same problem. I've attached log file again - Log File

--------
I'm not familiar with the underlying technology they are using but we have checked and confirmed that there is connection between our server to the 2 server that you provided. Please see below:

root@nebula [~]# telnet s3-2.amazonaws.com 80
Trying 207.171.185.129...
Connected to amazon.com (207.171.185.129).

root@nebula [~]# telnet s3-2.amazonaws.com 443
Trying 207.171.185.129...
Connected to amazon.com (207.171.185.129).
Escape character is '^]'.

Same result with s3-1.amazonaws.com 72.21.207.242

Please double check with the Amazon S3 tech that these are the 2 servers that required the connection and/or the port are correct or if there are more settings needs to be changed.

Wednesday, 23 June 2010 23:35 CDT

nicholas

Normally this should be adequate. I'm not absolutely sure because S3 is using the anycast domain s3.amazonaws.com to expose its API. Being "anycast" it resolves to a different IP and subdomain per region. For example, trying this from Greece it gives me s3-1.amazonaws.com with an IP of 72.21.202.164. If your host's support technician did the telnet from his office terminal and not while connected by SSH on the server he's getting the wrong IP!

To prove my point. First, let's see what dig s3.amazonaws.com returns from my office computer in Greece:
[code]; DiG 9.3.2 s3.amazonaws.com
;; global options: printcmd
;; Got answer:
;; ->>HEADER

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Thursday, 24 June 2010 16:11 CDT

peterleslie

Hi Nicholas

it is NOW WORKING ... YEAH!!

the problem was that in the S3 configuration i had not selected SSL ... as soon as i select Use SSL the backup are now working and uploading to my bucket on S3.

I have confirmed after a successful backup - that if i turn SSL off - then the backup fails (in that it WON'T load to S3).

so there appears to be a bug in the application. If you want more details about my hosting environment - let me know.

peter

Friday, 25 June 2010 02:47 CDT

nicholas

No, this is a host-specific issue. I tried disabling SSL and taking a cloud backup from 3 different servers (located in Denver, London and Athens) and they all worked without SSL. I also tried the same thing with SSL and they all worked. I think that your host has only opened port 443 on their firewall, therefore only the SSL mode works. Well, this is actually good! The extra protection of the SSL layer is always welcome :)

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Thursday, 08 July 2010 08:21 CDT

user11051

Could you please post some code to test the HTTP/HTTPS connection from the server, as you did to get the resolving IP address for the hostname 's3.amazonaws.com'?

I think that could be very useful to isolate this issue.

Kind regards, and thank you very much for your time.
César.

Thursday, 08 July 2010 09:17 CDT

nicholas

Well, sort of. You first need to install s3cmd - the unofficial and working command-line S3 client - on the server. Then, configure it with your access and private keys (just run s3cmd, it asks for this info). At the end of the config, it asks for connection method (palin or encrypted) and checks the connection. This is my standard test on the servers I manage on my own or I have root SSH access to.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Wednesday, 14 July 2010 02:53 CDT

spfaehler

For what it is worth...

I was receiving this same error when trying to backup to S3. After trouble shooting with my host, and confirming that the amazon site was not blocked as suggested above, I decided to try a few SIMPLE solutions. The first thought I had was to make sure that I used all lower case characters in my bucket and directory names. And to remove any special characters such as _ or spaces.

Turns out that this was the problem for me. Everything now works perfectly!

Hope this helps others.

Seth

Wednesday, 14 July 2010 02:58 CDT

nicholas

I wasn't aware of such a restriction in bucket names. I always use only lowercase alphanumeric letters and dashes, which work flawlessly. I'll try using some mixed case letters and special characters in order to replicate this.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Wednesday, 14 July 2010 03:15 CDT

spfaehler

My original bucket name has uppercase characters and a "_". I removed both and it now works, so I'm not sure which it was causing the issue for me.

Wednesday, 14 July 2010 23:44 CDT

nicholas

I did some research. Your bucket name could not contain uppercase letters and underscores, as these characters are disallowed by Amazon S3's system. If you try using S3Fox to manage your bucket - or even AWS Management Console - you'll see exactly what I mean. As a result, Akeeba Backup was requesting S3 for permission to connect to a non-existent bucket, causing the connection error. I will be updating the documentation to reflect that limitation.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Friday, 16 July 2010 03:13 CDT

user11051

My case was a little different, but I want to publish it in case it helps somebody.

My issue was that the server time wasn't correctly set, and thus the Amazon S3 system was rejecting the request because the time gap was too large (for security reasons, the maximum gap allowed by Amazon S3 is 15 minutes).

It was ifficult to get that information, because I had to write some PHP code to get to see the Amazon S3 XML response. I think it would be extremely useful that, in case the Amazon S3 request is rejected, the Akeeba Backup system handled the Amazon S3 XML response to display the error in the log file. This can be quite easy to develop, and it'd have saved me lots of time (and this can be true for many many other people).

I think it would be a good, small and useful feature to include in the next release. Anyhow, my congratulations to the development team for this superb tool, a must in every Joomla installation.

Friday, 16 July 2010 23:00 CDT

nicholas

I've just uploaded SVN186 which logs - and displays - the raw error code and error message received by the cloud storage service. This should give you a strong hint of what's going on, i.e. wrong auth key, too big time offset, network error etc. Thanks for hinting me on this! I always like to implement such good suggestions made by our users :)

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Saturday, 17 July 2010 14:52 CDT

user11523

I am having the exact same problem. I am usinng Inmotionhosting with a VPS. I have added the domain s3.amazonaws.com to my firewall but still no luck.

Where would I be able to view the logs for better troubleshooting? I get the same error as peterleslie and it happens immediately after the backup stage.

Thanks,
Jason

Saturday, 17 July 2010 15:41 CDT

dlb

Jason,

Nicholas was referring to the backup log. You can view it from the Akeeba Control Panel screen.

Dale L. Brackin
Support Specialist

English: native

Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

????
My time zone is EST (UTC -5) (click here to see my current time in Philadelphia, PA)

Saturday, 17 July 2010 16:38 CDT

user11523

So I figured it out. I must have missed the post about not having Capital letters in my bucket name. As soon as I removed the capitals from my bucket it worked.

Thank you for all your help.

Sunday, 18 July 2010 03:48 CDT

dlb

I'm glad you got it working. :)

Dale L. Brackin
Support Specialist

English: native

Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

????
My time zone is EST (UTC -5) (click here to see my current time in Philadelphia, PA)

Support