Support

Akeeba Backup for Joomla!

#8589 Backing up to S3

Posted in ‘Akeeba Backup for Joomla! 4 & 5’
This is a public ticket

Everybody will be able to see its contents. Do not include usernames, passwords or any other sensitive information.

Environment Information

Joomla! version
n/a
PHP version
n/a
Akeeba Backup version
n/a

Latest post by nicholas on Saturday, 09 April 2011 09:41 CDT

user13474
I'm new to this be gentle...

I installed. Backup is fine.
Upgrade to backup pro.
Sign up for S3.
Create bucket, get public/private keys do configuration.

Try backup.

Backup itself works - here's the proof.

Backup taken on Thursday, 26 August 2010 01:19 26.08.10 00:01:45 OK Backend Full site backup 1 278.06 Mb site-6-pack.org-20100826-011950.jpa
Part 00 • Part 01 • Part 02 • Part 03 • Part 04 • Part 05 • Part 06 • Part 07 • Part 08 • Part 09 • Part 10 • Part 11 • Part 12 • Part 13 • Part 14 • Part 15 • Part 16 • Part 17 • Part 18

Sadly it doesn't make it over to Amazon.

I get a couple of ambiguous warnings:

Failed to process file /home/aatkddny/public_html/j15/administrator/components/com_akeeba/backup/site-6-pack.org-20100826-011950.jpa
Post-processing interrupted -- no more files will be transferred

And that's all she wrote.
Here's the last part of the log:

DEBUG |100825 21:21:37|====== Starting Step number 18 ======
DEBUG |100825 21:21:37|Loading post-processing engine object
INFO |100825 21:21:37|Post processing file /administrator/components/com_akeeba/backup/site-6-pack.org-20100826-011950.jpa
DEBUG |100825 21:21:37|No quotas were defined; old backup files will be kept intact
DEBUG |100825 21:21:37|----- Finished operation 1 ------
DEBUG |100825 21:21:37|----- Finished operation 2 ------
DEBUG |100825 21:21:37|Successful Smart algorithm on AECoreDomainFinalization
DEBUG |100825 21:21:37|Kettenrad :: Domain 'finale' has finished.
DEBUG |100825 21:21:37|====== Finished Step number 18 ======
DEBUG |100825 21:21:37|*** Batching of engine steps finished. I will now return control to the caller.
DEBUG |100825 21:21:37|Sleeping for 1815.50192833 msec, using usleep()
DEBUG |100825 21:21:39|Saving Kettenrad instance
DEBUG |100825 21:21:40| -- Loaded stored Akeeba Factory
DEBUG |100825 21:21:40|====== Starting Step number 19 ======
DEBUG |100825 21:21:40|Kettenrad :: No more domains to process
DEBUG |100825 21:21:40|*** Batching of engine steps finished. I will now return control to the caller.
DEBUG |100825 21:21:40|Sleeping for 1978.89709473 msec, using usleep()
DEBUG |100825 21:21:42|Saving Kettenrad instance
DEBUG |100825 21:21:42| -- Loaded stored Akeeba Factory
DEBUG |100825 21:21:42|Kettenrad :: Just finished
DEBUG |100825 21:21:42|*** Batching of engine steps finished. I will now return control to the caller.
DEBUG |100825 21:21:42|Sleeping for 1979.16507721 msec, using usleep()
DEBUG |100825 21:21:44|Saving Kettenrad instance
DEBUG |100825 21:35:29|Kettenrad :: Attempting to load from database
--- END OF RAW LOG ---?

So I've probably got something badly configured, but I can't figure it out from these messages and the manual.

Anyone able to tell me what I did wrong?
TIA, Alan

user13474
Ok, I upgraded to 3.1.b2
Now I get this:
AEUtilAmazons3::putObject(): [AccessDenied] Access Denied

I can get to S3 using S3Fox, so I think I get the config right.
I spoke to my hosting provider.
No proxy server.
No firewall on S3.
80 and 443 open.

Not sure what's wrong or what to try next...

Also you might want to note that the 3 checkboxes inside the S3 configure portion don't actually save in this version...

nicholas
Akeeba Staff
Manager
This is an error coming from the S3 servers. It tells you that the account you're using doesn't have write access to that bucket. You can use S3fox to make sure that the account you're using in Akeeba Backup has write privileges on the bucket and you're set.

BTW, the S3 integration is the most well supported of all cloud storage engines as it is the one I use to keep daily backups of all of my own sites, including this one. If it breaks, trust me, I'll be the first to know ;)

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

user13474
Then I must be doing something completely stupid.
I've used the same public/private key pair in S3Fox as Akeeba.
I can transfer files in S3Fox without any issue.
Akeeba gives me this error.

I realise it's almost certainly a config issue, and probably a stupid one, but I'm at a loss to see what I've done wrong.

I've tried every combination of "process each part immediately", delete archive" and "use SSL" with archive split sizes from 5M to 20M and I'm still not having any success. Of course since none of these options appear to be persisted, I probably wasted my time here.

I even thought maybe there was an odd issue with the bucket name - the first one I tried started with a numeric (I've no idea why this would be an issue but I tried a different one anyway).

I've set up the archive in S3 so every combination of possibilities - Me, registered, everyone has all priviliges (which I don't need to do for S3Fox, but at that point I was ready to try anything).

I'm at a loss as to what else I can try. S3Fox works, this throws an error.
Any ideas?

nicholas
Akeeba Staff
Manager
The only thing I can think of is entering the wrong credentials. When copying the credentials from Amazon's site you have to be careful not to include a leading or trailing space. You may also want to see another explanation on resetting S3 permissions from a user of another solution which uses S3 as its storage back-end.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

user13474
It's not the credentials - you might want to investigate this yourself.

If you create a bucket and either try to add logging to it post create, or during create, it doesn't work.

If you create a bucket and totally ignore logging, it works just fine. If you delete one of the buckets from the earlier steps and recreate it without logging it works fine too. There's something to do with adding logging that messes things up.

I didn't investigate further, because it's clearly working for me now.
You might want to have a poke around to see why, or just change the PDF so it notes this. I spent way too long futzing with it to wish the same thing on anyone else.

b/w Alan

nicholas
Akeeba Staff
Manager
On my S3 account adding logging doesn't interfere with Akeeba Backup's ability to store the backups. Tested with Akeeba Backup 3.1.b2 using this site as the guinea pig of this experiment. Ii added logging to my bucket using CloudBerry Explorer for Amazon S3.

Most probably your S3 client messes up with the ACLs when it enables logging, or the backup and the log files have the same names (highly unlikely, but you never know).

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

tsp
Hi Nicholas,

after your recommendation for amazon s3, I am struggeling to deal with multi-part archives.

whilest a full upload in one file (31MB) is working perfect,
multi part files are not uploaded - only the last one which was renamed to the jpa.
the log file also is noting
S3 -- Legacy (single part) upload of site xxx
whilest in the admin of the backup files all 4 parts are showing up properly! (and can be downloaded from the site).
on Amazon S3 only the one jpa split part was uploaded.

I am using the current pro version
3.2.5

many thanks for a hint!
Torsten

nicholas
Akeeba Staff
Manager
Can you please ZIP and attach your backup log file so that I can see what is going on? Thank you!

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

tsp
of course, here we go.
many thanks!
Torsten

nicholas
Akeeba Staff
Manager
The log files never came through. I did some thorough testing today and I could not replicate the issue. I did see, however, that there are two options turned on by default in 3.2.5: "Process each part immediately" and "Use SSL". Please go to your Akeeba Backup Configuration page and uncheck those two options. Does uploading to S3 work now?

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

tsp
unfortunately not - still only the jpa file there, whilest in backend 4 files(parts) are created and accessible.

I try to attach the log again.

btw - is there a way to upgrade from the PRO to the DELUXE version of the subscription, I purchased PRO only yesterday.
I am highly implressed by your support.

add- upload doesnt work, i tried it several times, the zip file (size 79kb) doesn't upload. Can I email it to you?

nicholas
Akeeba Staff
Manager
Confirmed; I am working on the fix right now and will release an update in a couple of hours.

Regarding the subscription update, is it OK to send you a payment request for the price difference between the two releases? Once you effect the payment I will switch your subscription to AKEEBADELUXE.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

tsp
hi Nicholas,

yes, sure, please send the request.

nicholas
Akeeba Staff
Manager
OK, sent :)

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

tsp
confirm that the new version is running ok.
many thanks

nicholas
Akeeba Staff
Manager
Excellent! Thank you for your feedback :)

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Support Information

Working hours: We are open Monday to Friday, 9am to 7pm Cyprus timezone (EET / EEST). Support is provided by the same developers writing the software, all of which live in Europe. You can still file tickets outside of our working hours, but we cannot respond to them until we're back at the office.

Support policy: We would like to kindly inform you that when using our support you have already agreed to the Support Policy which is part of our Terms of Service. Thank you for your understanding and for helping us help you!