Support

Akeeba Backup for Joomla!

#8495 S3 Split Archive Size

Posted in ‘Akeeba Backup for Joomla! 4 & 5’
This is a public ticket

Everybody will be able to see its contents. Do not include usernames, passwords or any other sensitive information.

Environment Information

Joomla! version
n/a
PHP version
n/a
Akeeba Backup version
n/a

Latest post by nicholas on Wednesday, 23 June 2010 23:59 CDT

user6022
I was surprised to find that Akeeba successfully transfered a 90Mb complete backup as a single file.

On initial setup I did not make any changes to the configuration except to upload to S3. Apparently this successfully uploaded a single 95Mb file to Amazon without breaking into smaller chunks.

I am little concerned as everything thing I have read has indicated this is high unlikely. As they say 'if it seems to be too good to be true than it probably isn't'

Just after some feedback or reassurance that this is ok.

FYI i am on a VDE/VPS server.
Raoul

dlb
I really can't comment on this, I'll flag it for Nicholas. This is highly dependent on your connection speed. so it may be good and true. :)


Dale L. Brackin
Support Specialist


us.gifEnglish: native


Please keep in mind my timezone and cultural differences when reading my replies. Thank you!


????
My time zone is EST (UTC -5) (click here to see my current time in Philadelphia, PA)

nicholas
Akeeba Staff
Manager
Highly unlikely doesn't mean impossible. It all depends on PHP's max_execution_time, the file size and the transfer rate between your host and S3. I will quote the manual:

Before you begin, you should know the limitations. Amazon S3 does not allow appending to files, so the archive has to be transferred in a single step. PHP has a time limit restriction we can't overlook. The time required to upload a file to Amazon S3 equals the size of the file divided by the available bandwidth. We want to time to upload a file to be less than PHP's time limit restriction so as to avoid timing out. Since the available bandwidth is finite and constant, the only thing we can reduce in order to avoid timeouts is the file size. To this end, you have to produce split archives, by setting the part size for archive splitting in ZIP's or JPA's engine configuration pane. The suggested values are between 10Mb and 20Mb. Most servers have a bandwidth cap of 20Mbits, which equals to roughly 2Mb/sec (1 byte is 8 bits, plus there's some traffic overhead, lost packets, etc). With a time limit of 10 seconds, we can upload at most 2 Mb/sec * 10 sec = 20Mb without timing out. If you get timeouts during post-processing lower the part size.


If your host's max_execution_time is high enough and the transfer rate is also high enough what you dis is not unlikely at all. Moreover, if you are launching a backup using the native CRON script (backup.php) there is no time limit! The PHP CLI binary sets max_execution_time to 0, which really means infinite. I use the same trick on this site. I have a backup profile with 20Mb part size for the manually triggered backup jobs and an identical profile with the part size set to 0 for the CRON-scheduled backups. The latter transfers an archive file faring at some hundreds of megabytes without as much as a hiccup.

If at first glance it looks too good to be true, check your definition of reality ;)

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Support Information

Working hours: We are open Monday to Friday, 9am to 7pm Cyprus timezone (EET / EEST). Support is provided by the same developers writing the software, all of which live in Europe. You can still file tickets outside of our working hours, but we cannot respond to them until we're back at the office.

Support policy: We would like to kindly inform you that when using our support you have already agreed to the Support Policy which is part of our Terms of Service. Thank you for your understanding and for helping us help you!