Support

Akeeba Backup for Joomla!

#38518 Slow backup restore

Posted in ‘Akeeba Backup for Joomla! 4 & 5’
This is a public ticket

Everybody will be able to see its contents. Do not include usernames, passwords or any other sensitive information.

Environment Information

Joomla! version
3.10.11
PHP version
7.4
Akeeba Backup version
8.2.7

Latest post by nicholas on Monday, 13 February 2023 02:21 CST

trogladyte

Though I'd posted this earlier this morning, but don't see it. Apologies if it's still in the system and this is a double up.

I've noticed backups and restores have become a lot slower than I remember and wonder if I'm not setting up AB properly. I just install and add things like new default output directory (outside public_html) and backup to Google Drive.=, otherwise as it comes out of the box. My normal profile setup is attached.

In particular I have a large site (for me) that's taken a VERY long time. JPA size is about 3.3Gb, with a DB size of 5.9Gb (sh404SEF's fault there). It took almost a day to extract, and it's now been at the restore DB stage for 4+ hours and only at 63%.

I used Akeeba Extract to pull all the files out of the JPA I'm trying to restore and see that I could upload those simply enough. However, I'm no DB guru and have never done a MySQL restore where there are multiple files (12379 to be exact). Is it a matter of importing the one "site.sql" file and the others will automatically install behind it?

trogladyte

Forgot the attachment.

nicholas
Akeeba Staff
Manager

Backing up a database takes a lot of time. It's the nature of a database. The time to pull a number of rows from a database table is roughly O(N2) i.e. it increases exponentially with the number of rows. That is to say, backing up a table with a million rows won't be 10 times slower than backing up a table with 100,000 rows, it will be much more (empirically, about 20x longer, but it depends on too many factors such as the server load, filesystem, available memory, database server configuration and so on).

Let me give you an example. I have a site which backs up in 14 minutes, minus a table with about a million rows. If I include that table the backup time is almost an hour. The backup size only grows from 680Mb to 850Mb. Ouch, right?!

The other objective problem you have is that you are using Google Drive which is consumer-grade storage, really made for small files. It's  S L O W. About five to ten times slower than uploading to Amazon S3 according to my tests using both my 100Mbps internet connection and the 20GBps connection of a Linode instance which receives no traffic (therefore, has the entire bandwidth available to it for uploading backups during my benchmarks). This time increases approximately linearly with the backup size — it's not quite linear, it's a slightly exponential curve due to the fixed time per request to upload a chunk of the file, but linear is a good approximation. I would expect 3.3Gb to take hours to upload given that my ~70Mb test takes about 5' to upload on a good day.

Restoration is equally problematic. When you are restoring a database table with millions of rows the restoration time is again O(N2) since MySQL has to rebuild indices as the restoration progresses. You can't do much about it except throw tonnes of RAM into the problem (but it's an expensive solution…). This is further compounded by the constricted I/O and burst-only vCPU of affordable Lightsail instances. Sure, yes, you can get a dedicated instance with plenty of RAM and fast I/O but that's an absolute overkill, not to mention extremely expensive. Note that this is an inherent issue with restoring big tables. It doesn't matter if you use ANGIE or the command-line mysql tool. The runtime will be approximately the same (the command-line tool is about 5%–10% faster but you'd need to pre-process all 20+ thousand SQL files as per the Unorthodox: Emergency Manual Restoration instructions in the documentation, which means the process would take you three to five times longer than using ANGIE at best).

What you should consider, instead, is whether you need to back up all of 4SEOs tables, or whether you actually need 4SEO — Joomla 4 is very good at producing SEF URLs and managing microdata out of the box. Post Joomla 1.5 I have yet to see a correctly build site which needs an SEO component. Proper use of hidden menus for URL structure and user-visible menus with aliases, sensible use of menu items and categories, things like that will create a good URL structure (not that it matters anymore; search engines have been ignoring URLs for search indexing since 2011). Microdata is perfectly manageable in core Joomla. Security is best handled with a dedicated security extension like Admin Tools. Something to consider about; I won't pretend I know your use case well enough to give you authoritative advice on which or what kind of extensions to use.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

trogladyte

Thanks for an excellent, and enlightening, explanation. I do have a fairly minimal RAM configuration on my server, as, 99% of the time, based on past data, it provides more than enough computing power. However, it seems that the addition of a bit more may help my slow backup issues. Really, there's only about 4 site that will benefit and 2 of those are set up with CRON jobs and back up without my intervention, so it's really just 2 that are an issue.

In regards 4SEO, that's something that iI'm stick with. It was an issue from long ago that I could never nail down, and I had a 3rd party fix it. They used (back then) sh404SEF and it DID fix the issue. When upgrading to J!4, I figured out what was causing it (better late than never I guess) and removed sh404SEF. Alas, this has seriously and negatively affected the sites search rankings, so I'm having to put 4SEF (not 4SEO) back on the site and restore the thousands of sh404SEF entries = BIG DB!! :-) So the bottom line is I need those entries back in the DB and I'm stuck with them. Going forward, I could exclude 4SEF tables from backup and it may not make a big deal as long as, from time to time, I do a bakup including them. Perhaps set up a CRON like the other 2 sites using a different profile.

nicholas
Akeeba Staff
Manager

When you add a bit more memory remember to tweak the MySQL settings to slightly increase the query cache size — it makes a big positive impact when you have repeat visits to the same page over a fairly short amount of time.

The rest of the RAM will be used as disk cache which does speed up operations, not only because RAM is faster than the fastest SSD (the recently discontinued Intel Optane notwithstanding), but also because you will be substantially decreasing your server's iowait and kernel cache pressure (a.k.a. “memory pressure”).

Regarding your search engine rankings, do remember that the URLs generated by sh404SEF / 4SEO are different than the URLs generated by Joomla's URL router. This has to do with the fact that the router needs to be idempotent, i.e. taking a non-SEF URL, building it to a SEF URL, and parsing that SEF URL back to a non-SEF URL needs to result in the same non-SEF URL we started with. Sounds straightforward but given the fact that Joomla's menu structure affects the way URLs are build (it's not merely cosmetic as is, for example, in WordPress) you get a lot of devils in way too many details. Ask me how I know… But I digress.

The point is that when you move away from a SEF component you need to map your site's URLs and create redirection rules. Hint: you can very easily handle these URLs in Excel, come up with a spreadsheet structure identical to the #__admintools_redirects, export that sheet to CSV and import it to the #__admintools_redirects table using phpMyAdmin. That's how I have managed to not lose any search engine ranking benefits despite the number of times I redid this site's menu structure over the course of 16 years (same for my blog, by the way, which goes into its 18th year now).

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Support Information

Working hours: We are open Monday to Friday, 9am to 7pm Cyprus timezone (EET / EEST). Support is provided by the same developers writing the software, all of which live in Europe. You can still file tickets outside of our working hours, but we cannot respond to them until we're back at the office.

Support policy: We would like to kindly inform you that when using our support you have already agreed to the Support Policy which is part of our Terms of Service. Thank you for your understanding and for helping us help you!