Support

Akeeba Backup for Joomla!

#37252 verification and intelligent backup

Posted in ‘Akeeba Backup for Joomla! 4 & 5’
This is a public ticket

Everybody will be able to see its contents. Do not include usernames, passwords or any other sensitive information.

Environment Information

Joomla! version
4.1
PHP version
8.0
Akeeba Backup version
9.2.4

Latest post by nicholas on Friday, 10 June 2022 01:47 CDT

HDcms

Hello

I now automate akeeba backups with yourSites. It works well and creates backups.

1/ I want to ensure an automatic verification of the integrity of the backups. Is this possible ?

2/ "Smart" backup
I currently use duplicati for my personal backups. It has a smart backup strategy "Backups will be automatically deleted. There will be one backup left for each of the last seven days, each of the last four weeks, and each of the last twelve months. There will always be at least one backup."
Is it possible to set this up without creating as many backups as a method?
is it possible now? or is it planned? an article on this?

Regards

nicholas
Akeeba Staff
Manager

1. The only reliable way to determine that is doing test restorations. There's BackupMonkey.io which can do that. Or you could do a test restoration once a week or once a month (depending on what is considered an acceptable possible loss for you).

You could of course automate that on your own box with UNiTE β€” that's what BackupMonkey does β€” but I assume that if you are asking this question you don't know how to set up your own local test server and CRON jobs to run UNiTE.

2. You can do something like that but it's kinda tricky. You need three backup profiles:

  1. Runs on the 1st of each month and has a count quota of 12. This is the monthly backup you keep for a year.
  2. Runs each Sunday except the 1st of the month and has a count quota of 4.  This is the weekly backup you keep for four weeks.
  3. Runs every day except Sundays and the 1st of the month and has a count quota of 7. This is the last 7 daily backups.

It makes more sense to back using day-based quotas and count quotas. You can set up the day quotas to keep the last 30 backups, always keep the backup on the 1st of the month and set the count quotas to 42 (30 daily backups + each of the 12 months). This is more backups than what you said but at least it's automated with a single backup profile.

On our main backup profile I do 30 daily backups and keep the one from the 1st of each month, uploading everything to Amazon S3. I have set up lifecycle rules on the S3 bucket for backups over 3 months old to be set to Intelligent Tiering, backups over 6 months to be moved to Glacier, backups over 12 months to be moved to Glacier Deep Archive and backups over 10 years old to be deleted. The idea is that I will very likely need a hot backup from the last 3 months to restore as a dev site, backups 3 to 6 months are less likely to be restored e.g. to see the state of the site because someone deleted their account and they contacted me about a subscription they had, backups between 6 to 12 months are extremely unlikely to be needed for restoration (only happened once in 12 years) and the rest 9 years is what I am obliged by law to keep as a proof that transactions involving money changing hands really did happen in the way described in the company's books (the law says we're obliged to keep at least 7 years and the tax office guidance is ideally 10 years because a legal case could move slowly enough to warrant the additional three years; beyond a decade the statute of limitations has expired).

The S3 transitions are explained in https://docs.aws.amazon.com/AmazonS3/latest/userguide/lifecycle-transition-general-considerations.html. Do note that they are based on the object (file) age in days, not the date it was created. That is to say, you can't tell it to keep weekly backups or monthly backups. All transitions have to be about the age of the file in days. So what you are trying to do cannot be implemented. You could, however, do the following:

  • Set Akeeba Backup to take a daily backup and send it to Amazon S3. Enable remote quotas, enable the day quotas for 30 days and keep the 1st day of each month.
  • On the S3 bucket set the following transitions:
    • After 30 days: move to S3 Standard-IA. There's a minimum of 30 days Standard storage requirement on S3. So, your last 30 days will be your hot backups. Then your monthly backups for the past 3 months will be nearline but not hot.
    • After 90 days: move to S3 Glacier Instant Retrieval. Your backups from 3 months ago to 6 months ago will need a few minutes to be retrieved. They are in your cool storage.
    • After 180 days: move to S3 Glacier Deep Archive. Your backups from 6 to 12 months ago will take a few hours to be retrieved. They are your cold storage.
    • After 365 days: expire (delete). You only want to keep up to one year of backups.

This is a good policy and rather cheap. With 1GiB backups stored in Amazon's Paris datacenter you'd be paying under $10 a year (pricing: https://aws.amazon.com/s3/pricing/). I think that paying less than the price of a large pizza and cola per year to be able to sleep at night without worrying about data loss is well worth it!

Nicholas K. Dionysopoulos

Lead Developer and Director

πŸ‡¬πŸ‡·Greek: native πŸ‡¬πŸ‡§English: excellent πŸ‡«πŸ‡·French: basic β€’ πŸ• My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

HDcms
Hello
Thank you for this answer always precise.

I understand and give up the idea for 1/ because it is indeed out of reach of akeeba except ... to set up such a service ;-)

2/ I am disappointed. It's not only the low cost announced (even if I don't like GAFAM), it's mostly that it seems heavy and complex to duplicate on x sites. I am a small webmaster who just tries to have some good practices.

My only hope is that other subscribers to akeeba ask you the same thing as me and that you improve something simple equivalent. perhaps an option of the configuration wizard with a setting in 1 click :-)
In short implement a great smart backup, but I know it's not won!

regards

nicholas
Akeeba Staff
Manager

About 2, I already gave you a solution to do what you are asking? Let me paste this again since you seem to have missed it.

You can do something like that but it's kinda tricky. You need three backup profiles:

  1. Runs on the 1st of each month and has a count quota of 12. This is the monthly backup you keep for a year.
  2. Runs each Sunday except the 1st of the month and has a count quota of 4.  This is the weekly backup you keep for four weeks.
  3. Runs every day except Sundays and the 1st of the month and has a count quota of (I made a typo earlier saying 7). This is the last 7 daily backups.

All of that is perfectly possible if you are using CRON or Joomla's Scheduled Tasks to take your backups.

Each CRON entry has a setting for running on a specific Minute, Hour, Day of Month (DOM), Month and Day of Week (DOW). Let's say you want to run backups at midnight.

First profile: 0 0 1 * *
Runs on Minute 0 and Hour 0 of the 1st day of the month, whatever the month, whatever the day of the week.

Second profile: 0 0 2-31 * 7
Runs on Minute 0 and Hour 0 of the 2nd to 31st day of the month (therefore NOT the 1st), whatever the month, only the 7th day of the week (Sunday)

Third profile: 0 0 2-31 * 1-6
Runs on Minute 0 and Hour 0 of the 2nd to 31st day of the month (therefore NOT the 1st), whatever the month, only the 1st to 6th day of the week (Monday through Saturday)

This is exactly what you asked for and it's perfectly possible. Are you disappointed it can be done?! It's what you asked for! I don't understand.

Nicholas K. Dionysopoulos

Lead Developer and Director

πŸ‡¬πŸ‡·Greek: native πŸ‡¬πŸ‡§English: excellent πŸ‡«πŸ‡·French: basic β€’ πŸ• My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

HDcms

Hi

Ah, I knew it was going to be difficult!
I read perfectly your proposal, I had thought of a similar solution.

I just said that I wanted an improvement of akeeba with an option ... like "wizard" ?
to automatically generate either a backup profile ? or 3 profiles of backups ? to avoid doing this manually on x sites.

If I expressed myself well, I hope you understood but I guess you do not want to add this new feature since you proposed a manual solution.

Thanks

Regards

nicholas
Akeeba Staff
Manager

We can't possibly have a wizard as it's not possible to create CRON jobs from inside PHP code. So it'd be a wizard to do... what? Try to divine your backup strategy to create profiles which you need to figure out how to run at the right dates and times anyway? That's a useless feature. You can do it much faster by clicking a button to duplicate a backup profile, another button to edit it, change one quota setting, Save & Close. Five clicks in total. A wizard can't make this any faster. The hard part is figuring out the CRON jobs and that's the part we objectively can't help.

Moreover, the overwhelming majority of our clients use the existing quota management features and they are happy with them. Having multiple day limits is a far cry from being a common use case.

Nicholas K. Dionysopoulos

Lead Developer and Director

πŸ‡¬πŸ‡·Greek: native πŸ‡¬πŸ‡§English: excellent πŸ‡«πŸ‡·French: basic β€’ πŸ• My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

HDcms

HI

ok I understand that the majority of our clients use (only) the existing quota
I've been doing it for years and was looking to improve it!
Yes for me I was not attaching the wizard to the cron creation. Let the client put it on a cron or not. By the way I use yoursite which handles backups well.

My suggestion was to generate 1 or more profiles easily and quickly (useful when you are the webmaster of several sites), with the idea of having a smart backup, even simple, week, month (or more complicated with backup in the cloud).
Thank you for taking the time to study my proposal and propose your more complete solution. If my idea is not good, I will look for another solution!

Question: I can automate the creation of a backup before updating joomla either by
a) your plugin
b) the yoursSites monitoring extension
A preference to use a) or b)
Regards

nicholas
Akeeba Staff
Manager

Either way you choose you are using Akeeba Backup to take a backup of your site before updating. My preference is that you do take a backup. Whether you do it with the Backup on Update plugin, a third party service using the Akeeba Backup JSON API, manually from the backend, using CLI commands, or using the frontend backup URL is inconsequential and ultimately up to you. We give you plenty of options to find one which works with your use case and your personal preferences.

PS: Never ask a woman her age, a man his salary, and a developer their preference for the best user interface. Developers universally agree that the best user interface is the command line. In my ~/.bash_profile I have this:

j4update() {
php ./joomla.php core:check-updates && php ./joomla.php akeeba:backup:take && php ./joomla.php core:update
}

So for me, updating a Joomla 4 site is this:

cd <path to my site>/cli && j4update

I can even automate it across all sites in a folder with

for d in `find $PWD -type d -depth 1`; do pushd $d; j4update; popd; done

This is not what I expect you to do. I expect you to use the GUI. The command line is for developers and systems administrators. It's not for most users. So, please, don't ask me my preference because my use case is so far removed from your use case that my preference does not apply to you :)

Nicholas K. Dionysopoulos

Lead Developer and Director

πŸ‡¬πŸ‡·Greek: native πŸ‡¬πŸ‡§English: excellent πŸ‡«πŸ‡·French: basic β€’ πŸ• My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Support Information

Working hours: We are open Monday to Friday, 9am to 7pm Cyprus timezone (EET / EEST). Support is provided by the same developers writing the software, all of which live in Europe. You can still file tickets outside of our working hours, but we cannot respond to them until we're back at the office.

Support policy: We would like to kindly inform you that when using our support you have already agreed to the Support Policy which is part of our Terms of Service. Thank you for your understanding and for helping us help you!