Support

Akeeba Backup for Joomla!

#36160 Large backups with jo1, jo2 etc not completely removed

Posted in ‘Akeeba Backup for Joomla!’
This is a public ticket

Everybody will be able to see its contents. Do not include usernames, passwords or any other sensitive information.

Environment Information

Joomla! version
n/a
PHP version
n/a
Akeeba Backup version
n/a

pwouda

Hi there,

It looks like not all parts of a large backup are removed. I have set the max number of backups to 3, but only the jpa file is removed. The j01 and j02 files remain on the server.

It is not visible in "Manage backups" but with ftp I still can see the j01-files.

Best regards,
Peter

nicholas
Akeeba Staff
Manager

I cannot reproduce this issue.

The size of parts does not play any role. So that's something we can rule out.

I will have to ask a few questions, though, to help me get more context into what is going on:

  • Is this a backup profile configured to transfer the backup archives to remote storage?
    If yes, please answer the following questions too:
    • Did the entirety of the backup archive (all parts) transfer? (If not, you get a β€œTransfer to XYX” button in the Manage Backups page instead of β€œManage Remotely Stored Files”)
    • Have you set up the profile to remove the part files from your server upon successful upload?
  • Before running a backup, do all of the part files (.jpa, .j01, .j02, ...) of the backup archive exist or is anything missing?
    • If something is missing, which one is it?

 

Nicholas K. Dionysopoulos

Lead Developer and Director

πŸ‡¬πŸ‡·Greek: native πŸ‡¬πŸ‡§English: excellent πŸ‡«πŸ‡·French: basic β€’ πŸ• My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

pwouda

Hi Nicholas,

The files are indeed uploaded with ftp to a remote server.

As I can see now, not all parts are uploaded: only two (Β  jpa and j01 ) of the total four (jpa, j01, j02 and j03) are transfered. The parts are 2GB each, except for the jpa-part.

I do have enabled the option "delete archive after processing", but that is for the local files and not the remote files I guess? If I disable this, I get an Gateway timeout error.

I also tried "process each part immediately" but that did also not work out.

Before running a backup, al the part files of the previous backup exist. So I gues before post-processing all parts are available.

Might be a time-out problem?

PS: the backup is started by the MySites.guru service, but I tested it for this post in the back-end of Joomla.

Regards,
Peter

nicholas
Akeeba Staff
Manager

OK, this explains what is going on.

The deletion algorithm goes like this. We take the filename and remove the last two letters, i.e. foo.jpa becomes foo.j. Then we set a counter, let's call it j, to 0: j=0. At this point we enter a loop:

  • Add 1 to j. First iteration: j becomes 1
  • Append j to the filename making sure it's always at least two digits. First iteration: foo.j becomes foo.j01
  • If the file does not exists break out of the loop
  • If the file exists delete it and repeat this loop

Finally, we check if the original filename (foo.jpa) exists. If it does, we delete it.

Since the .j01 file is already uploaded AND removed from your server the loop above breaks as soon as j = 1. It never gets to 2 and 3, therefore these parts are never deleted. The last part (foo.jpa) is also not there. Therefore there's nothing to delete as far as we know.

So the problem is not that the files are not deleted, it's that the part files were not uploaded. The reason? You already mentioned it. Each part is 2GB. Go and read the documentation for the Upload to FTP feature. See what it says? Since FTP doesn't generally support continuing uploads we have to upload the ENTIRE file in one go. Uploading 2GB takes a while. At this point PHP has timed out. Therefore it never gets to upload the other part files.

Solution, as per the documentation: use a smaller part size for split archives. In most cases sizes of 10 to 50MB work best. Depending on your server you MAY be able to get away with 100–200MB. Anything bigger is very likely to cause a timeout, leaving stuff behind which causes the other problems you've encountered. Since the timeout is something that has to do with how servers work there's nothing really we can do about it.

 

Nicholas K. Dionysopoulos

Lead Developer and Director

πŸ‡¬πŸ‡·Greek: native πŸ‡¬πŸ‡§English: excellent πŸ‡«πŸ‡·French: basic β€’ πŸ• My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

pwouda

Thank you Nicholas for your complete answer.It makes sense now.

I reduced the file-size, and it works fine now. The backup contains many part-files now, but it restores perfect, so I leave it this way.

Best regards,
Peter

Β 

nicholas
Akeeba Staff
Manager

You're welcome! I am glad I could help :)

Nicholas K. Dionysopoulos

Lead Developer and Director

πŸ‡¬πŸ‡·Greek: native πŸ‡¬πŸ‡§English: excellent πŸ‡«πŸ‡·French: basic β€’ πŸ• My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

System Task
system
This ticket has been automatically closed. All tickets which have been inactive for a long time are automatically closed. If you believe that this ticket was closed in error, please contact us.

Support Information

Working hours: Typically we work Monday to Friday, 9am to 7pm Cyprus timezone (EEST). Support is provided by the same developers writing the software, all of which live in Europe. You can still file tickets, but we cannot respond to them, outside of our working hours.

Support policy: We would like to kindly inform you that when using our support you have already agreed to the Support Policy which is part of our Terms of Service. Thank you for your understanding and for helping us help you!

Summer vacations: Our support will be closed for replies and new tickets from August 6th to August 21st, 2022 due to summer vacations.