UNiTE, Remote CLI, eXtract Wizard

Saturday, 06 December 2008 21:24 CST

user1615

This is an excellent component which has saved me heaps of time and have it installed on each of my 5 websites. I have some features to request.

1. Can the front end backup facility also automatically download the file to my PC? If it could, I would not need to log into each of my websites one at a time and then download each backup file individually.

2. If for e.g. I am logged into my main development site, would be it be possible to backup the current logged in site + all of my other websites automatically all in one go and also automatically download all of these backup files onto my PC?

The above 2 features would save even more time and basically eliminate the need for me to go into my webhosts cPanel, download the entire file system for backup and then extract each website separately if i need to install any of the individual websites for testing or demo purposes. In any case, thank you for this excellent component. :D )

Sunday, 07 December 2008 01:27 CST

nicholas

We just released JoomlaPack Native Tools, a set of Windows native graphical interface applications. One of the applications contained in this package is JoomlaPack Remote which allows you to backup your sites and download the backup files from your desktop without even having to open your browser at all!. Let me show you how easy it is.

First, for each site you want to use with Remote you have to:
[ul]
[li]Install JoomlaPack 2.0 Stable. OLDER VERSIONS WON'T WORK![/li]
[li]Configure JoomlaPack and make sure you can take a backup.[/li]
[li]Download and install plg_jpremote (it's a necessary plugin).[/li]
[li]Go to Global Configuration, click on the Server tab and set Web Services to On. Save you global configuration.[/li]
[li]Go to plugin manager and make sure that plg_jpremote is enabled.[/li][/ul]

After you're done, go back to your desktop and configure the backup process:
[ul]
[li]Download and install JoomlaPack Native Tools.[/li]
[li]Download the JPRemote.pdf as well; it's the manual and you must read it before configuring sites to be backed up.[/li]
[li]Fire up JoomlaPack Remote and set up your sites.[/li][/ul]

All the above have to be done only once.

Next up, backing up all of your sites at the same time and having the backup archives downloaded to your PC:
[ul]
[li]Fire up JoomlaPack Remote[/li]
[li]Press SHIFT-F11[/li][/ul]

That's all. Really. Dead simple? Sure thing! Is this really free? Yeap. Has anybody done this before? Nope, that's why you might run into bugs; in this case just drop us a line.

Enjoy :D

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Sunday, 07 December 2008 10:46 CST

user1615

Thank you Nicholas. I have tried to find and download the tools package without success so far. Could you please let me know where the download link is? Thank you. Cheers, Arsen.

Sunday, 07 December 2008 12:14 CST

user736

http://www.joomlapack.net/download/Joom ... nload.html

thanks

mike

Monday, 08 December 2008 18:59 CST

user1615

Hi Nicholas, I have tried to get the remote component to work and so far without success. The ftp section tests ok however. Is there anything else that i need to know? I have checked with my ISP and my site address is not blocked in any way. They also cannot get it to work. I am certain that i have setup everything correctly. Thank you for your help. Arsen

Monday, 08 December 2008 22:09 CST

nicholas

First of all, check that the Web Services are enabled and the XMLRPC plugin itselft (plg_jpremote) is published as well. Make sure you have installed the 2.0 STABLE version of JoomlaPack can take backups with it.

If all of these preconditions are met, try to setup your site within JoomlaPack Remote and press the "Test Connection" button on the first tab. If it throws any error, please tell me the exact message shown, or take a screenshot of the JP Remote window and attach it here.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Tuesday, 09 December 2008 01:10 CST

user736

Video Tutorials are forth coming. I am having some issues with getting them uploaded to a video host since the file size is pretty big.

Tuesday, 09 December 2008 04:09 CST

nicholas

Michael,

you can downscale them to 320x200, 15fps, 64Kbps audio with the most compressing video format available. There are tons of tools for transcoding video, including the ever popular avidemux. I am mostly a Linux guy, so I preffer using mencoder from the command line. Of course, that's just me ;)

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Tuesday, 09 December 2008 06:30 CST

user736

Nicholas,

I will look into this. I have finals starting so if I am not studying, with my girlfriend lol, packing or taking a final I will try to get this done.

Thanks

Mike

Tuesday, 09 December 2008 21:44 CST

user1615

Hi Nicholas. Thankyou for your help. My issue was resolved by installing revision 2.0 of the component. Not sure how i missed this. Sorry to have wasted your time.

However i have 3 websites working with the remote. One of these 3 websites downloads the backup file but the download never actually finishes, although the horizontal progress line shows that it is finished. The other 2 websites work perfectly.

The 4'th website does not work at all with the remote function. The message displayed is this.......Could not connect to FTP server. Please check your connection parameters. I checked and still no luck.

The setup is exactly the same as for the first 3 websites, so i do not understand why it does not work. I have tried everything that i can think of without success. The 4'th website does however have a Joomla firewall, JUGA, mosets tree and some other components installed. I disabled some of these without success.

I wonder if there is a bug in you PC remote software. Please advise. Thank you.

Tuesday, 09 December 2008 23:22 CST

nicholas

On the third website make sure that your FTP server supports passive connections and it doesn't time out when a client downloads files through passive FTP. What I described is the common configuration used on 99% of commercial hosts, but I can't assume deviations from this rule.

About the 4th website, does this message appear with Test Connection or during backup? If all your sites are on the same hosting account or hosting service, we had some cases where the host would limit the number of connections per account/IP, causing simultaneous downloads of multiple sites to fail. We are working on a fix around this limitation for the next release.

If it's still a no-go, you can PM me a screenshot of your options. Sometimes it's just a matter of entering something more or something less than the software expects in the parameters :)

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Thursday, 11 December 2008 13:06 CST

user1615

Hi Nicholas. My isp tells me that i have 20 FTP connections available with 15 min timeouts, so this could not be the issue. I have tried many times to execute the backup for the website in the attached image without any success. The FTP test in the second tab says that it works. I hope that you can assist. The other websites still work so far. All my websites are under 1 account on a shared server. Thank you for all your help to date. Cheers, Arsen

Thursday, 11 December 2008 19:49 CST

nicholas

OK, since they are on the same account we can rule out network related and service configuration related issues, so the problem has to lie with the settings.

On the site where downloading the backup is impossible, are you sure the FTP directory is set up correctly? It should point to the location of this site's backup archives. The default setting there might not be valid for your site. The best way to find out is use FileZilla to connect to your site, browse to the backups directory and copy the path FileZilla displays above the remote file list pane. This is my fail-safe method :)

On the other site, where nothing is downloaded, are you sure there are no backup errors? It is possible to setup Remote so that it downloads the log file on backup failure, maybe the progress bar refers to the log?

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Friday, 12 December 2008 15:22 CST

user1615

Hi Nicholas. The website that had the issue of a non completed downloaded file seems to be ok now.

I still have the problem of pixelpages.com.au backup not being able to be started. Web services is enabled in Joomla and all other settings are the same as on my other websites. I have attached an image showing what happens when i click the Get Profiles button. The ftp test button in the Downloads tab still works fine.

I am at a loss for a solution. Perhaps there is an issue with the Joomla installation and/or installed components? Do not know what to do now except perform backups after logging into the site and then downloading. Really do want a one button click solution though. Do you have any software updates that might fix the problem? Thank you. Arsen

Saturday, 13 December 2008 06:41 CST

user736

enable web services in the global configure for joomla

Saturday, 13 December 2008 18:01 CST

user1615

Joomla 1.5.8 web services was already enabled. Maybe there is an issue with the Joomla installation. I have run out of ideas for now. Thanks.

Saturday, 13 December 2008 21:16 CST

nicholas

If you have both enabled Web Services on the Global Configuration and published the plg_jpremote plug-in, the only thing I can think of is that a PHP Notice/Warning/Error gets output before the XML-RPC response, giving you the depicted error message.

You can try setting the Error Reporting to None (Global Configuration, Server tab).

If this still doesn't work, make perfectly sure you are using JoomlaPack 2.0 Stable or 2.0.1 on your site. This error is expected to happen if you have earlier JoomlaPack versions installed, up to and including 2.0.b1.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Sunday, 14 December 2008 19:43 CST

user1615

I have checked everything including upgrading to the latest Joomlapack 2.0.1. I am also checking with my ISP in the meantime as the message i get when attempting to execute the backup is ...... Could not connect to FTP server. Please check your connection parameters. Otherwise i am at a loss as to how to resolve this problem. Thanks for your help to date.

Sunday, 14 December 2008 21:17 CST

nicholas

If automatically downloading the backup is not of uttmost importance to you, you can always disable this feature from the editor dialog (the Downloads tab, disable the top two check-boxes).

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Tuesday, 16 December 2008 16:51 CST

user1615

The ftp downloads test seems to work as it displays an..... Connection Established OK when tested. It just does not work for backup commencement. Nor does Get Profiles work. When i click Get Profiles I keep getting this...... Could not connect to server. The error message was: Invalid Payload received from xml-rpc server. When i click on the backup icon i still get.................Could not connect to FTP server. Please check your connection parameters. I have tried disabling some of my components to see the effect and so far no change. No matter what i do, i cannot get it to work. My other 3 sites work without any issues so far.

Tuesday, 16 December 2008 20:49 CST

nicholas

OK, so it's a problem on your site. If you are sure everything is otherwise set up correctly (Web Services is on, the plg_jpremote plugin is installed and published, JoomlaPack is configured and ready to take backups) there's nothing more I can suggest, except turning Error Reporting to None (Joomla! Global Configuration, Server tab). If this is still a no-go, I can't do anything unless you PM me the URL of your site and administrator credentials, so that I can take a closer look.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Thursday, 18 December 2008 02:53 CST

user736

Moving to Native Tools Support since the topic has turned to a support request.

Friday, 19 December 2008 14:42 CST

user1615

It seems that the culprit was the installed component AEC from globalnerd.org which is a very popular subscription management component. As soon as it was un-installed, the Get Profiles and backup execution worked on my localhost. I don't know why though. I have not managed to get the backup file to download automatically however. I do need to use the AEC, so hopefully you can find out the issues causing this. Thank you.

Friday, 19 December 2008 21:55 CST

user1615

In addition, when i re-installed the AEC component, the backup execution ceased to work as before. This proves that the issue lies with the AEC somehow and interaction with jpremote. I hope someone can resolve this. Thank you.

Saturday, 20 December 2008 02:19 CST

nicholas

Thank you for your submission. I will try to take a look at the combination of plg_jpremote + AEC.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Thursday, 08 January 2009 00:27 CST

user1615

Hi Nicholas. Just wondering about your progress if any on the issues with AEC compatibility. Thank you.

Thursday, 08 January 2009 01:57 CST

nicholas

To be honest, I installed AEC and did some very basic configuration (I'd need a lifetime to fully configure it, the available parametrization is almost infinite) but I couldn't reproduce the problem with 2.0.1 or 2.1.b1.

My guess is that since AEC interferes with the Joomla! user authentication, it may be responsible for plg_jpremote not being able to authenticate the user, thus failing. Using a Super Administrator's login credentials should work, though.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Thursday, 08 January 2009 09:40 CST

user1615

Thanks Nicholas. I have found that if i disable the Authentication - AECaccess plugin, the Remote Joomlapack function works correctly. Upon being re-enabled, it does not work. I have tried to exclude it in JUGA, however without any success. Any ideas?

Thursday, 08 January 2009 19:54 CST

nicholas

Um... JUGA is a hack, meaning that it modifies core files. Since plg_jpremote relies on loading Joomla! user authentication libraries it is indeed possible that this hack prevent arbitrary user authentications. Of course, all of this is mere speculation.

I can't really tell what exactly is wrong unless I can reproduce this in a controlled environment. I will try to simulate such an environment on my test servers next week and see if I can dupliacte this.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Monday, 12 January 2009 16:10 CST

user1615

Hi Nicholas. After reading another post earlier, I have today downloaded and tried out the new plg_jpremote-2009 and voila, it works. I am also trying out the other updated files and still need to check the JUGA configuration thoroughly. Thanks very much for your efforts in resolving this issue. It is appreciated. Will keep you updated on this and any other issues. Cheers.

Monday, 12 January 2009 19:15 CST

user1615

Unsure if anyone has suggested this before. Have you considered an automated onsite update process for Joomlapack and remote plugin? Also an user scheduled backup and download onto a PC? Pretty much a set and forget. What do you think?

Tuesday, 13 January 2009 01:07 CST

nicholas

The fix was a bit obscure. It turns out that AEC (and perhaps other authentication modules as well) take for granted that they are going to be used by the Joomla! CMS, not the Joomla! Framework. The XML-RPC server is a Joomla! Framework applications, relatively loosely coupled with the site. One part misconception, one part assumption, the authentication plugin failed because of a bug. So, the new plugin circumvents the site's authentication preferences and forcibly uses the Joomla! User Authentication plugin, no matter if it's published or not. Quite hack-ish, but now it works as intended.

@arjo2000. It is something we are considering, but we haven't decided how to do it yet. Yes, we'll do it, but probably not in 2.1. As far as the user scheduled backup and download, it's what Remote does (minus the scheduling bit). Adding scheduling to Remote is on the top of my TO-DO list for the next version. Indeed, I am trying to provide a set and forget backup solution! :lol:

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Wednesday, 14 January 2009 22:46 CST

user1917

Hi Nicholas, I have tried to get the remote component to work and so far without success. The ftp section tests ok however. Is there anything else that i need to know? I have checked with my ISP and my site address is not blocked in any way. They also cannot get it to work. I am certain that i have setup everything correctly. Thank you for your help. Arsen

I'm quite sure you haven't omitted the http:// part when you insert in JoomlaPack Remote the XMLRPC url.

I say this because I was smashing my head having the same problem :)

m.

Sunday, 08 February 2009 12:46 CST

user1615

I have all my sites working with the latest Joomlapack and with the backup folder locations being outside of public_html. There are however a few points to mention.

1. If for example the backup folder contains 3 backup files and i then move this folder to another location without clearing (deleting) the backup files list within Joomlapack admin, the backup does not work properly. After attempting to backup and download, the backup files list in admin shows approx 20+ files marked as obsolete instead of 3. Joomlapack Remote also displays an ftp transport error on my PC. If i then delete the backup files list, the backup and download to PC works fine first time.

2. I have 5 sites. The backup file download progress bar does not display the progress correctly. Some sites will complete their download and then display idle, but 1 or 2 sites show that they are still downloading with the progress bar partially completed, however the downloads have actually completed.

I hope the above makes sense. Thank you Nicholas for this great component. I do intend to donate very soon. Thanks to others as well for their contribution. Cheers.

Monday, 09 February 2009 00:54 CST

nicholas

1. When you move the backup folder to another location you must also change the JoomlaPack Configuration to reflect the change. Since JP stores the absolute path to the backup files, it's perfectly reasonable that all your previous backups are marked as "obsolete"; JoomlaPack can't find the backup files in the new location. This is a feature, not a bug. Really! I have not tried moving the output directory around, so I'll have to check if it throws an error.

2. It is an issue with the current version of JP Remote. It can't read the backup archive's size. This is being addressed on the next version of Remote. Hang in there, it'll be fixed!

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Monday, 09 February 2009 12:30 CST

user1615

I have noticed another issue. Unsure of what is causing it though. I attempted to perform a restore on one of my sites due a Virtuemart issue. The restoration gets to the point of trying to run kickstart.php and just stops there with an error as follows............

Not Found

The requested URL /installation/index.php was not found on this server. Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request.

The backup file was copied across from its folder outside of public_html and the kickstart.php file is also there as well as the .htaccess file having been renamed to htaccess.bak, but it won't run. I tried running it by typing in the link to the file a new tab, but cannot get it to work. My security setup in php.ini and .htaccess files is quite tight. The restoration used to work before. Any ideas. Thank you.

Monday, 09 February 2009 15:17 CST

user1615

I forgot to mention that i have the GuardXT component installed on the website which is a security component that monitors the sites file and folder permissions settings and other configuration settings. When i tried to restore this site locally on my test pc, i get an error that relates to GuardXT. GuardXT is an excellent component but it seems that there are issues with it when using Joomlapack. I will continue investigating. Cheers.

Monday, 09 February 2009 23:01 CST

nicholas

Normally, at restoration time, the GuardXT component should be inactive. After all, your site is not installed yet, so the component can't be activated. The 404 error is related to either file permissions (making the installation directory 0755 recursively is both sufficient and secure - but only when using JPI3) or mod_rewrite magic in your .htaccess file. However, as long as you say that the .htaccess is renamed, you'd better check the permissions. I guess that the installation directory IS extracted, as you didn't mention any Kickstart errors. Can you please confirm this?

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Monday, 09 February 2009 23:41 CST

user1615

I have noticed that the installation directory is not created after attempting a restore. I also have a .htaccess file in the administrator folder limiting access to only my IP address. Also in this folder is a php.ini file with directives for additional security as follows.

magic_quotes_gpc = On
allow_url_fopen = Off
disable_functions = show_source, system, shell_exec, passthru, exec, phpinfo, popen, proc_open

All my folders are 755 and files 644. I have removed the GuardXT component and its database locations. This removal did not have any positive effect.

What do you think? Thank you Cheers.

Wednesday, 11 February 2009 06:13 CST

nicholas

Removing the GuardXT component should have no effect because, as I told you, the component is not running during the restoration.

You have written that the installation directory is not created after attempting a restore. Which method do you use? I mean, did you try Kickstart or did you try extracting locally and uploading? I have noticed that on servers with messed up permissions, Kickstart may be unable to successfully extract the contents of the installation directory, or extract them as 0 byte sized files. I am still trying to figure this out. In such a case, you can try to extract the backup archive locally using eXtract from the JoomlaPack Native Tools package and upload the installation directory you've just extracted. This should get you going.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Monday, 16 February 2009 11:03 CST

user1615

Thanks Nicholas. I am going to backtrack, starting afresh on one site first and check the restore process thoroughly in case my configuration or something else is affecting this. Will keep you posted.

Thursday, 26 February 2009 14:12 CST

user1615

I am finally back to testing Joomlapack again. Backup operation seem to be fine. I am so far testing restorations onto my own PC using both the Joomlapack Extract Wizard and the kickstart method. The wizard extractor on 2 sites worked fine, but on the third it would not extract properly. I will need to get back to you after trying this out again with the errors. Kickstart on the sites worked to the point of a partial restoration of the database with errors. Each time i tried kickstart a different error was displayed. Once again i will need to document these errors and get back to you. With one particular site which has the ioncube loader in its Administrator folder, the extract wizard fails at the point of extracting SH404SEF file with the same error each time............. IO error 206 when restoring file SH404SEF_AntiFlood_Data.dat. This does not happen with my other sites so far all of which have SH404SEF installed. I know i don't have sufficient info for you at the moment, however will try to provide more accurate error info later. Your comments would be appreciated. Thank you.

Thursday, 26 February 2009 21:19 CST

nicholas

Are you using the eXtract wizard from Native Tools 2009? The older version is known to have severe limitations. Other than that, the latest version of eXtract is as far as I can go. I am relying on Delphi's ZLib implementation, but it just doesn't look too sharp :(

About the partial restoration, pleas note that it is not Kickstart you are referring to, it is the JoomlaPack Installer (JPI) which gets called by Kickstart. Kickstart only extracts the files and performs post-installation housekeeping.

You have to document your errors during database restoration. The most common cause of such errors is using components which use database tables with foreign keys. JoomlaPack is not able to produce a correct restoration order. The latest version of JPI3 tries to address this issue, instructing the database server to ignore missing tables in foreign key references, but it highly depends on the database server.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Support

UNiTE, Remote CLI, eXtract Wizard

#3437 Joomlapack backup and download

This is a public ticket

Environment Information

Support Information