Support

 Sorry this is I know a long support ticket.

I am about to a launch a new Joomla component and as you have with your Akeeba components I decided to not use your LiveUpdate process but instead use the core Joomla Installer Update process in 2.5 and 3.x. I understand and agree with the desire of end users and the logic of having all updates in one place.
However, I have found several problems with this approach when trying to integrate it with Akeeba Release System. In fact most of these are really just the way Joomla Update works but I have raised them to see if you have enconuntered them and whether there are any known workarounds.

I started off by using a publicly accessible download (from ARS so no downloadid) but this is in a secure area of S3 for which I gave ARS the Access/Security keys. I have seen the tickets ( specifically 19712 Live update folder?) about modifications to allow passing of downloadid's in 2.5.19 and 3.2.3 and will add as required.
But the first component is to be publicly available on ARS - albeit I will try and limit it to only being seen after people register on the site but if people get the direct link that's ok.

The extension.xml file is stored as a public file on S3 and referenced in the updateservers of the component's install xml file (comparch.xml is the extension xml and is attached as comparch.txt). This all works fine after a little fiddling with <folder> and <element> and I can see my avaialble updates.

It is in the actual Update process that I have encountered problems.

I have only traced these problems thus far in Joomla 2.5.19. In Joomla 3.2.3 I haven't had a chance to trace through the update process, I just get an error page with 'An error has occurred' and then just 'malformed'. I thought I'd see what the J 2.5.19 problem was and see if fixes for that solve the problem in J 3.2.3 as well.

First Problem - Joomla Installer Update sometimes has an empty User Agent string

When I first tried to combine Joomla update with ARS I was getting an error "Forbidden - you don't have permission to access /component/ars/ on this server." (See Problem1.jpg attached)
I checked all my permissions and then made everything public, checked my extension xml file, and even added hardcoded download ids and non-sef urls into the extension xml file. The extension xml files are public files on S3 which are created from the ARS update streams.
Nothing seemed to get over this problem with anything I did in Joomla, ARS or S3. I then noticed that in the raw access logs the requests from the Joomla update process had no user_agent.

I have this set in my htaccess, on the site where ARS is, as a rewrite condition which will fail (RewriteCond %{HTTP_USER_AGENT} ^$ [OR] ). So easy enough to solve as I just commented this out.

I haven't fully traced why this occurred in Joomla Update as I believed it should be generated as something like the version and installer e.g. Joomla!/2.5.19 Installer/2.5.

Problem: Joomla Update not always sending requests with a User Agent.
Solution: Temporary Fix - remove htaccess conditions. Permanent Fix: Joomla Update to ensure a user_agent is always sent.

Second Problem - Joomla Update not recognising the file type downloaded

After fixing the htaccess I was still getting a problem in that messgages saying 'Unknown Archive type, Update path does not exist' and 'Error updating COM_INSTALLER_TYPE_TYPE_.' were the repsonse to an Update of my component (see Problem2.jpg)

Again I double checked all permissions and the set up of the extension xml. Read all the Support tickets on Akeeba and checked documentation. But nothing solved this so I Debugged the code.

The problem was occuring in the Joomla Installer (libraries/joomla/installer/helper.php in downloadPackage function)

Address being provided for download was the correct one from the extension xml file on S3 i.e.:

"http://www.componentarchitect.com/index.php?option=com_ars&view=download&id=4&dlid=&dummy=my.zip"

This returns download response of:
code: 302
location: "http://liveupdate.componentarchitect.com.S3.amazonaws.com/componentarchitect/v_1_0_1/com_componentarchitect_v_1_0_1.zip?AWSAccessKeyId=AKIAJGEBIQVAVCGBRX7A&Expires=1397299058&Signature=LVMmso8cPn6jSnFMfybnw815Xfc%3D"
body: empty

This redirect is then handled and a new call to the downloadPackage function is made with this S3 url

The download from S3 returns a download response of :
code: 200
Headers: Seem to point to correct file based on Last-modified, size and type
body: The contents of the zip file are returned

This then tries to save the body to a tmp file for which the following url is derived:
"E:\xampp\htdocs\j_2_5_19_test\tmp\com_componentarchitect_v_1_0_1.zip?AWSAccessKeyId=AKIAJGEBIQVAVCGBRX7A&Expires=1397298166&Signature=9sUnGTPQQ7urTWNoSARssDzLoEQ%3D"

The JFile:write of the body to this address fails but this is not trapped by Joomla Update. So when the extract archive process runs on the file, which has not been created, it fails and gives the messages shown in Problem2.jpg

Problem: Joomla Update is not stripping the S3 security details from the file name when it is trying to save to tmp. I have verified this is the problem by, in debug, adjusting the name by removing the S3 security and the file is created, extracted and the update is completed.
This problem occurs after the onInstallerBeforePackageDownload event is invoked so cannot use a plugin to clean this up.
Solution:
Temporary Fix - I will consider not using S3 for storage and host the files on my server which will remove the security string from the url. But for those that I want secured i.e. Professional subscribers - is there any way of securing this on my server with ARS?
Permanent Fix - Not sure Joomla Update should, or anyone would want it to, handle this. At the moment it is just not designed to be used with S3 unless a public file. Perhaps a solution is to have Akeeba Release System cache a copy of the S3 file on its server and serve that to Joomla Update?


Third Problem - Response from ARS when a protected release/item is accessed with the wrong or missing download id

As a first step to my final desired ARS configuration I added a category/release and items for a protected (i.e. will be a paid for) component (this is protected by it being in a Subscribed access level and only available to Professional ARS subscription levels). I have not included the code to handle the download id in 2.5.19 and 3.2.2 just thought I'd see what would happen without it.
But immediately see a problem in that even if I added that code then if the download id is not entered in Options or it is an incorrect one (password change by user on my site would generated a new download id) then the response sent back to the Joomla Update is a 403 message and this serves a 403 page which then corrupts the view (see Problem3.jpg)

Problem: this is working correctly in terms of rejecting the Update but corrupts the view and is not good error processing
Solution:
Again a difficult one as the access to the Release/Item is controlled by an access level, then Joomla (on my component selling site) detects this and sends the 403 page.
Should I make the access level on the component Category/Release/Item level 'Public' and allow control only by subscription to a Professional Subscription level? Will this send a better response or a 403 error page?
Should Joomla Update handle this by detecting the response and not putting out a 403 page mixed with the update administration page?
Have some 403 response which changes depending on whether it is a download request from ARS or not?


Have you encountered any of these problems yet?

Overall maybe Joomla Update is still not sophisticated enough for handling S3 AWS storage and paid for/restricted access components, even with the fixes you got included in J 2.5.19 and J 3.2.3.

Perhaps I should use LiveUpdate for now? Although I also intend to have updates to plugins for the component which can be separately installed and these would be for paid for Subscription Levels only. I couldn't see a way of doing individual plugins using LiveUpdate. Is that possible?

As I said at the start these are not necessarily things that should lie with Akeeba Release System but if using Joomla Update is going to be the way you do your own components, and recommend to others, I'm sure you will want to identify solutions, if you haven't already to the above problems which I think will be common to other users.

I have tried to be thorough in this description but if you need any more info or access to my sites let me know.

Thanks for you excellent components which I am using extensively in my site.

I'm afraid I don't think I have over thought this.

> First Problem - Joomla Installer Update sometimes has an empty User Agent string
I agree it is not an ARS problem. I raised it as other people may be trying to use Joomla Updater in conjunction with ARS.
In general I think it is a good rule to reject via htaccess any request for which there is no user_agent i.e. I view this the same as a Bad Agent. Naturally, others may not see this as a Bad Agent.
Joomla Updater does actually try to store a user_agent by doing an ini-set but I don't think this gets used in the actual HTTP request.

I will raise this via the Joomla Forums.

> Second Problem - Joomla Update not recognising the file type downloaded

If you look at the comparch.txt file in the original post (which is my comparch.xml which you can also see as a public file in my S3 bucket at: http://liveupdate.componentarchitect.com/componentarchitect/comparch.xml) you will see that I have included '&dummy=my.zip' at the end of all downloadurls and in fact this file is a direct copy of the xml generated by the ARS update stream with default settings i.e.
Extension Type = Components
Site area (client_id) = Backend
Folder (for plugins) = left blank as not a plugin

I completed Stream Name, Alias, Category, Package naming pattern and element.

Note: I do not populate the 'dlid=' value as for this component it is not restricted to a particular Subscription level later ones will be. But in case that was a problem then in my testing I hard coded a value in to the 'dlid' in teh comparch.xml file and this did not change the problem.

As I described in detail in my original post the downloadurl in this does correctly link to ARS on my server and ARS responds with the url for the download file on a protected S3 bucket i.e. "http://liveupdate.componentarchitect.com.S3.amazonaws.com/componentarchitect/v_1_0_1/com_componentarchitect_v_1_0_1.zip?AWSAccessKeyId=AKIAJGEBIQVAVCGBRX7A&Expires=1397299058&Signature=LVMmso8cPn6jSnFMfybnw815Xfc%3D".

It is this which causes the error because it cannot be saved in the tmp directory. Yes this is because it does not have a valid file type suffix.

If you are saying I should add '&dummy=my.zip' to the end of this then OK can you explain where I set this within ARS.
'http://liveupdate.componentarchitect.com.S3.amazonaws.com/componentarchitect/v_1_0_1/com_componentarchitect_v_1_0_1.zip' is my file and this is set in the ARS Item for the release.

'AWSAccessKeyId=AKIAJGEBIQVAVCGBRX7A&Expires=1397299058&Signature=LVMmso8cPn6jSnFMfybnw815Xfc%3D".' is the security details appended by ARS from my S3 Options.

How can I add the '&dummy=my.zip' to the end of this?

>Third Problem - Response from ARS when a protected release/item is accessed with the wrong or missing download id

I entirely agree that 403 is the correct response. But my point was that this is again a problem within Joomla Updater as when it receives the 403 page it just tries to display it within the Joomla Update page but the display of this 403 page corrupts the page as shown in the image I sent (Problem3.jpg). It works in a sense but it is definitely not pretty

I was again raising this so that other users of Joomla Update with ARS may see this ticket and recognise a problem they may be having.

Again it is something which I can raise on the Joomla Forums but the corrupted page I think is something specific to the combination of Joomla Update with ARS. Normally Joomla Update would go straight to a file and either get the file or a HTTP response code (403, 404 whatever). It is not expecting to go to a server process which could and does respond with an error page not just a file or a HTTP response code.

I would appreciate if you could look one more time at this, particularly my Second Problem and explain, if I am incorrect, where I need to add the '&dummy=my.zip'.

Note: I have looked at both code for Joomla 3.2.3 to update extra_query and the plugin code for Joomla 2.5.19 but cannot see anywhere where the '&dummy=my.zip' can be added to the response from ARS. I have also looked at your update streams on your cdn for all versions of your software. I understand where these are free and therefore direct links to files and where they are restricted and accessed via ARS and have a '&dlid=' and '&dummy=my.zip'. I see no difference between these and what I have used.