Chapter 6. Information for removed or canceled features

Sometimes, despite our best intentions, some features prove to either be far less than ideal or impossible to create in a way that makes practical sense. The reasons behind the decisions to remove or cancel features may not be obvious. These articles try to shed some light.

Microsoft OneDrive for Business

This article was written June 26th, 2018.

Executive summary

Executive summary: Microsoft's convoluted and self-contradicting documentation promised that this feature would be possible and it was (mostly) working when we released it. Soon after they removed this functionality and required the use of a different API to access OneDrive for Business. However, the new API is rate-limited, meaning that you can only make a handful of API calls every ten minutes. This makes it impossible to upload backup archives in small chunks to prevent your server from timing out. Therefore the new API is unsuitable for the purpose of storing backups. We had no option but to cancel this feature.

We might reconsider this decision if Microsoft Graph lifts the API rate limiting or increases it to practical levels.

Technical information

Despite the deceptively similar name, Microsoft OneDrive for Business (a feature of Office 365 for Business, let's call it 1DB for short) has nothing to do with Microsoft OneDrive (the consumer storage solution, let's call it 1D for short). The differences are in both authentication and API.

1D works with any individual Microsoft account. These accounts can be opened free of charge with just an email address. If you had a Hotmail, or Skype account it's already a Microsoft account. As part of your Microsoft account you get free 1D storage, no matter if you want it or not. If you buy a personal Office 365 subscription (NOT the Business one!) you get even more space on your 1D.

An application can talk to 1D by authenticating the user against the Microsoft Account using OAuth2 and using the returned token to access the OneDrive API (formerly: Windows Live API). This is straightforward and we have supported this since several years ago.

1DB is an entirely different beast. You cannot get it for free. Your organization or school (the Organization) needs to purchase an Office 365 For Business subscription. These come in different tiers. All tiers but the cheapest one allow the Organization's users to have 1DB storage.

The authentication for 1DB storage does not happen against a regular Microsoft account. Since 1DB is part of the Organization's Office 365 For Business plan the authentication has to happen against the Organization's accounts. These are special Microsoft account which only work within the Organization. The Organization is referred to as a "tenant" in Microsoft's documentation.

Regular OneDrive applications are not aware of "tenants". Therefore they can only authenticate regular Microsoft accounts through OAuth2. Now starts the real confusion. Microsoft's documentation indicates that there are two and a half ways to overcome this.

The first way is to create a Microsoft Azure Active Directory application for the tenant. This means that for every organization which would like to use Akeeba Backup or Akeeba Solo we would need to create a new authentication endpoint on our server and a different upload engine, maintaining them forever. This is impractical. If we had, say, 100 organizations using our software we'd need to create 100 different packages for each version of Akeeba Backup / Akeeba Solo and create 100 update documents on our servers, maintain 100 different authentication endpoints and so on. This would require us to charge more 4,000 Euros per year for these clients, not 40 Euros per year, and have an army of code monkeys to maintain these integrations. This makes no economic sense, primarily for our clients: using Amazon S3 or BackBlaze or anything would still be cheaper for them!

The one and a half way is to create a multi-tenant Microsoft Azure Active Directory application. Following Microsoft's documentation it's absolutely clear that this no longer works. In fact, Synology seems to be using the same solution because when I (Nicholas, the lead developer) tried to use their OneDrive backup solution on my Synology NAS I got the same error I was getting when I was trying to do authentication with this method. I guess if a multi-million-dollar company can't figure this out either I shouldn't feel bad about my developer skills... But I digress. I was wondering why this happens. Digging around I figured it's because...

...Microsoft now offers a second way, creating a "converged app" on the same platform that the old, regular-Microsoft-Account-only apps were created. Well, OK. That's what we did.

The other problem we have to solve is which API we can use to talk to 1DB. Microsoft claims we have two options.

First, they offer the OneDrive API (formerly Windows Live API) which is the traditional way to talk to OneDrive. This is what we use for our consumer OneDrive integration. According to Microsoft we can use this to access 1DB as long as we have the special endpoint for the tenant our user belongs to.

The second option is Microsoft Graph which tries to integrate all Microsoft services' APIs under a unified umbrella. It can be used not only to upload stuff to 1DB but also give us the all important information we need for the OneDrive API: the special endpoint for the tenant of the user we just authentication.

So, following Microsoft's convoluted and self-contradicting documentation we have now a token we can use with Microsoft Graph API to get the OneDrive API endpoint for the tenant of the user we logged in. That works and an endpoint is indeed returned. This endpoint is then used with the OneDrive API to talk to the user's 1DB storage.

The latter part worked when we launched the integration. A month later we started receiving reports that the integration was not working. Indeed, creating a fresh authentication token no longer worked but the old authentication token we had created a month earlier worked!

That's of course because Microsoft changed something in their APIs AGAIN. Each "converged" application has two sets of permissions, "delegated" and "application". Delegated permissions are given to the application when a user logs in and cannot exceed the permissions the user has. These are permissions like "create files", "read the contents of files" and "let me store the token so I don't have to ask the user before every action". Application permissions must be given by an administrator of the tenant and are applied regardless of who actually logs in. Microsoft's documentation explains that we should be using delegated permissions with the OneDrive API and indeed that used to work. What happened recently is that the "legacy" OneDrive API was applying Application permissions, not Delegated permissions, at least for new tokens. This meant that we had to make a change in our "converged application" but these changes only applied if an administrator re-authenticated our backup software.

In short: if you were given an 1DB account from your organization or school and you were not an administrator there you could NOT use it. This is not an integration that makes sense; we couldn't release that crap and claim that we support OneDrive for Business! So we tried to find a solution.

Enter day 2 of debugging (that's on top of 3 days of developing the feature in the first place).

Remember Microsoft Graph API? Microsoft is trying to shove it down developers' throats so we said what the heck, let's try it. The first hurdle is that Microsoft's documentation for the Graph API is wrong. The information they claim is returned is not. The parameters to call it are not what they document. So, implementation has to be the same as with all other Microsoft's APIs and their shoddy documentation: trial and error. Here comes the second hurdle. Microsoft Graph API is rate limited. You can only do about 5 requests every ten minutes. At this point it was very clear that this meant two things. First, developing anything would take weeks just because the documentation is wrong and trying to figure things out by educated guesses would require looooooooooooooooooooooooooooooooooong breaks between actions. Long enough to make a developer lose his train of thought and have to start over, adding an overhead of 20' every time. Second, uploading a backup archive in small chunks to prevent a server timeout requires much more than 5 requests in 10 minutes. So this method would fail if we were to use it for storing backups.

So, it ends up that Microsoft has made it so that OneDrive for Business is impossible to use efficiently for storing backups UNLESS you concede that only organization administrators should be able to set up OneDrive for Business as backup storage. This is absurd. Also, Microsoft's behaviour the last 10 years shows that their APIs change drastically every year, making this a very expensive proposition both for us and the businesses using our software. At some point we will drop old Joomla and PHP versions. If your site still runs on them, using an older version of our software, and Microsoft changes their APIs -which they definitely will!- you are out of luck.