Blog Site of the SamlMan, Connecting Apps in a Cloudy World

o365

I’ve been spending some time lately fooling around the o365 API’s. Frankly, it has been a hugely frustrating experience; I’ve never seen so many documentation gaps and examples and sample code that didn’t work, to flat out wouldn’t compile in a long time.So, once I finally stumbled up on the “right” collection of code that worked to get me the all important access token for the o365 APIs I decided to take a quick detour to see if I could use the same approach to manage my Azure subscription with the Service Management APIs.Turns out you can, and actually do so in a way that leverages both of the APIs together in kind of a weird way.Here’s the story of how it works (with a special twist at the end, you’ll have to read all the way down to see it).

The first step in getting any of this working is to create an application in Azure. In that app, you can grant the rights that it is going to require – to things like Office 365 and the Azure Service APIs.The easiest way to do this scenario is to create the application using the Microsoft Office 365 API Tools for Visual Studio, which as of this writing you can find here:https://visualstudiogallery.msdn.microsoft.com/a15b85e6-69a7-4fdf-adda-a38066bb5155.

Once you have those installed, the next step is to create a new project – whatever kind of project type is appropriate for what you are trying to accomplish. Once you’ve done that you want right-click on your project name and select “Add Connected Service”, like this:

When you add your Connected Service the first thing you’ll do is click on the Register your app link. It will first ask you to log in using your Azure organizational account.This account needs to be an admin account – one that has rights to create a new application in an Azure Active Directory:

After entering your credentials Visual Studio will take care of registering the app with Azure. It then presents you with a list of permission scopes, and for each one you can click on the scope to select which permissions in that scope you want your app to have.Remember that when a user installs your application it will ask if it’s okay for the app to have access to these things you’re defining in the permissions list.

In this case I’m just going to select the Users and Groups scope and select the permissions to Read directory data and Access your organization’s directory:

That should be enough permissions to use the o365 Discovery Service API; I’ll explain why we want to do that in a bit. For now just click the Apply button, then click the OK button to save your changes.When you do that Visual Studio will begin modifying the application with the permissions you requested.In the Output window in fact you should see it add the Discovery Service NuGet package to your project (“ServiceApi” is the name of my project in Visual Studio):

Adding ‘Microsoft.Office365.Discovery’ to ServiceApi.

…

When it’s done with that it should open up a page in the Visual Studio browser window that includes the interesting information about what you need to do next, and of course, no details on how to actually do it. That’s why you’re reading this blog post now, right??🙂

The next thing we’re going to do is to go into the Azure Management Portal and add the other rights we want for our application, which is to use the Service Management API. So open up your browser and log into the Azure Management Portal at https://manage.windowsazure.com.Scroll down and click on the Active Directory icon in the left navigation, then click on your Azure Active Directory domain in the right pane.When you go into the details for your directory, click on the Applications tab, then find the application you just created.Click on it and then click on the Configure link.

One of the things you’ll see at the top of the page is the CLIENT ID field. If you look in Visual Studio at your project, you should have had something like an app.config file added to it when you set up the application (app.config is added to winforms project; the config file that is added will vary based on your project type).Open up the app.config file and you will see an appSettings section and an entry for ida:ClientId.The value for it should be the same as the CLIENT ID value you see for your application in the Azure portal.

Scroll down to the bottom of the page for the application to the “permissions to other applications” section. You should see one entry in there already, and that’s for the permission we requested for working with the directory data.Now click on the drop down in that section that says “Select application”, and select Windows Azure Service Management API.On the drop down next to it that says “Delegated Permissions: 0”, click on it and check the box next to the item that says “Access Azure Service Management”.

Once you’ve done that click the SAVE button at the bottom of the page. You’re done now with all of the configuration you need to do in Azure, so let’s shift gears back into our project.

The next thing we need to do is plug the application key values we need into our application. That includes the client ID, redirect URI, the Discovery Service resource ID and the Azure Service Management resource ID.You can get the client ID and redirect URI values from the app.config file that was added to your project.If for some reason you can’t find it in your project, you can also find it in the Configure tab for the application in the Azure management portal.The client ID value is in the CLIENT ID field, and the redirect URI value is in the REDIRECT URIS list.Finally, we are also going to use what’s called the “Common Authority” for getting an authentication context – an ADAL class (Active Directory Authentication Library that is used under the covers to get an access token to the o365 and Azure resources).

The resource IDs are fixed – meaning they are always the same, no matter what application you are using them from. So with that in mind, here’s what the values look like once I’ve added them to the code behind for my winforms application:

Okay, we’re getting awfully close to actually writing some code…the last thing we need to do before we start doing that though is to add the ADAL NuGet package to our application. So in Visual Studio open up the NuGet package manager, search find and add the ADAL package and add it.When you’re done you should see it AND the Microsoft Office 365 Discovery Library for .NET, which was added when we configured our application previously:

Now we’ll add the last couple of helper variables we need – one which is the base Url that can be used to talk to any specific Azure tenant, and the other is a property to track our AuthenticationContext:

The first thing we’re doing is creating a new AuthenticationContext for a particular authority. We use an authority for working with an AuthenticationContext so that we can manage the cache of AuthenticationResults for an authority.ADAL provides a cache out of the box, so once we get an AuthenticationResult from an AuthenticationContext, we can just pull it from that cache without having to go through the whole oAuth flow all over again.The AuthenticationResult by the way is where we get the things we need to access a resource – an access token and a refresh token.

Once we’ve created our AuthenticationContext again then we can try and acquire the AuthenticationResult out of the cache, which is done with the call to AcquireTokenSilentAsync. If there’s an AuthenticationResult in cache then it will be returned to you.If not then it throws an exception, which we catch directly below that call.

Once we get through that part of the code we look to see if we were able to get an AuthenticationResult. If not, we’ll go ahead and use the oAuth flow to obtain one.That means a dialog will pop up and the user will have to enter their credentials and approve our application to access the content we said we needed when we configured the permissions for our application.

Now that we’ve got the code out of the way to get an AuthenticationResult, we can write the code to actually go work with our data. This is where you can see this kind of interesting intersection between the o365 APIs and the Azure Service Management API that I was describing at the start of this post.As I alluded to earlier, I really put this code together and wrote this post for two reasons:1) there are SO MANY o365 API examples that either don’t include clear instructions on how to obtain an access token, or the code they have either does not even compile or does not work.This is proof I guess that APIs change, right?Now you have an example that works (at least for today). 2) the Service Management API requires the tenant ID to work with it.Well, as it turns out, when you get your AuthenticationResult from a call to the DisoveryService through the o365 APIs, you will get the tenant ID back.

So you can use this pattern – call the DiscoveryService, use the CommonAuthority for the authContextUrl and authenticate, then you will have the tenant ID. Then take the tenant ID, use the login Url for the specific tenant as the authContextUrl and get an AuthenticationResult to use with the Service Management APIs (we’ll end up getting the access token silently).Once you have that you can take the access token from it to do whatever you’re going to do with Service Management APIs.

Now, let me explain one other behavior that you may not be aware of, and then explain how that impacts the code I described above. If you read my description of the pattern above, you may notice this:in the first call to create an AuthenticationContext I use the CommonAuthority for the authContextUrl, which is https://login.windows.net/Common.In the next call I use an authContextUrl of https://login.windows.net/myTenantIdGuid.So if I’m using two different Urls to create my AuthenticationContext, then how could there be something in the cache for me to use when as I say above, on the second call I’m going to get the AuthenticationResult out of the cache?Well, it turns out that when you create the AuthenticationContext with the CommonAuthority, after the user actually authenticates ADAL changes the Authority property of the AuthenticationContext from CommonAuthority to https://login.windows.net/myTenantIdGuid.This is actually pretty cool.The net result of all this is – I actually don’t need to authenticateinto the DiscoveryService at all to get the tenant ID for the tenant being used.So what’s the net of everything I’ve given you so far?Well as I explained at the beginning, you now have some nice code that actually does work with the o365 APIs.However now that you understand how it works you can see you really don’t need to call into the DiscoveryService to get the tenant ID, so you learned a little something about how the AuthenticationContext works in the process.

Okay, so now that we know that, let’s take a look at the code to get our subscription data from Azure. You’ll see that it makes just a single call to get an AuthenticationResult for working with the Service Management APIs.Note, this code uses the new HttpClient library, which is installed as a NuGet package; here’s the package I added:

So just to recap what we’ve been talking about…the first thing I do is get an AuthenticationResult using the CommonAuthority. If the person that signs in does not have rights to use the Service Management APIs for the tenant they sign into, then they won’t get the chance to grant the app permissions to do what it wants.Otherwise they person will ostensibly grant rights to the app, and you’ll get your AuthenticationResult.From that we take our access token and we add two headers to our request to the subscriptions REST endpoint:the authorization header with our access token, and a special Microsoft version header, which is configured per the Microsoft SDK.Then we just make our request, hopefully get some data back, and if we do we stick it in the text box in our application.Voila – mission accomplished!Here’s an example of the XML for a subscription (and yes, the GUIDs shown here are not the actual ones from my tenant):

My biggest regret here is that even though in some ways this is “simplified”, it still takes me nine pages of a Word document to explain it. However I do think you have some good code that you can go out and run with today to start working with both the o365 APIs as well as the Azure Service Management APIs, so I think it’s worth it.Just bookmark this post for a rainy day, and then when you’re ready to start developing on either SDK you know where to find the info to get you started. I’ve also attached to this post the complete source code of the application I wrote here. You’ll just need to update the client ID and return URI in order to use it. I’ve also included the original Word document from which this somewhat ugly post was pasted.

Many fine folks were generous enough to point out to me this week that you can now (actually since June’ish I’m told) use an access token you get from ADAL in conjunction with the o365 APIs to use ALSO with the SharePoint REST API as well CSOM. Shocking!! This may be what they call “asleep at the wheel” (or taking the red pill), but now that I’m wide awake again and aware of my surroundings it seemed like a good time to augment and update some of the content that’s out there.

Jeremy Thake did a nice write up on this concept originally at http://www.jeremythake.com/2014/06/using-the-sharepoint-csom-and-rest-api-with-office-365-api-via-azure-ad/. While much of it still applies, I decided to revisit it myself so a) I would appear to know what the hell I’m talking about and b) I could update the really awful goo that he was forced to use to get an access token. So in this little sample I’m going to use a standard flow to get an access token using ADAL (Active Directory Authentication Library). I’m also going to be a little verbose and do a few things that you can do “less manually” with some tools for Visual Studio, just to make sure you are clear on what those tools are actually doing for you (especially if you’re <gasp> not using Visual Studio). Here we go.

Add a New Application

You can use the Office 365 Tools add in for Visual Studio, or just create the application yourself manually in the Azure portal. That’s what I’m doing in this case. I begin by opening the portal and selecting my Azure Active Directory instance that’s used with my Office 365 tenant. Then I click on the Applications tab, and then click on the ADD button at the bottom of the page to create a new application. Here’s how we do it:

Click the check button and you’ve completed the first step – your application has been created. Now go ahead and click on the Configure link so you can add the permissions you need to access your o365 content.

Click the Add Application button at the bottom of the page and a Permission to Other Applications dialog pops up. Click on the Office 365 SharePoint Online application and then click on the + sign next to it. When you’ve completed that click the check button to save your changes:

Now you’ll have an item in the list of applications that your application wants permissions to, and you can click on the Permissions drop down for it to configure the permissions you want your application to have in the Office 365 tenant where it’s used. Here’s what that looks like:

After you’ve made all of these changes, click the SAVE button at the bottom of the page but DON’T close the application configuration page just yet, there’s some info we need to grab out of there for connecting to it. Scroll up to the top of the page and copy the a) CLIENT ID and b) REDIRECT URIS values. You are going to use this when you ask a user to consent to having your application access his or her SharePoint content:

Okay, now we’re going to create a little winforms app that uses ADAL to get an access token, and then CSOM and REST to get data out of SharePoint. So to begin the next step I’m going to create a new winforms application, and dive right into the code-behind in my form. Once in there I’m going to plug in my Client ID and Redirect URI that I got from my application configuration in Azure.

Now I’m going to use NuGet to add in support for ADAL and o365; when I’m done it looks like this:

I’m also going to add references to Microsoft.SharePoint.Client and Microsoft.SharePoint.Client.Runtime so I can use CSOM from my application. Next I’m going to add that same helper method I described in my previous blog post to get the access token, so let’s use this code here:

Okay, now we have the most important part of the guts done, let’s go back and look at the basics of what Jeremy was showing in his post:

Retrieving Data via REST

This one is virtually identical to Jeremy’s code, it just a) uses my updated code for obtaining an access token and b) shows how to create the Resource ID that you want to use for getting content from your SharePoint sites. I tested this code against a couple of different o365 tenants and I can happily <small tear in eye> say that it worked in both.

private async void RestListBtn_Click(object sender, EventArgs e)

{

try

{

AuthenticationResult ar = await AcquireTokenAsync(CommonAuthority,

GetSharePointHost(SiteUrlTxt.Text));

if (ar != null)

{

string requestUrl = SiteUrlTxt.Text + “/_api/Web/Lists”;

HttpClient hc = new HttpClient();

//add the header with the access token

hc.DefaultRequestHeaders.Authorization = new

System.Net.Http.Headers.AuthenticationHeaderValue(

“Bearer”, ar.AccessToken);

HttpResponseMessage hrm = await hc.GetAsync(new Uri(requestUrl));

if (hrm.IsSuccessStatusCode)

ResultsTxt.Text = await hrm.Content.ReadAsStringAsync();

else

MessageBox.Show(“Unable to get subscription information.”);

}

}

catch (Exception ex)

{

MessageBox.Show(“Error: ” + ex.Message);

}

}

private string GetSharePointHost(string url)

{

Uri theHost = new Uri(url);

return theHost.Scheme + “://” + theHost.Host + “/”;

}

The main thing to call out here really is the GetSharePointHost method that creates the Url that is used as the Resource ID when getting an access token. The net of this advice is that no matter where your SharePoint site lives, you really want to use the root site as the Resource ID. So if your o365 site is https://steve.sharepoint.com/sites/blazers, the Resource ID to get an access token should be https://steve.sharepoint.com. Doesn’t matter where the site is in that hierarchy – from the root all the way down as far as you want to go – you want to make sure you use the root site as the Resource ID. About a billion thanks to Dan K. for clearing that up for me; your Christmas card is in the mail! The other thing worth noting is that you MUST include a trailing slash on the Url that you use as the Resource ID. It cause ADAL no end of pain and suffering when you leave that out. Both of these little gems are captured in those two lines of code above. Yay!

Retrieving Data via CSOM

This one is also quite similar to Jeremy’s…again the main differences are that it uses my standard method to get the access token, and for some reason I get a somewhat different class naming structure for my WebRequestEventArgs. Here’s the relevant code:

So there you have it. I must say, I find this ridiculously cool. Many thanks to Jeremy for his original post and for the loads of people who basked in the glory of pointing out to me that I missed it. 🙂 Just kidding of course, I love having a legion of friends that can read on my behalf and get me pointed in the right direction as needed. Hopefully this post will help one of you in the same way. I’ve attached the complete source code to this post, so just change the ClientID and ReturnUri variables after you create your application and you should be good to go. As usual, I’ve also included the Word document from which this sorry mess of Notepad-like content was created.

I’ve been spending a lot of time recently talking to different partners and customers about building applications using the Apps for Office and Apps for SharePoint platforms. One of the questions I seem to get most frequently though has been about one of the new additions to our development platform, which is the o365 APIs and specifically OneDrive.As I’ve been trying to field the various requests about how to use the o365 APIs for OneDrive, I decided that probably the best solution would be just to write a little browser application myself that I demonstrates using the basic features of the API and then leaving it up here for all you to download and deconstruct to your heart’s content.So…guess what I did over my Christmas break??Yeah, that’s why I called it Christmas “break” and not “vacation”.😉

The other thing that I felt has been somewhat lacking from a documentation perspective is how to use ADAL generally and the o365 APIs specifically from a web app, so I decided to use that as the starting point for this application. The result is an ASP.NET MVC app that shows off the process for getting an access token that you can use with the o365 APIs and “managing” it.Once we have our access token there are a variety of different ways in which we can use it, and different options for working with OneDrive.I kind of run the gamut in the sample application – from storing and retrieving an access token in Azure table storage to passing it along as a parameter in a GET and/or POST request.In working with the o365 APIs I show a few different options:getting the whole set of content at one time on the server side and then rendering it on the client, to building it one piece at a time using jQuery with an MVC controller and partial view, to using a Web API controller.Also it does more than just list the contents of my OneDrive site – you can also create and delete folders, and upload and delete files.I also added some code in the controllers to download files for completeness, but didn’t really do anything with it in my browser (because you can just click on the link I render for the file and download it that way).

Here’s what it looks like when I actually go to my OneDrive site:

Here’s what my browser looks like when all of the content has been retrieved on the server side and rendered all at once on the client:

Here’s what it looks like in the view where I retrieve everything client side (and also where I do the other file management tasks):

Create Your Applications

To begin, before you ever start writing any code, the first thing you’re going to need to do is to create two applications in your Windows Azure Active Directory (AAD) tenant – the first one is going to be used to authenticate users into the MVC application; the second one is going to be used to access OneDrive contents through the o365 APIs.There are multiple options for both of these applications.

Create Application for MVC Authentication

If you’re using Visual Studio 2013 then when you create a new MVC application you have the option to configure it use AAD for authentication. The process in pictures looks like this:

Alternatively you can just go into the Azure management portal and create the application manually. This is what I did for two reasons:1) you need to go there anyways to get the client ID and other information necessary to configure your application and 2) starting the project using the authentication wizard in Visual Studio makes it virtually impossible (or very difficult) to configure it such that you can use the same AAD authentication process to secure both your MVC and Web API controllers.So if you decide to create the application manually you just need to grant the application permission to “Enable sign-on and read users’ profiles”.Here’s what my application looks like in AAD:

Create Application to Access OneDrive Content

To create the second application for accessing OneDrive content, you again have a couple of choices: 1) you can install Update 4 or later for Visual Studio 2013 and select the Add Connected Service menu option when you right click on your project in Visual Studio or 2) you can create it manually.I think you get how to add it manually at this point; if you want to configure it using Visual Studio then look at the Add a New Application section of this Share-n-Dipity post:http://blogs.technet.com/b/speschka/archive/2014/12/17/using-adal-access-tokens-with-o365-rest-apis-and-csom.aspx.My completed application looks like this; you can see that I just granted the application the right to read, edit and delete users’ files:

Now that we have all the application security aspects defined we can start writing code. That being said…I’m going to preface this by saying I’m not going to do a super deep dive on all of the code in the project in this blog post.The reason for that is because a) it’s somewhat complicated, b) there’s a lot of it, and c) I’m attaching the entire project’s source code to this post.I’ll focus on conceptually what was done and why (with appropriate detail, don’t worry), but I won’t be covering every line of code like I do in some posts.You’re going to have to be a little adventurous and download the project and open it up yourself.Trust me when I say that it will be way more valuable than reading a 50 page whitepaper / blog post on using the o365 APIs.As it stands now Word is telling me that I’m already on page 10.Blech.

Working with Access Tokens in MVC

There are a few things worth noting as it relates to AAD, ADAL, access tokens and o365 APIs. The first and most important thing to remember is the every resource needs its own access token.I can’t emphasize this enough…it’s easy to get lost quickly managing your access tokens.You’ll probably start out getting an access token for the Discovery Service to get the Url for a user’s OneDrive site, and then not understand why you can’t read someone’s OneDrive files using that same access token.Well it’s because the Discovery Service and the OneDrive service are two different resources so you need to get separate access tokens for each.Now you may ask, well, I though ADAL supported multi-resource refresh tokens?Yes this is true, but that just means that you don’t have to prompt a user each time you want to access a different resource – however, you still have to go get an access token for that resource and manage it in some way.

The process becomes more complicated in a web application, which is why I wanted to take the opportunity to cover that here as well. I often do my proof of concept apps using winforms or a console app and it is genuinely easier to work through the goo that way.But a little harsh reality is always good to make sure we understand what’s required and how we can make things easier.The biggest difference is that when you are doing authentication using ADAL from a web app you need to use a client secret.This is unlike almost every other example you’ll see posted out there, which is probably why there have been so many questions about it.In a nutshell, you need to get an authorization code to access the resource, and then you can use a client ID and password (i.e. secret) to get an AuthenticationResult, which contains an access token and refresh token.Once you have the tokens then you really want to persist them somewhere, because otherwise you’ll be pestering your users non-stop to authenticate back to AAD to get a token.So given that background, here is the basic pattern that I used to get an access token and refresh token for accessing these resources:

Query the database to see if I have an access token for the current user and resource. In my solution I used Azure table storage to persist the tokens.

If there is an access token:

Is it valid still?

Yes – grab the access token

No – ask for a new access token using the refresh token

Did it return a new access token?

Yes – grab the access token and update the database with the new access token and token expiration time

No – the refresh token has expired too so I need to start over and request a code again that I can use to get another AuthenticationResult.

If there is NOT an access token:

Ask for a code that I can use to get an AuthenticationResult

If I have to get a code then I:

Use it to get an AuthenticationResult using the AcquireTokenByAuthorizationCode method of the AuthenticationContext class; this is where I’ll pass in the client ID and password.

Store the access and refresh token in the database.

I built some helper classes for working with Azure so it simplifies my code somewhat, but here’s what it looks like:

//create an instance of the Azure storage helper

StorageHelper sh = new StorageHelper();

//look first for the Discovery resource token

OneDriveRequest odr = GetOneDriveToken(ref upn, DiscoveryResourceId);

//see if we have a record, but the token is invalid; if that’s the case then we

So in short, I’m looking for a valid access token – if I have one or I can get one from my refresh token then I’m going to continue onto the next step, which I do by redirecting to another action in my controller. If I don’t have a valid token then I’m going to redirect the user’s browser to the AAD oauth authorization page where they can consent to using my application.When they do that, AAD will look at the redirect_uri in the query string and send the browser back there when it’s finished; the query string will contain the code I can use to get the access token.

The other thing that may not be obvious is that I also include some information in the “state” query string. That’s because I use the same controller action to process code requests for all resources.Again, remember that every resource requires its own access token, so since I’m using both the Discovery and OneDrive services, that means I’ll be acquiring two access tokens.That’s actually how I store it in the database – the user’s UPN and the resource create the key for each row in the table.So I can ask the database for the Discovery access token for speschka@contoso.com and know that it will be separate from the access token for the OneDrive service for speschka@contoso.com.

All roads in that first controller method lead to my ProcessCode action. That code can get complicated rather quickly if you’re just trying to grok through it in blog post so again, I’ll just walk through it more from a conceptual level.To begin with, my method signature for the method looks like this:

Since I’m redirecting to this code whether I’ve asked for an access code or not, the first thing I do is check for whether the “code” parameter is null or empty. If it’s not, then I know that we’re responding to a request for a code that I can use to get an AuthenticationResult.What I’ll do then is this:

Get the client ID and password out of my web.config file. I’m going to use the client ID and password for the o365 AAD application, not the MVC AAD application.

I get a new AuthenticationResult by calling the AcquireTokenByAuthorizationCode on the AuthenticationContext.

I store the access and refresh token from my AuthenticationResult in the database.

If the “code” parameter is null or empty then I just query the database for the access token. Since I’ve passed the resource I’m trying to access in the “state” parameter when I call this action, I can extract that out now so I know what data to get from the database.

Now that I’ve gone through that little bit of gymnastics, I’ve got my pattern in place that I can use for the rest of the app. At a high level it’s basically going to go like this:

Does the “state” parameter indicate that I want to access the Discovery service?

Yes – I’m going to create a new DiscoveryClient and use the access token I got above. After I have that I’m going to ask for information about the “MyFiles” capability. That will return two important pieces of information for me: a resource ID and an endpoint URI. The resource ID is what I’ll need to use to get an access token to access the “MyFiles” capability; the endpoint URI is where I’ll send my requests for data.

Once I have the ID and URI I query the database to see if I have an access token for it. It then follows the same exact pattern I described above – does it exist and is it still valid? If it exists but isn’t valid then try using the refresh token to get a new access token. If that works, or if it was already valid, then I have an access token I can use with the “MyFiles” capability (i.e. OneDrive) and I’m going to redirect to this action all over again, but this time I’m going to configure the “state” parameter to say I want to access the OneDrive service.If I don’t have a valid access token for OneDrive, then I need to get a code for that resource that I can turn into an access token. To do that, I’m going to use virtually the same exact code I showed above, I’m just going to use the resource ID for the “MyFiles” capability instead of the Discovery Service resource ID.I’ll use the same redirect_uri so that it comes back to this same exact controller method and we start this little dance all over again.

No – I’m not trying to use the Discovery Service, I’m trying to use the OneDrive service (a.k.a. the “MyFiles” capability). Based on the code I just described above, at this point I know I have a valid access token for the OneDrive service. What I’m doing initially in my scenario is just getting a list of all the content in my OneDrive site, so I’ll create a new SharePointClient using my access token. I ask for the list of files, which by default just returns the root folder (“Shared with Everyone”). With that, I’ll call a method I created that gets all of the details about the folder, along with all of the folders and files it contains. In this case for each folder I find I’ll call this same method recursively so I end up with all of the folders and all of the files in my OneDrive site. When I’m done getting all of my content I plug it into the TempData dictionary that MVC provides and I redirect one more time back to my Home controller. From there it returns a view that uses the OneDrive content in the TempData dictionary and the view emits the result on the page.

That, in one ginormous nutshell, is how the first page gets loaded.

Working with Content

Now that the whole issue of dealing with access tokens is explained, most of the rest of the details are downhill. For a variety of reasons, I used both the Microsoft Office 365 Discovery and My Files Library clients, as well as the o365 REST APIs to work with the content in the OneDrive site itself.The Discovery client is really important to get started because it provides the two things you need to query OneDrive:the resource identifier for the OneDrive site, and the actual endpoint you use to work with content in that OneDrive site.In the Discovery client, you get these attributes from a CapabilityDiscoveryResult, which has ServiceResourceId and ServiceEndpointUri properties for this purpose.

With those valuable pieces of information at hand, I start out by getting the root folder of the OneDrive site by using an instance of the SharePoint client that comes with the Microsoft Office o365 libraries. It was really designed to get the access token asynchronously as part of the client creation process, but my scenario doesn’t fit into that little box because I already have an access token.Instead then, I create my client like this:

var odClient = new SharePointClient(new Uri(EndpointUri), () =>

Task.Run(() =>

{

return accessToken;

}));

Using that I can get the root folder like this:

IPagedCollection<IItem> items = await odClient.Files.ExecuteAsync();

That collection is going to return the “Shared with Everyone” folder that is common at the root of every OneDrive site. In theory you could also have additional folders and what not at the root, but it is so rare that I chose not to focus on that for my scenario.Really all I need is the first item the collection because it has the ID for the item, and that ID is what I use to key all of my subsequent requests for data.Simply stated, I took a look at the data that’s returned by a REST call for content and I mapped that into my own class, which I call a OneDriveItem.In addition to all of the details about an item (which can be either a folder or a file), each OneDriveItem also has a List<OneDriveItem> of Folders and Files.The content schema is the same for both, so this is why OneDriveItem is used for everything in my code sample.

You can probably imagine the rest of how that scenario plays out (or just download the code to look at it) – I have my item ID, so I a) get all of the details for that item, b) get all of the files contained in that item and c) get all of the folders contained in that item. For each folder I find, I can optionally call the same method recursively to get all of the details, files and folders in it.That’s really the main difference between the view that shows the entire contents of the OneDrive site at load time, and the view that loads it one folder at a time; when I show everything I call the method recursively and when I show it one folder at a time I do not.

Working with Content from Client Script

At this point I’ve explained how I get the access token and how I retrieve contents on the server side. Working with data from client script involved using jQuery and calling either a standard MVC controller (when I want a chunk of HTML back) or a Web API controller (when I want to perform some action, like adding or deleting a folder).I chose to write the access token to a hidden field on the page and send it back to the controller either in a query string or HTML form when I requested data.Since everything is working over SSL I’m okay with that.If you are not, another alternative could be to store some key to your database where you keep the access token, send that up, and look it up.There are a few minor nuances in terms of what it means from a security standpoint, but I’ll leave it up to you and your requirements to dictate the appropriate implementation.

That aside, the implementation for this sample was fairly straightforward. When I needed HTML I called an MVC controller action because I can return a view from that, which returns HTML.In fact I ended up using the same exact partial view to render both the view where I showed the entire contents all at once, and the view where I build the contents one folder at a time.

In addition to rendering contents, on the client view you can also add folders, delete folders, upload files, and delete files. I also wrote methods in the controllers to download files but didn’t actually implement then on the client script view.The reason for that is because I rendered each file as a hyperlink to that file, so you can just click on the file to open it up.I added the code in the controller though in case you are working with some other modality (like a console app, or winforms app, or whatever).Fortunately both the REST endpoint and Office 365 libraries make those other functions pretty straightforward to do.Here’s how to create a new folder for example; the parameter NewFolderRequest is just a custom class I created that contains all of the HTML form values that were submitted with the request.The return value from creating a folder is the same set of JSON that you get when you query the folder details, so I’m able to deserialize that back into a OneDriveItem again so it plugs into my various rendering methods really well.

The gist is pretty simple really – I create a Url in the format that the o365 REST endpoint requires for creating a new folder. I create a new HttpClient and add the authorization header using the access token I have.Then I make the PUT request to create the new folder.If it works, I take the JSON I get back and deserialize it into a OneDriveItem that I can then send back so it can be rendered by the client script into the view of the contents I’ve built in the page.The other commands for adding and deleting items are pretty similar, so you can take a look at the code included with this post for specific details.

Summary

You have what should be a nice pattern now for working with o365 access tokens in an MVC app. In addition the solution includes working examples of the most common tasks when using the o365 APIs for OneDrive.It also demonstrates using both the o365 libraries as well as the REST endpoints for working with data in OneDrive sites.Using this sample as a reference should provide you with enough examples to get any of your o365 API applications up and running for OneDrive sites.

One of the things you frequently want to do in your custom applications is send out emails. Historically this could be solved by using an SMTP server you have on site, using Exchange Web Services with an Exchange server either on prem or in the cloud, using a public SMTP service, etc. As I was thinking about this recently I of course thought, wow, I should just do this using the new o365 APIs. This proved a little more challenging that I was originally expecting so I figured I’d blog about it here and try and lower another barrier of use.

I haven’t actually written anything on using the o365 APIs and email yet (although I’m thinking about another little sample app for it like I did with the OneDrive browser…let me know in the comments if this is of any interest to you). The o365 APIs for working with email (so far) has actually been refreshingly easy…not as full featured as Exchange Web Services, but honestly a lot easier for the common tasks. The real challenge in getting this going actually was more on the authentication side – getting the application authenticated and able to use a mailbox. We do have a sample out on github that I was originally looking at that talks about an application getting an access token and using that to impersonate a user (http://msdn.microsoft.com/en-us/library/azure/dn499820.aspx#BKMK_Server). Unfortunately that scenario assumes you have a user that’s already authenticated to your application so you can use that access token; that’s not going to be the case with a typical unattended console type application. Then there was an app that uses an application token instead of a user token (https://github.com/AzureADSamples/Daemon-DotNet), but that doesn’t help either because an app doesn’t have a mailbox.

Finally I settled upon the magic overload for acquiring an AuthenticationResult that you should use in this scenario:

Now here’s part of the problem that I ran into and what this tougher to figure out. When I started my application in Visual Studio, I had it create the application in Azure for me using the Add Connected Service wizard, which I explained in this blog post: http://blogs.technet.com/b/speschka/archive/2014/12/08/oauth-o365-apis-and-azure-service-management-apis-using-them-all-together.aspx. Well when it created the app it set up the permissions I asked for related to Outlook – basically reading, writing and sending emails. That was the only set of permissions it granted though. So when I first called AcquireToken as shown above, it threw an exception like this:

AADSTS65001: No permission to access user information is configured for ‘f276aaaa-aaaa-aaaa-aaaa-bbbbbbbbbbbb’ application, or it is expired or revoked.

(In this case ‘f276aaaa-aaaa-aaaa-aaaa-bbbbbbbbbbbb’ is the client ID of my application in Azure)

So I went to look at the application in Azure and saw that it had the correct o365 permissions configured; my hunch though was that it needed Azure AD permissions too. Here’s where it got ugly – I tried adding permissions to Azure AD to login and read users profiles, but it would not let me do it. It kept failing in fact when I tried to add any Azure AD permission to my application. Good grief.

Long story short, that sent me on a multi-hour goose chase trying out different authentication options, but none of them work. I really thought I had the right one though, so what I ended up doing was going back into the Azure management portal and manually creating a new application. I created it for as a Native Client application, and it was automatically given permissions to Azure AD to sign-on and read users’s profiles. I then added my o365 application permissions to it, tried my code again and – voila’ – the call to get an AuthenticationResult started working. Yay! The hardest part of this story is now done.

With that code out of the way, the process goes something like this:

Get an access token using a service account username and password. Where and how you store and retrieve the username and password is up to you; to simplify my demo I just hard-coded it in my console application.

Get an instance of the OutlookServicesClient. There’s really two things you need to know about doing this – the resource ID and endpoint URI. Again, normally one would probably use the Discovery service to get these values and then use that to get your OutlookServicesClient. In my case I actually did that originally and then found the values to be so generic that I decided (for now) to just use them directly instead of querying the Directory service every time. I’m sure someone will take great joy in pointing out how “unfuture proof” such an approach is but hey – it’s a demo, you know what you’re working with here, so adjust to fit your needs. To complete this thought, for Outlook the resource ID is https://outlook.office365.com/ and the endpoint URI is https://outlook.office365.com/api/v1.0.

Create your email message.

Save it as a draft (so you can get the message ID)

Send it.

There are two other things worth noting for #4 and #5. In a winforms or ASP.NET application I would do those operations asynchronously with the await keyword. However, that doesn’t really work in a console app since there is just the single main app thread (the app continues without waiting basically). To work around that in a console app I used a simple pattern to wait on a task, so for example #4 looks like this:

Task t = oc.Me.Messages.AddMessageAsync(draftMessage);

t.Wait();

The rest of it is pretty straightforward. I’ve included the complete project attached to this post. You just need to:

Use the username and password

Go create your own application in Azure

Use the client ID and return URI from it to replace the values in the app

Coding Implementation

The code itself is pretty straightforward. To begin with I created a new project and added the ADAL v2 Nuget package. I then created a new class and had it inherit from Microsoft.IdentityModel.Clients.ActiveDirectory.TokenCache. In my implementation I’m storing the tokens in Azure table storage. I’m using it in an ASP.NET MVC application, and I need to track the tokens on a per user basis. In my case the main customizations then that I need to account for are a) tracking tokens for individual users and b) managing access to Azure storage. I provide hooks to both of those things through my constructor.

For tracking individual users I’ve used the user UPN. I’ve configured my MVC app so that it’s secured with Azure Active Directory. Since a user has to authenticate before they even get into my web app, I can be assured that they have a UPN. For managing access to Azure storage I left in a couple of options. I have the storage name and key included in one of the helper classes in my custom TokenCache project. That way if you just want to take the code and compile it yourself, you can replace those values with the storage name and key that you are using. As another option, I created a second constructor that takes the storage name and key in addition to UPN so you can set your credentials on the fly. Here’s an example of what the two constructors look like (my class name is TokenStore):

Inside the class I’ve implemented BeforeAccessNotification, AfterAccessNotification, and DeleteItem. The notifications are hooked up from my constructors with code like this:

this.BeforeAccess = BeforeAccessNotification;

this.AfterAccess = AfterAccessNotification;

DeleteItem doesn’t have an event handler to hook to so in that case I just have to override it, invoke the base class method first, then do my stuff. Again, the actual code for all of this is pretty simple. I created a helper class to manage all of the Azure stuff, so in pseudo code terms my implementations look like this:

BeforeAccess: use the UPN that was passed into the constructor and the resource ID that is being requested. Look in Azure storage to see if I have a token in there for that combination; if so, then get the byte array out of table storage and call the Deserialize method that comes from the base class.

AfterAccess: look to see if the cache was updated (for example, if the access token was refreshed). If so, save the byte array back to table storage and set the HasStateChanged property of my token cache to false so that it doesn’t keep trying to overwrite the same token.

DeleteItem: call the base class so the deleted token is removed from the in-memory cache, then delete the same item out of Azure table storage.

That’s basically it. All of the source code is included with this post so you can review it and use it as you see fit.

App Implementation

As I mentioned above the trickier part in using ADAL with my custom token cache was to ensure that I was using ADAL “the right way” in my app. Arguably this really isn’t even specific to my custom token cache, it’s the way that I should be using ADAL anywhere, regardless of what caching mechanism is being used for it. As I was describing before I’m using this with an ASP.NET app; that means that in order to get an access token I need to use the acquire token method in ADAL that uses an authorization code and client credential (so I can pass in the app secret). What’s important in this process as well though (according to the Vittorio blog I referenced at the beginning) is that you use the user’s actual login Authority when you create the AuthenticationContext, versus just using common (i.e. https://login.windows.net/common/). The last part of the flow is that when a user is ready to get an AuthenticationResult in subsequent requests, you should no longer ask for it using the authorization code – instead you should try one of the AcquireTokenSilent overloads.

So with all of this information in hand, here’s how the process works of getting an AuthenticationResult (which has the access token):

I have a table in Azure storage where I keep track of each user’s authentication parameters – UPN, resource ID, and login Authority. When the user goes into a controller action that needs an access token I look in that table to see if I have a record matching the UPN and resource they are requesting.

The first time a user logs in, there won’t be any record so I redirect out to Azure to get an authorization code.

When Azure redirects to my controller, I see that there is an authorization code so I:

Create a new instance of AuthenticationContext, using my custom token cache: authContext = new AuthenticationContext(“https://login.windows.net/common/&#8221;, new AdalAzureCache.TokenStore(upn));

Request an AuthenticationResult: result = authContext.AcquireTokenByAuthorizationCode(AuthCode, new Uri(Request.Url.GetLeftPart(UriPartial.Path)), credential);

Store the user’s UPN, the ID of the resource they were requesting, and the login Authority in my custom storage for this app. To be clear – this is storage just for this app – it is different from the storage that my custom token cache class uses. The way I get the Authority for the user is after I’ve acquired an AuthenticationResult for them, the AuthenticationContext.Authority property has their specific Authority.

The next time the user needs an AuthenticationResult, I look in storage for my app and I find a record with the user’s UPN, resource and Authority. In that case I:

Create a new instance of the AuthenticationContext; same as before but this time I use the user’s Authority: authContext = new AuthenticationContext(“user’s authority here”, new AdalAzureCache.TokenStore(upn));

Request an AuthenticationResult silently: result = authContext.AcquireTokenSilent(ResourceUri, credential, UserIdentifier.AnyUser); This particular overload is one that I just found through trial and error to work with my token cache.

That’s basically it. When the request to acquire the token silently is called, I see my custom token cache getting invoked. It finds the token cache in Azure table storage for the user and resource and then sends it out for ADAL to use. Everything just plugs in and out like a well-oiled machine and works.

All of the source code for the OneDriveBrowser application that has been updated to use the new App Implementation described above and has been included, as well as the custom ADAL token cache that uses Azure table storage. As with the previous OneDriveBrowser post, you will need to go into the Globals.cs in the AdalAzureCache and CloudHelper projects and plug in your Azure storage name and key (exception: if you don’t want to compile it in AdalAzureCache you can pass the values in through the constructor as explained previously). In the web.config file for the OneDriveBrowser MVC application you will need to go plug in your Azure application values. Hope you find it useful.

I started hitting this little gem of an error recently and was ready to pull out the little remaining hair I had, when a very kind Program Manager named Jim S. told me how to fix it. The scenario is something like this: you install an App for SharePoint in Office 365. You make some changes to the app that require you to remove and add the app again. So you go about doing that process of removing the app from the site, and then selecting and choosing to add it again. You even get the consent dialog asking you if you trust the app and you say okay. After that, things go a little sideways and you get an error that says Sorry, something went wrong…The provided App differs from another App with the same version and product ID. You look around and you don’t see the app installed so it leaves you scratching your head.

If you search around on the interwebs you will find some posts that suggest you can fix this problem by increasing the version number of your application and reinstalling, and that will in fact allow you to add the app again. However, that is really not a solution, that is a work around. Well what Jim was good enough to point out is that the behavior for removing an app changed recently. Now, instead of actually removing the app from the site, it just puts it in the site’s recycle bin. So the app really does live on, and that’s why you can’t add it again. The solution he suggested (and which I found to work) is to empty both the primary and secondary site collection. That removes the app completely from the site, and then you can add your application again. Whew!

One other thing to note – they’ve also recently modified the behavior again in Office 365 so that you should not hit this issue any more. When you are installing an app for the first time now, you can remove it and add it again many times and now it work successfully. However what I found is that for apps that were already installed, I had to go through the process described above to remove it from the recycle bin before I could add the app again. However once the app was completely cleared out of the site’s recycle bins, after I added the app to the site again I no longer needed to clean out the recycle bin during this remove and add update process. So it looks like a little pain, but only until you get those apps cleaned out and added again.

Search Share-n-Dipity

Search for:

SamlMan.Com

Hi, I'm Steve Peschka and I am the SamlMan - come see me at http://samlman.com. I'm always happy to talk to you about your SharePoint, o365, Azure and other cloud-related security and authentication projects.