Mr Sharepoint: Azure Storage REST API: Authenticate with C#

Tuesday, November 1, 2016

Azure Storage REST API: Authenticate with C#

In one of my projects where I've been refactoring a traditional .NET project into a .NET Core project, I used the Azure Storage nugets. As of this posting, the current version of the NuGet supports .NET Core which is awesome - but the dependencies doesn't.

Why is this a problem? Well, because if you want to migrate this code to run on .NET core and you rely on the Windows Azure Storage NuGet Package, it will not be possible to run it in .NET Core currently.

That's why I chose to use the Azure Storage REST API instead for all my things - and I haven't regretted a single moment of it (except for trying to figure the auth part out).

The authentication/authorization bits were not really clear. Well, clear as mud perhaps - the documentation is there, but it's quite confusing and lacks any good samples. So with that, I decided to make a sample.

Enjoy this tip-of-the-day post, and feel free to drop me a comment or e-mail.

Authenticate Azure Storage REST requests in C#

Everything I've built is based on information from this page: Authentication for the Azure Storage Services.

Pre-requisites

In order to use this code, there's a few pre-requisites that I'd like to note down:

You should have an Azure Storage account.
You should have your Storage Account Key.
You should have your Storage Account Secret.
NO need for the Storage Connection string.
The Client object in my code is a normal new HttpClient();

Required Headers

As mentioned in the public documentation, there's a few headers that are required as of this posting:

Date
Authorization

The rest of the headers are optional, but depending on what operations you want to do, and which service you're targeting, they will differ. This is focused on Table Storage currently, but can be applied to others as well.

Creating the Date Header

This is a required header, and the easiest way to demonstrate how to build it is like this:

var RequestDateString = DateTime.UtcNow.ToString("R", CultureInfo.InvariantCulture);

if (Client.DefaultRequestHeaders.Contains("x-ms-date"))  
    Client.DefaultRequestHeaders.Remove("x-ms-date");
Client.DefaultRequestHeaders.Add("x-ms-date", RequestDateString);

If there's already a Date header present, remove it and add it again with the proper value.

Creating the Authorization Header

This is where the tricky part came into play. Seeing it now in retrospective, it's fairly straight forward - but before figuring out in what order, and how to properly encode this header it was a slight struggle.

var StorageAccountName = "YourStorageAccountName";  
var StorageKey = "YourStorageAccountKey";  
var requestUri = new Uri("YourTableName(PartitionKey='ThePartitionKey',RowKey='TheRowKey')");

if (Client.DefaultRequestHeaders.Contains("Authorization"))  
    Client.DefaultRequestHeaders.Remove("Authorization");

var canonicalizedStringToBuild = string.Format("{0}\n{1}", RequestDateString, $"/{StorageAccountName}/{requestUri.AbsolutePath.TrimStart('/')}");  
string signature;  
using (var hmac = new HMACSHA256(Convert.FromBase64String(StorageKey)))  
{
    byte[] dataToHmac = Encoding.UTF8.GetBytes(canonicalizedStringToBuild);
    signature = Convert.ToBase64String(hmac.ComputeHash(dataToHmac));
}

string authorizationHeader = string.Format($"{StorageAccountName}:" + signature);  
Client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("SharedKeyLite", authorizationHeader);

As you can see, it's not entirely straight forward. These are the steps and things to consider:

You first decode the StorageKey from Base64
You then pass this into the constructor for the HMACSHA256 class
You then create a byte[] array to get the UTF8 bytes from the canonicalizedStringToBuild (the date + request information)
You then Base64 encode the hmac.ComputeHash() results
The result of this, is your signature, in combination with adding the Storage Account Name, so it ends up as a string looking like this format: StorageAccountName: Signature.
Then you create a new Authorization Header called Authorization as you can see in the snippet above, with SharedKeyLite and your signature added.

I'm going to be honest. This took some time to figure out - but once it was working, it's blazingly fast and I love it.

Accept Header

Since I want the response to be application/json this is exactly what I need to tell the request:

Client.DefaultRequestHeaders.Accept.Clear();  
Client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));

Request version

I'm specifying which version to use, so if there's new versions coming out I am still targeting the one I know works throughout all of my unit tests and tenants using the code.

This is done using the x-ms-version header.

if (Client.DefaultRequestHeaders.Contains("x-ms-version"))  
    Client.DefaultRequestHeaders.Remove("x-ms-version");

Client.DefaultRequestHeaders.Add("x-ms-version", "2015-12-11");

DataService Version Headers

Since I'm working with entities, I need to specify the DataServiceVersion headers as such:

if (Client.DefaultRequestHeaders.Contains("DataServiceVersion"))  
    Client.DefaultRequestHeaders.Remove("DataServiceVersion");
Client.DefaultRequestHeaders.Add("DataServiceVersion", "3.0;NetFx");

if (Client.DefaultRequestHeaders.Contains("MaxDataServiceVersion"))  
    Client.DefaultRequestHeaders.Remove("MaxDataServiceVersion");
Client.DefaultRequestHeaders.Add("MaxDataServiceVersion", "3.0;NetFx");

If-Match Header

In my specific case I'm doing PUT and DELETE operations sometimes, and when doing that, there's an additional required header you need. The If-Match header.

if (httpMethod == HttpMethod.Delete || httpMethod == HttpMethod.Put)  
{
    if (Client.DefaultRequestHeaders.Contains("If-Match"))
        Client.DefaultRequestHeaders.Remove("If-Match");
    // Currently I'm not using optimistic concurrency :-(
    Client.DefaultRequestHeaders.Add("If-Match", "*");
}

Known issues: Forbidden: Server failed to authenticate the request.

Before I hit the jackpot on how to format my Authorize header, it generated a lot of different errors. The most common one though, being this:

Status Code: Forbidden, Reason: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.

There was no "easy fix" for this, as it simply meant the header was incorrect for Authorization - but it doesn't state what is incorrect or malformed (which I suppose is good, for security). So after a lot of Fiddler4 magic and experimentation with this, I could resolve the issue and the code you see in this post is the one that is currently (2016-11-01) working as expected throughout all of my projects.

Resources

The snippets here are part of a bigger project of mine, hence I can't easily share the entire source. However, should you be inclined in a full working sample, please drop a comment and if there's enough interest perhaps I'll create a new github project for it.

Required headers for querying entities: http://ift.tt/1ERsy6m
Authentication for the Azure Storage Services: http://ift.tt/1DiWTao

by Tobias Zimmergren via Zimmergren's thoughts on tech

Mr Sharepoint