.NET: Using Azure Key Vault With Certificate Outside of Azure

Introduction to Azure Key Vault

In this tutorial I will show you how to create a Key Vault in Azure, keep your secrets there and access those secrets from your application, which is not hosted in Azure but on another host, using certificate. Hiding secrets like database connection string or some password in the Azure Key Vault is the recommended way by Microsoft. 

What will I describe

First I will show you how to create such a vault, add some secrets and provide access to applications that are using a particular certificate. Then I will show you how to create a self-signed certificate, which you can use during the development if you don’t have already some certificate. With that certificate we will see how to access the Azure Key Vault. Last, I will explain you how to use your application and a certificate on a server, which is outside Azure or so called 3rd. party server.  

Prerequisites

First you need to register in Azure. Microsoft provides free 12 months trial to try their services. I suggest to start with that before buying anything. You can register from here. The free trial provides you with 200$ credit so you can check their services.

SSL certificate

If you are hosting your application on some host provider like SmarterAsp or some other Windows Server host, you need to check with their support if you can install certificates. In my case with SmarterAsp I had the option to install a SSL certificate. It is possible, that you already have some SSL certificate installed on your application. You will have the option to use that certificate instead of buying a new one. If you already have an installed SSL certificate, you should contact your hosting support so they can provide access for your application to that certificate. For me, there was already an SSL certificate installed so I was able to use it. I will explain you how to do that later in this tutorial.

Create an Azure Key Vault

In order to create a Key Vault where we are going to keep production secrets like database connection string open the Azure portal and select Create a resource. 

In the Search box enter Key Vault and hit Enter.

This will open the Key Vault page. Press the Create button.

In the next section fill the following:

• Subscription: Select the Free Trial or if you already have another subscription, select it.
• Resource group: Press the Create new option and enter some name for it
• Key vault name: Enter some unique name for the key vault
• Region: From the dropdown menu select a region, which is closer to you
• Pricing Tier: Select the Standard option (unless you require the Premium one)

Leave the rest as it is. Hit the Review + create button.

Azure will validate your selection. If everything is fine you will see a green text stating that the validation passed. Hit the Create button and wait until the deployment is done.

Create an Azure Active Directory Application

Next you need to create an application which will have access to the Key Vault. From the home screen select Azure Active Directory.

From the next section select App Registration.

Then click on New registration. 

Enter a name for your application, select one of the four options (I leave the first one) and enter the URL of your application (only if you have one, if not leave it blank). Press the Register button. You are done with this.

Give the Azure AD Application access to the Azure Key Vault

After we have created a Key Vault and an Azure AD application, now it is time to give access to that application to our key vault. 

From the home screen of the portal select All resources.

From the list of resources that appear, select the Key Vault which we created previously. 

Then click on the Access policies option on the left panel

Click on the Add Access Policy

For the Configure from template (optional) select Key, Secret & Certificate Management, leave the next three options as suggested and then click on the Select principal. On the right side a list with applications will appear. In the search box you can type the name of your application and then select it. Then press the Select button. After that press Add button.

When the application is added it will return to the previous section where you need to press the Save button.

Add secrets to the Azure Key Vault

Here I will explain how to add production secrets like connection strings and passwords to the Key Vault. In the home screen select All resources and then from the resources list select the Key Vault we have created. On the left panel click the Secrets button.

From the section that appears click on the Generate/Import button. Here you can see a list with all your previously added secrets.

Fill the fields in the next section.

• Upload options – leave it to Manual
• Name – Some unique name for the secret
• Value – The value

Leave the rest with its default values. You have the option to set an activation and expiration date if you want your secret to become active at a later period or if you want Azure to delete it automatically at some point. 

When you are done hit the Create button.

Creating a certificate

We will now create a self signed certificate file, which we will use during development if we don’t have an existing one. If you have existing SSL certificate on your hosting and you are able to export it, you can use it instead and skip this section. If you don’t have any certificate to use during development, then we will create one and we will add it to Azure.

Running the Developer Command Prompt

You need to run the Developer Command Prompt for Visual Studio as an administrator. It is mandatory.

Creating a private key and certificate file

You need to enter the following command. Once it is executed, it will prompt you for a password. Enter some unique and strong password.

makecert -sv <KeyName>.pvk -n "cn=<NameOfCertificate>" <NameOfCertificate>.cer -b <StartDate> -e <End Date> -r

Here you need to provide the following information:

• <KeyName> – The name of the private key file
• <NameOfCertificate> – The name of the certificate file
• <StartDate> – The date this certificate becomes active. It should be in the following format – /mm/dd/yyyy
• <EndDate> – The date this certificate is deactivated. It should be in the following format – /mm/dd/yyyy

After you hit the OK button, it will ask you once again to enter the same password. When it is finished you should see the message “Succeeded” on the console.

Creating a .pfx file

Next we need to create a .pfx file. You do this by running the following command:

pvk2pfx -pvk <KeyName>.pvk -spc <NameOfCertificate>.cer -pfx <NameOfCertificate>.pfx -po <Password>

Here you need to provide the following information:

• <KeyName> – The name of the private key file (Same as the one we already created)
• <NameOfCertificate> – The name of the certificate file (Same as the one we already created)
• <Password> – A password for the certificate. Can use the same as before or use a new one. You will have to provide this one when installing the certificate

When you are prompted for a password, use the one that was set in the previous step

This is done. The program will generate the file in the folder, where the console was opened. In my case it is C:\Windows\System32, but since I am using 64bit version of Windows, the files are actually in C:\Windows\SysWOW64. You should find the following three files, which we will use. You can move them in some other folder, which is easier for you to find in. 

Install the certificates

Install the certificate on Azure

First we will install the certificate on Azure for our Key Vault. In order to do so, from the Home screen open Azure Active Directory and then App registrations. From the list with applications, click on the one we created previously. When the new window is opened click on Certificates & secrets. In the next section click on the Upload certificate button.

Fill the field in the next section

A pop-up will appear which will require you to browse and select the certificate file which we created earlier. Select it. Then click on the Add button. This will add the certificate.

Install the certificate on the local machine

For our test purposes we will install the certificate on our local machine. Click on the .pfx file twice in order to install it. Follow the instructions and install the certificate. It’s mostly clicking on the Next buttons. Leave the default settings. When prompted for a password, enter the one which we set with the second command (the one we set when converting from .pvk to .pfx).

Updating the application to use the Azure Key Vault in development

We are now going to develop some code piece to access secrets from our key vault. What I did was to implement that code in a separate library so I can use it later with other Web Applications. But you can add it directly to your application. It’s a matter of preferences. Here is the code I wrote to access the secrets:

The code for retrieving Azure Key Vault secrets

using System;
using System.Security;
using System.Security.Cryptography;
using System.Security.Cryptography.X509Certificates;
using System.Threading.Tasks;
using Microsoft.Azure.KeyVault;
using Microsoft.IdentityModel.Clients.ActiveDirectory;

namespace XQ
{
    namespace XAzureKeyVaultHandler
    {
        public class XAzureKeyVaultCertificate
        {
            /// <summary>
            /// Will keep the thumbprint of the certificate
            /// </summary>
            private readonly string mThumbprint;

            /// <summary>
            /// Id of the application, which has access to the key vault
            /// </summary>
            private readonly string mClientApplicationId;

            /// <summary>
            /// The URL to the azure key vault
            /// </summary>
            private readonly string mVaultUrl;


            /// <summary>
            /// Search all installed certificates by a thumbprint
            /// </summary>
            /// <returns>The certificate which matches the thumbprint 
            ///     or null if no certificate is found</returns>
            private X509Certificate2 GetCertificateByThumbprint()
            {
                X509Store lX509Store = new X509Store( StoreLocation.LocalMachine);
                try
                {
                    lX509Store.Open(OpenFlags.ReadOnly);
                    X509Certificate2Collection lCertificatesCollection = 
                        lX509Store.Certificates.Find(X509FindType.FindByThumbprint, mThumbprint, false);
                    if (lCertificatesCollection == null || lCertificatesCollection.Count == 0)
                    {
                        return null;
                    }
                    return lCertificatesCollection[0];
                }
                catch (Exception lException) when 
                    (lException is CryptographicException || lException is SecurityException 
                        || lException is ArgumentException)
                {
                    return null;
                }
                finally
                {
                    lX509Store.Close();
                }
            }

            /// <summary>
            /// Get the required certificate
            /// </summary>
            /// <returns>The certificate or null if the certificate is not found</returns>
            private ClientAssertionCertificate GetCertification()
            {
                var lClientAssertionCertPfx = GetCertificateByThumbprint();
                if (lClientAssertionCertPfx == null)
                {
                    return null;
                }
                else 
                { 
                    return new ClientAssertionCertificate(mClientApplicationId, lClientAssertionCertPfx);
                }
            }

            /// <summary>
            /// Get the access token from the certificate
            /// </summary>
            /// <param name="aAuthority">The authority of the token</param>
            /// <param name="aResource">The token resource</param>
            /// <param name="aScope">The scope of the token</param>
            /// <returns>The token value of empty string if certificate is not found</returns>
            private async Task<string> GetAccessToken(string aAuthority, string aResource, string aScope)
            {
                ClientAssertionCertificate lAssertionCert = GetCertification();
                if(lAssertionCert == null)
                {
                    return "";
                }

                var lContext = new AuthenticationContext(aAuthority, TokenCache.DefaultShared);
                var lResult = await lContext.AcquireTokenAsync(aResource, lAssertionCert);
                return lResult.AccessToken;
            }

            /// <summary>
            /// Get a secret from the Key Vault by its secret name
            /// </summary>
            /// <param name="aSecretName">The name of the secret in the key vault</param>
            /// <returns>The value of the secret</returns>
            public string GetKeyVaultSecret(string aSecretName)
            {
                var lSecretUri = string.Format("{0}{1}", mVaultUrl + "/secrets/", aSecretName);
                var lKeyVaultClient = new KeyVaultClient(new KeyVaultClient.AuthenticationCallback(GetAccessToken));

                return lKeyVaultClient.GetSecretAsync(lSecretUri).Result.Value;
            }

            /// <summary>
            /// Constructs an object
            /// </summary>
            /// <param name="aThumbprint">The thumbprint of the certificate installed on the Azure server</param>
            /// <param name="aClientApplicationId">The Id of the client application, which has an access to the key vault</param>
            /// <param name="aVaultUrl">The Url of the key Vault</param>
            public XAzureKeyVaultCertificate(string aThumbprint, string aClientApplicationId,
                string aVaultUrl)
            {
                mThumbprint = aThumbprint;
                mClientApplicationId = aClientApplicationId;
                mVaultUrl = aVaultUrl;
            }
        }
    }
}

You will have to add the following two packages to your application. You can install them from the NuGet package manager

• Microsoft.Azure.KeyVault
• Microsoft.IdentityModel.Clients.ActiveDirectory

The member variables

I have added the following three things as member variables. You need to provide them in order to access the Key Vault. Here is also explanation on how to get them:

• Thumbprint – the thumbprint of the certificate. You can get it from the Home panel >> Azure Active Director >> App registrations >> The app which we created >> Certificates & secrets. In that panel you will see a list with all the certificates you have installed and their thumbprint will be in the first column.
• ClientApplicationId – The Id of the application, which we created. You can get it from the Home panel >> Azure Active Director >> App registrations >> The app which we created. It will be displayed in that panel under Application (client) ID 
• VaultUrl – The URL of the key vault which we created. You can find it from All resources >> The key vault which we created. In that panel it is found under the DNS Name. In the provided code I have already added the last “/” so provide the DNS without it. 

I have written some helper methods to obtain the certificate and access token. And there is one public method to get a secret based on its name.  

Exemplary usage of the code

After you have this library ready, you can test it by adding the following lines in your application:

public static void Main(string[] args)
{
  // Some other code
    XAzureKeyVaultCertificate lKeyVault = new XAzureKeyVaultCertificate("CF56463AD983901289DFEADE78789DEAD7879DCA",
                    "76fa6578-9321-6f6e-911a-89898998bad9", "https://myvault.vault.azure.net");
     string lMySecret = lKeyVault.GetKeyVaultSecret("MyFirstSecret");
     Console.WriteLine(lMySecret);
  // Some other code
}

Here I have provided the thumbprint, client Id and the URL directly in the code but you can go ahead and add it from the configuration or however you like.

Using the Azure Key Vault in production

After we have tested our application in development environment and we have verified that everything works fine and we are able to retrieve production secrets from Azure Key Vault, now it’s time to move to production and host our application on some server. So far we have used a self-signed certificate, which we created and installed on our PCs. Unfortunately, the hosting providers do not allow such certificates and you have to buy one from authorized provider.

Getting a certificate for the application in production

There are free certificates available from Let’s Encrypt . Follow their instructions on how to get a certificate. 

Another option is to check your hosting provider if you already have a certificate. If your web site uses SSL already, there is a good chance you can re-use that certificate. Some hosting providers like SmarterAsp have the option to export an existing certificate, which is used on your website. The certificate should be exportable as .pfx file. My web site had a SSL installed so I exported the certificate from the Control Panel.  

So whether you are going to buy a certificate from authorized provider or get it for free from Let’s Encrypt or your hosting provider, the final goal is to get a certified .pfx file.

Generating the .cert file from the .pfx certificate

Once you have the .pfx file, install it on your PC as it was shown above. You will need this in order to create a .cert file, which needs to be uploaded to Azure. 

After the certificate is installed, open the Manage user certificates program.  

Find the certificate which you installed, right – click on it then All Tasks >> Export…

Click Next, select the No, do not export the private key option and then Next again. Select the first option: DER encoded binary X.509 (.CER) and click Next. Provide a file name to save the certificate then click Next. Click the Finish button and you are all set. You will have a .cer file.

Upload the .cer file to Azure as it was shown in the above section: Install the certificates

Once you are done with this you can upload your application to your web hosting and test it. Before that make sure to use the appropriate thumbprint with the new certificate in your code. Once the certificate has been installed on your PC you can get the thumbprint by opening the Manage user certificates, finding the certificate there, clicking twice on it and then from the Details tab you will find information about the thumbprint.

Don’t forget to ask your hosting provider support to provide access to the certificate for your application. If everything is done you should be able to retrieve secrets from Azure Key Vault in your production environment.

Conclusion

In this post I showed you how to create an Azure Key Vault, provide access to it for your applications using certificates and how to store/retrieve secrets from it. I have given some code to implement on your side. If you have done everything as explained, you will be able to retrieve your secrets from Azure. If you have any problems, please, let me know in the comments section.

Ahmed Karaibrahimov

Passionate developer, loving husband and caring father. As an introvert programming is my life. I work as a senior software engineer in a big international company. My hobbies include playing computer games (mostly World of Warcraft), watching TV series and hiking.

www.xequtor.com