This tutorial details how to manage secrets effectively using Azure Key Vault. You'll learn how to create secrets and access them in both virtual machines and Kubernetes clusters.

Greetings! In my upcoming series of articles, I will delve into tutorials and cheat sheets on managing secrets effectively using secrets managers offered by the leading public cloud providers: AWS, GCP, and Azure.

To begin our journey, let's kick it off with Azure first.

But instead of getting hands-on with the practical aspects immediately, it's essential to establish a solid understanding of general data security principles. Let's get started.

1. Zero Trust

The fact that you probably have already heard about it shows how important it is nowadays.

TL;DR: Zero Trust is a security strategy.

I know the word "strategy" doesn't really say much, so let's get into it a bit more. Zero Trust isn't a product or a service, but rather, an approach, a framework, a model, or even a philosophy in system design, based on one simple principle: never trust, always verify.

Instead of unconditionally trusting that everything behind the corporate firewall is safe, the Zero Trust model assumes breach and verifies each request as though it originated from an uncontrolled party. Regardless of where the request originates or what resource it accesses, the Zero Trust model encourages us to "never trust, always verify".

To be more specific, there are three main pillars of zero trust:

• Verify explicitly: always authenticate and authorize based on all available data points.
• Least privilege access: limit user access with Just-In-Time (JIT) and Just-Enough-Access (JEA), risk-based adaptive policies, and data protection.
• Assume breach: minimize blast radius and segment access. Verify end-to-end encryption and use analytics to get visibility, drive threat detection, and improve defenses.

And these are the cores of Zero Trust.

2. Secure App and Data with Zero Trust

In a software system, two of the major components are applications and data. Apps provide the interface to consume data, and data is ultimately what security teams are protecting. So, achieving secure apps and secure data are two of the main goals of the Zero Trust mode.

There are several detailed aspects regarding apps and data:

• Secrets management: apps need access to databases and services using secrets and API keys, which, if stored in plaintext or even hardcoded in code, could increase the risk of unauthorized access in case of a security breach.
• Key management: crucial for protecting sensitive data by data encryption.
• Certificate management: since apps talk to each other, cert management is vital for securing network communications with SSL/TLS certificates.

In summary, secrets management, key management, and certificate management are essential components of a robust security infrastructure, enabling secure storage, controlled access, and encryption of sensitive information in cloud environments. And, they all need to follow the three principles of Zero Trust: verify explicitly, least privilege access, and assume breach. To achieve this goal, dedicated tools are required, and this is where Azure Key Vault comes in.

3. What is Azure Key Vault?

Microsoft Azure offers several options for storing and managing your keys in the cloud, and Key Vault is one of the leading secrets management solutions available. Key Vault is more than just a secrets manager; it does secrets management, key management, and certificate management. It's more of a key management solution in Azure.

For the secrets manager part, Azure Key Vault offers centralized storage for application secrets, reducing the risk of accidental leaks and eliminating the need to embed security information in application code.

There are a few reasons you might want to consider Key Vault:

• Proper authentication and authorization through Azure Active Directory and Azure RBAC.
• Keys and secrets can be software-protected or hardware-protected using HSMs for added assurance.
• Simple administration, high availability through replication.
• Key Vault access can be monitored by enabling logging.
• If you are already using other Azure services, Key Vault integrates well with them, like Azure Disk Encryption, SQL server, and Azure App Service.

Today, let's focus on the secrets management part only, and do a hands-on tutorial.

4. How Azure Key Vault Works?

Key Vault provides secure storage of generic secrets, such as passwords and database connection strings.

All secrets in Key Vault are stored encrypted. Key Vault encrypts secrets at rest with a hierarchy of encryption keys, with all keys in that hierarchy protected by modules that are FIPS 140-2 compliant. The encryption is transparent and requires no action from the user. The Azure Key Vault service encrypts your secrets when you add them, and decrypts them automatically when you read them.

Key Vault accepts data, encrypts it, stores and manages secrets as sequences of octets (8-bit bytes), and returns a secret identifier (id). The identifier can be used to retrieve the secret at a later time.

Access Control for secrets managed in Key Vault is provided at the level of the Key Vault that contains those secrets. The access control policy for secrets is distinct from the access control policy for keys in the same Key Vault. Users may create one or more vaults to hold secrets and are required to maintain scenario-appropriate segmentation and management of secrets.

Azure Secrets Management Best Practices and Security Considerations

While Azure Key Vault provides robust secrets storage capabilities, implementing effective Azure secrets management requires adherence to security best practices that extend beyond basic setup. Organizations must establish comprehensive access control policies that follow the principle of least privilege, ensuring engineers and applications only access the secrets they absolutely need. This approach significantly reduces the attack surface and potential for credential compromise.

Key security considerations include implementing proper secret rotation policies, establishing audit trails for all secret access, and ensuring high availability for critical applications. The centralization of secrets in Azure Key Vault should be coupled with standardized lifecycle management processes that define how secrets are created, updated, and retired. Additionally, organizations should implement monitoring and alerting mechanisms to detect unusual access patterns or potential security breaches. When designing your Azure secrets management strategy, consider implementing multiple layers of authentication and authorization, including Azure Active Directory integration and role-based access controls (RBAC) to maintain strict governance over sensitive credentials.

5. How to create a Key Vault using Azure CLI?

The easiest way to use Azure CLI is via the Azure Cloud Shell. Simply go to the Azure portal, click the "Cloud Shell" button in the top-right corner of the navigation bar, and choose "Bash".

5.1 Create a Resource Group

In Azure, a resource group is a logical container into which Azure resources are deployed and managed. We will create a single resource group and use it across this whole tutorial so that it's easy for you to visualize and manage all the resources created in this tutorial.

Run the following command in the cloud shell:

az group create --name "myResourceGroup" --location "EastUS"

5.2 Create a Key Vault

Note: each key vault must have a unique name. Replace the name ("tiexin-keyvault-test") with your own unique name in the following command examples.

Then, let's create a Key Vault using the "myResourceGroup" resource group:

az keyvault create --name "tiexin-keyvault-test" --resource-group "myResourceGroup" --location "EastUS"

5.3 Create a Secret in the Key Vault

Run the following command to create a plain-text secret named "ExampleSecret" with the value "MyAKSExampleSecret" in the Key Vault named "tiexin-keyvault-test":

az keyvault secret set --vault-name "tiexin-keyvault-test" -n ExampleSecret --value MyAKSExampleSecret

Please take note of the Azure tenant ID that the subscription belongs to, which will be used later.

We can verify the secret is created successfully by reading it:

az keyvault secret show --vault-name "tiexin-keyvault-test" -n ExampleSecret

The az keyvault secret show command gets a specified secret from a given key vault.

6. Access Secrets from Virtual Machines

In this section, let's access secrets in Key Vault from a virtual machine.

6.1 Create a VM and Assign Permissions

Now, let's use the az vm create command to create a virtual machine, add a user account named "azureuser", generate an SSH key automatically, and put it in the default key location (~/.ssh):

az vm create \
  --resource-group myResourceGroup \
  --name myVM \
  --image UbuntuLTS \
  --admin-username azureuser \
  --generate-ssh-keys

Take note of the public IP of the VM.

Please note that this is only a tutorial for demo purposes; do not do this (exposing VM to the public internet) in a production environment without knowing the consequences.

The VM needs an identity, and permissions can be assigned to the identity so that the VM can access Key Vault using those permissions defined in the identity.

Create a system-assigned identity for the VM

az vm identity assign --name "myVM" --resource-group "myResourceGroup"

Copy the "systemAssignedIdentity" value from the output, and assign the previously created identity permissions to the key vault:

az keyvault set-policy --name "tiexin-keyvault-test" --object-id "551c51fd-f86a-41c8-ac81-d47e4f09c4af" --secret-permissions get list

6.2 Create a Sample App to Read the Secret

Login to the VM (replace the public IP address with your own value):

ssh azureuser@172.190.235.224

On the virtual machine, let's write a simple Python app to read the secret. First, install the two Python libraries we'll be using:

sudo apt update && sudo apt upgrade -y
sudo apt install python3-pip
python3 -m pip install --upgrade pip
pip3 install azure-keyvault-secrets
pip3 install azure.identity

Create the Python app main.py, and remember to replace the Key Vault name with your own value:

from azure.keyvault.secrets import SecretClient
from azure.identity import DefaultAzureCredential

key_vault_name = "tiexin-keyvault-test"
key_vault_uri = f"https://tiexin-keyvault-test.vault.azure.net"
secret_name = "ExampleSecret"

credential = DefaultAzureCredential()
client = SecretClient(vault_url=key_vault_uri, credential=credential)
retrieved_secret = client.get_secret(secret_name)

print(f"The value of secret '{secret_name}' in '{key_vault_name}' is: '{retrieved_secret.value}'")

Lastly, run main.py. If all has gone well, it should return the value of the secret:

python3 main.py

The value of secret 'ExampleSecret' in 'tiexin-keyvault-test' is: 'MyAKSExampleSecret'

7. How to Access Secrets from AKS?

In the previous section, we accessed the secret from a VM successfully. In the cloud-native era, we are likely running our workload in containerized applications, in, say, Azure AKS clusters. So, let's try to access the same secret, but this time, from an AKS cluster, building on fundamental Kubernetes secrets management principles.

7.1 Create an AKS Cluster

Create an AKS cluster with Azure Key Vault Provider for Secrets Store CSI Driver capability:

az aks create -n myAKSCluster -g myResourceGroup --node-count 1 --enable-addons azure-keyvault-secrets-provider

An identity azureKeyvaultSecretsProvider is created by the add-on to access Azure resources. Take note of the identity's clientId in the output, which will be used to access the Key Vault.

Verify the installation is finished:

kubectl get pods -n kube-system -l 'app in (secrets-store-csi-driver,secrets-store-provider-azure)'

Then, set the policy to access secrets in your key vault

az keyvault set-policy -n tiexin-keyvault-test --secret-permissions get --spn a87c1e0f-ca8e-4064-abbf-b2749c333d65

7.2 Create the SecretProviderClass

Next, we will use Secrets Store CSI Driver for Kubernetes secrets to access secrets from Key Vault. CSI driver integrates secrets stores with Kubernetes via a Container Storage Interface (CSI) volume. For more information, read the official doc here.

Let's create a SecretProviderClass using the following YAML; make sure to use your own values for userAssignedIdentityID, keyvaultName, tenantId, and the objects to retrieve from your key vault:

File spc.yaml:

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: azure-kvname-user-msi
spec:
  provider: azure
  parameters:
    usePodIdentity: "false"
    useVMManagedIdentity: "true"
    userAssignedIdentityID: a87c1e0f-ca8e-4064-abbf-b2749c333d65
    keyvaultName: tiexin-keyvault-test
    cloudName: ""
    objects:  |
      array:
        - |
          objectName: ExampleSecret
          objectType: secret
          objectVersion: ""
    tenantId: 3199f464-7cbd-4c6d-b60f-f87b69ac72a4

Apply it:

kubectl apply -f spc.yaml

Then create a test pod that uses the SPC to mount the secret as a file:

File pod.yaml:

kind: Pod
apiVersion: v1
metadata:
  name: busybox-secrets-store-inline
spec:
  containers:
    - name: busybox
      image: registry.k8s.io/e2e-test-images/busybox:1.29-1
      command:
        - "/bin/sleep"
        - "10000"
      volumeMounts:
      - name: secrets-store01-inline
        mountPath: "/mnt/secrets-store"
        readOnly: true
  volumes:
    - name: secrets-store01-inline
      csi:
        driver: secrets-store.csi.k8s.io
        readOnly: true
        volumeAttributes:
          secretProviderClass: "azure-kvname-user-msi"

Apply it:

kubectl apply -f pod.yaml

After the pod is up and running, we can verify that the secret is created and mounted as a file in the pod:

kubectl exec busybox-secrets-store-inline -- cat /mnt/secrets-store/ExampleSecret
MyAKSExampleSecret

Comparing Azure Key Vault with Alternative Secrets Managers

Understanding how Azure Key Vault positions against other cloud-native secrets management solutions helps organizations make informed architectural decisions. When evaluating AWS Secrets Manager vs Azure Key Vault, several key differentiators emerge beyond basic functionality. Azure Key Vault offers deeper integration with Microsoft's ecosystem, including seamless connectivity with Azure Active Directory, Azure DevOps, and other Azure services, making it particularly advantageous for organizations already invested in the Microsoft cloud platform.

The AWS Secrets Manager Azure equivalent comparison reveals that while both services provide similar core capabilities—encrypted storage, automatic rotation, and programmatic access—their implementation approaches differ significantly. Azure Key Vault's hardware security module (HSM) backing provides FIPS 140-2 Level 2 compliance by default, whereas AWS Secrets Manager requires additional configuration for similar security levels. Additionally, Azure Key Vault's pricing model often proves more cost-effective for organizations with predictable access patterns, while AWS Secrets Manager's per-secret pricing can be advantageous for environments with fewer secrets but higher access frequency.

1. Ecosystem & Integration

Azure Key Vault strengths:

• Deep integration with Microsoft’s cloud ecosystem.
• Seamless connectivity with Azure Active Directory (AAD), Azure DevOps, and other Azure-native services.
• Especially beneficial for organizations already standardized on Microsoft/Azure.

AWS Secrets Manager positioning:

• Integrates naturally into AWS environments.
• Best suited for teams primarily operating within the AWS stack.

2. Core Capabilities (Similarities)

Both services provide the standard foundations of secrets management:

• Encrypted secrets storage
• Automatic rotation options
• Programmatic/API-based access for applications and pipelines

3. Security & Compliance Differences

Azure Key Vault:

• Backed by Hardware Security Modules (HSM) by default.
• Provides FIPS 140-2 Level 2 compliance out of the box.

AWS Secrets Manager:

• Comparable security levels are possible.
• But may require additional configuration to reach similar compliance posture.

4. Pricing & Cost Behavior

Azure Key Vault:

• Pricing model tends to be more cost-effective when access patterns are predictable and large numbers of secrets are stored and accessed steadily over time.

AWS Secrets Manager:

• Per-secret pricing can be advantageous when fewer secrets are stored but access frequency is high per secret.

Integration Patterns for Azure DevOps and CI/CD Pipelines

Modern Azure DevOps secrets management requires integration patterns that maintain security while enabling automated deployment workflows. Azure Key Vault integrates natively with Azure DevOps through service connections and variable groups, allowing secrets to be securely injected into build and release pipelines without exposing them in pipeline definitions or logs. This integration supports both classic and YAML-based pipelines, providing flexibility for different organizational preferences.

The Azure DevOps secrets manager integration enables advanced scenarios such as environment-specific secret management, where different Key Vault instances can be used for development, staging, and production environments. Pipeline security can be further enhanced by implementing approval workflows for production secret access and using Azure DevOps service principals with time-limited permissions. Organizations should establish clear governance for which secrets are accessible in which environments, and implement automated secret rotation that aligns with deployment schedules. This approach ensures that secrets remain fresh while minimizing the risk of deployment failures due to expired credentials.

8. Clean Up

To delete everything created in this tutorial, run:

az group delete --name "myResourceGroup"

Note that the Azure cloud shell, AKS, and network watcher will also create their resource groups; do delete those if you wish.

9. Cheat Sheet

Here's a cheat sheet with some examples for quick references:

• Create resource group: az group create --name "myResourceGroup" --location "EastUS"
• Create Key Vault: az keyvault create --name "tiexin-keyvault-test" --resource-group "myResourceGroup" --location "EastUS"
• Create secret: az keyvault create --name "tiexin-keyvault-test" --resource-group "myResourceGroup" --location "EastUS"
• Read secret: az keyvault secret set --vault-name "tiexin-keyvault-test" -n ExampleSecret --value MyAKSExampleSecret

SecretProviderClass for Azure AKS example:

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: azure-kvname-user-msi
spec:
  provider: azure
  parameters:
    usePodIdentity: "false"
    useVMManagedIdentity: "true"
    userAssignedIdentityID: a87c1e0f-ca8e-4064-abbf-b2749c333d65
    keyvaultName: tiexin-keyvault-test
    cloudName: ""
    objects:  |
      array:
        - |
          objectName: ExampleSecret
          objectType: secret
          objectVersion: ""
    tenantId: 3199f464-7cbd-4c6d-b60f-f87b69ac72a4

Using SPC as volumes/volumeMounts in a pod:

kind: Pod
apiVersion: v1
metadata:
  name: busybox-secrets-store-inline
spec:
  containers:
    - name: busybox
      image: registry.k8s.io/e2e-test-images/busybox:1.29-1
      command:
        - "/bin/sleep"
        - "10000"
      volumeMounts:
      - name: secrets-store01-inline
        mountPath: "/mnt/secrets-store"
        readOnly: true
  volumes:
    - name: secrets-store01-inline
      csi:
        driver: secrets-store.csi.k8s.io
        readOnly: true
        volumeAttributes:
          secretProviderClass: "azure-kvname-user-msi"

Summary

In this tutorial, we had a look at the Zero Trust security strategy, how to use it to secure applications and data, and how the Azure Key Vault can help to achieve the Zero Trust goal. Then we did two tutorials to demonstrate how to create secrets in Key Vault and access them in both virtual machines and Kubernetes clusters.

If you are mainly using AWS or GCP rather than Azure, don't worry; in the next two articles, we will cover GCP Secret Manager as well as AWS Secrets Manager. Please like, comment, and subscribe. See you in the next article!

Additional resources about managing secrets

How to handle secrets in X

In this series of articles, GitGuardian specialists answer all the questions you never dared to ask on secrets management. How to Handle Secrets in TerraformDevOps engineers must handle secrets with care. In this series, we summarize best practices for leveraging secrets with your everyday tools.Gi…

GitGuardian Blog - Automated Secrets DetectionZiad Ghalleb

This article is a guest post. Views and opinions expressed in this publication are solely those of the author and do not reflect the official policy, position, or views of GitGuardian, The content is provided for informational purposes, GitGuardian assumes no responsibility for any errors, omissions, or outcomes resulting from the use of this information. Should you have any enquiry with regard to the content, please contact the author directly.