Secret management using Pulumi ESC SDK and Azure Key Vault

This is a submission for the Pulumi Deploy and Document Challenge: Shhh, It's a Secret! What I Built I built an automated infrastructure provisioning solution using Pulumi and Azure. The project leverages Pulumi's Infrastructure as code (IaC) to deploy Azure resources such as a virtual machine (VM) and network and manage secrets with Pulumi ESC. The infrastructure provisions a virtual network, a subnet, a public IP, and a Linux-based VM and integrates with Azure Key Vault to handle credentials securely. Live Demo Link Project Repo This project can be replicated in your own Azure environment by following the steps in the project repository. The full code can be found here I created a VM, kept the secrets in Azure Key Vault replicated it in Pulumi ESC SDK, and used the credentials with the to provision script. My Journey The process of building this solution was an enlightening experience. Initially, I faced some challenges with managing Azure resources using Pulumi, especially when dealing with multiple secret management systems. One of the major hurdles was working with Pulumi ESC for secure secret handling and ensuring that the right credentials were passed into the VM during provisioning. Here’s how I overcame them: Challenge with Azure Key Vault: Fetching secrets securely from Azure Key Vault required the use of Pulumi's integration with the azure_native package and keyvault.get_secret. The complexity came from ensuring that these secrets were properly passed into the virtual machine for secure authentication. ESC Integration: I struggled with integrating Pulumi ESC effectively, but after reading the documentation and experimenting, I successfully automated secret management with Pulumi ESC, creating or updating environments as needed. What I learned: How to work with Pulumi to provision Infrastructure as Code (IaC) in Azure. How to use Pulumi ESC for secure secret management, and how it integrates with other tools. The importance of correctly structuring and testing infrastructure code to ensure repeatability and security. Using Pulumi ESC I utilized Pulumi ESC (External Secret Control) to securely manage sensitive data such as admin credentials. By storing sensitive information in Azure Key Vault, I was able to pull these secrets into the Pulumi project and inject them into the virtual machine during provisioning. This is the snippet of code that retrieves credentials from Azure Key Vault and insert them into Pulumi ESC # === Get secrets from Azure Key Vault === key_vault_id = f"/subscriptions/{subscription_id}/resourceGroups/{resource_group_name}/providers/Microsoft.KeyVault/vaults/{key_vault_name}" admin_username_secret = azure.keyvault.get_secret(name="adminUsername", key_vault_id=key_vault_id) admin_password_secret = azure.keyvault.get_secret(name="adminPassword", key_vault_id=key_vault_id) admin_username = admin_username_secret.value admin_password = admin_password_secret.value # === Upload secrets to ESC === env_def.values.additional_properties = { "adminUsername": {"fn::secret": admin_username}, "adminPassword": {"fn::secret": admin_password}, } client.update_environment(org_name, project_name, esc_env_name, env_def) Here’s how Pulumi ESC helped: Secret Management: Instead of manually entering sensitive information into my configuration files, Pulumi ESC provided a secure and automated way to fetch secrets. Environment Definitions: I used ESC to define an environment where these secrets could be securely updated. This is essential for managing different environments (e.g., development, staging, production) securely. Special Thanks: To the Pulumi community for their ESC SDK examples and the ESC team for their exceptional documentation

Apr 7, 2025 - 04:43

Secret management using Pulumi ESC SDK and Azure Key Vault

This is a submission for the Pulumi Deploy and Document Challenge: Shhh, It's a Secret!

What I Built

I built an automated infrastructure provisioning solution using Pulumi and Azure. The project leverages Pulumi's Infrastructure as code (IaC) to deploy Azure resources such as a virtual machine (VM) and network and manage secrets with Pulumi ESC. The infrastructure provisions a virtual network, a subnet, a public IP, and a Linux-based VM and integrates with Azure Key Vault to handle credentials securely.

Live Demo Link

Project Repo

This project can be replicated in your own Azure environment by following the steps in the project repository.

The full code can be found here

I created a VM, kept the secrets in Azure Key Vault replicated it in Pulumi ESC SDK, and used the credentials with the to provision script.

My Journey

The process of building this solution was an enlightening experience. Initially, I faced some challenges with managing Azure resources using Pulumi, especially when dealing with multiple secret management systems. One of the major hurdles was working with Pulumi ESC for secure secret handling and ensuring that the right credentials were passed into the VM during provisioning.

Here’s how I overcame them:

Challenge with Azure Key Vault: Fetching secrets securely from Azure Key Vault required the use of Pulumi's integration with the azure_native package and keyvault.get_secret. The complexity came from ensuring that these secrets were properly passed into the virtual machine for secure authentication.
ESC Integration: I struggled with integrating Pulumi ESC effectively, but after reading the documentation and experimenting, I successfully automated secret management with Pulumi ESC, creating or updating environments as needed.

What I learned:

How to work with Pulumi to provision Infrastructure as Code (IaC) in Azure.
How to use Pulumi ESC for secure secret management, and how it integrates with other tools.
The importance of correctly structuring and testing infrastructure code to ensure repeatability and security.

Using Pulumi ESC

I utilized Pulumi ESC (External Secret Control) to securely manage sensitive data such as admin credentials. By storing sensitive information in Azure Key Vault, I was able to pull these secrets into the Pulumi project and inject them into the virtual machine during provisioning.

This is the snippet of code that retrieves credentials from Azure Key Vault and insert them into Pulumi ESC

# === Get secrets from Azure Key Vault ===
key_vault_id = f"/subscriptions/{subscription_id}/resourceGroups/{resource_group_name}/providers/Microsoft.KeyVault/vaults/{key_vault_name}"

admin_username_secret = azure.keyvault.get_secret(name="adminUsername", key_vault_id=key_vault_id)
admin_password_secret = azure.keyvault.get_secret(name="adminPassword", key_vault_id=key_vault_id)

admin_username = admin_username_secret.value
admin_password = admin_password_secret.value

# === Upload secrets to ESC ===
env_def.values.additional_properties = {
    "adminUsername": {"fn::secret": admin_username},
    "adminPassword": {"fn::secret": admin_password},
}

client.update_environment(org_name, project_name, esc_env_name, env_def)

Here’s how Pulumi ESC helped:

Secret Management: Instead of manually entering sensitive information into my configuration files, Pulumi ESC provided a secure and automated way to fetch secrets.
Environment Definitions: I used ESC to define an environment where these secrets could be securely updated. This is essential for managing different environments (e.g., development, staging, production) securely.

Special Thanks:

To the Pulumi community for their ESC SDK examples and the ESC team for their exceptional documentation