Storing 100k keys for bulk retrieval from Azure

We're setting up a device-to-cloud channel using TLS-PSK (Pre-Shared Key), where each device uses a unique key to encrypt communications. Each key is known only to the individual device that uses it and to the server, of course, since the key is symmetric. I'm used to working with devices that use individual X.509 certificates for both identity and encryption, but we couldn’t go that route due to commercial and technical constraints. Initially, I expected a process where the device connects to the server, sends the "name" or ID of the key it will use, and then begins encrypting communications using that key. The server would then retrieve the key from Azure Key Vault using the name/ID and use it to decrypt messages. However, this approach isn’t feasible due to technical limitations, so we'll need to retrieve all keys in bulk when the server starts. Azure Key Vault has a rate limit though, so you won't be able to retrieve 100k keys quickly. What are my options for storing these keys efficiently? So far I’ve considered: Storing the keys in a database, encrypted with a key managed by Azure Key Vault Storing all keys in a single encrypted file in Azure Storage Are there better alternatives? Thanks! Additional details The technical issue that forces us to bulk retrieve all keys is that the server is written in Node.js and the callback that processes an incoming TLS-PSK request only allows synchronous calls; but Key Vault calls are async only and the same goes e.g. for HTTP calls (there a few packages for sync HTTP calls but are labeled as not ok for production). The number of connections is highly variable, so we assume we'll have 10 connections per hour per device We have 10 server instances, each hosted in a container instance.

May 30, 2025 - 16:40

We're setting up a device-to-cloud channel using TLS-PSK (Pre-Shared Key), where each device uses a unique key to encrypt communications. Each key is known only to the individual device that uses it and to the server, of course, since the key is symmetric.

I'm used to working with devices that use individual X.509 certificates for both identity and encryption, but we couldn’t go that route due to commercial and technical constraints.

Initially, I expected a process where the device connects to the server, sends the "name" or ID of the key it will use, and then begins encrypting communications using that key. The server would then retrieve the key from Azure Key Vault using the name/ID and use it to decrypt messages. However, this approach isn’t feasible due to technical limitations, so we'll need to retrieve all keys in bulk when the server starts.

Azure Key Vault has a rate limit though, so you won't be able to retrieve 100k keys quickly. What are my options for storing these keys efficiently?

So far I’ve considered:

Storing the keys in a database, encrypted with a key managed by Azure Key Vault
Storing all keys in a single encrypted file in Azure Storage

Are there better alternatives?

Thanks!

Additional details

The technical issue that forces us to bulk retrieve all keys is that the server is written in Node.js and the callback that processes an incoming TLS-PSK request only allows synchronous calls; but Key Vault calls are async only and the same goes e.g. for HTTP calls (there a few packages for sync HTTP calls but are labeled as not ok for production).

The number of connections is highly variable, so we assume we'll have 10 connections per hour per device

We have 10 server instances, each hosted in a container instance.