Securely Storing Secrets on Vespa Cloud
Vespa applications sometimes need to interact with external services, be it for retrieving external data, authenticating users, or running specialized functions. These interactions often require secrets such as API keys, tokens, or private configuration values - essential pieces of information that need to be securely stored and managed.
Consider a common example: integrating a third-party large language model (LLM) API. This typically requires an API key. Safeguarding such secrets is important; we don’t want them to be exposed or passed around through URLs or query parameters. Vespa’s secret store provides a secure, built-in way to handle this, making it easier to manage sensitive information without compromising your application’s security.
In this blog post, we’ll explore how to set up the secret store on Vespa Cloud, and how to access these secrets in your application. Note that the secret store leverages functionality found only on Vespa Cloud.
Setting Up Secrets
Setting up secrets starts with configuration in the Vespa console. The process is straightforward: you can define a secret by navigating to the account settings for your tenant. Under the secret store section, you first add a vault to contain a set of secrets. Then you simply add a new secret with a name, a value and a description. The value can be anything as long as it’s representable as a string.
You can control which application in your tenant has access to which secrets by using the access control settings associated with each vault.
Note that after the secret has been stored, it can only be updated or deleted; the actual value is never viewable again. After updating a secret, any application using the secret will have the new value within 60 seconds.
Using Secrets
Once the secret is configured, it becomes available for your Vespa Cloud application, but in a secure way that ensures it won’t be inadvertently exposed. You can reference these secrets within your Java code by leveraging the Secrets API, ensuring the secrets are always securely retrieved and accessed only by the intended components.
<services version="1.0">
<container id="default" version="1.0">
<secrets>
<mySecret vault="my-vault" name="my-secret" />
</secrets>
</container>
</services>
Here, we refer to the secret my-secret in the vault my-vault with the alias
mySecret which is what is used in your application.
Some components and searchers in Vespa can utilize secrets directly. One example is the RAG searcher which is used for setting up RAG chains in Vespa. This component uses an LLM client for content generation and the OpenAI client can retrieve the OpenAI API key directly from the secret store. The key is thus securely stored in Vespa’s secret store, and can be used by any application in the tenant, as long as the access controls have been set.
Conclusion
Vespa Cloud’s secret store offers a secure, integrated solution that makes it simple and safe to manage sensitive secrets. By setting up secrets directly through the console, and using them in your Java code or leveraging built-in components like the LLM searcher, you can ensure that your application stays secure while interacting with external services. Plus, by no longer passing sensitive information via queries, you avoid potential pitfalls related to logging or inadvertent exposure.
This feature also opens up new possibilities, such as connecting to external APIs during data feeding, something we’ll cover in more detail in an upcoming blog post.
For more information, please refer to the secret store documentation and try the retrieval-augmented-generation sample application with your OpenAI API key!
Read more
Vespa and LLMs
Introducing LLM support in Vespa using both external and local LLMs
Secret store
Vespa Cloud supports secure storage and management of secrets for use in your application, by organizing secrets into vaults, setting application-specific access controls.