Can you keep a Secret, Micro Integrator?

Protecting your secrets (passwords) is an essential task when managing enterprise software. When your system is connecting to multiple systems, each system will have some kind of authentication and you will have some kind of secret(password, keys) that will provide you with access. In this blog, I will be explaining how WSO2 Micro Integrator protects your secret and how you can manage them in standalone and container-based deployments.

Normally it is not advisable to use plain values for your secret data in your configurations. Therefore, you can encrypt the plain-text secrets using the secure vault that is built into the Micro Integrator runtime. Running the Cipher Tool also ensures that the secrets are enabled in the Micro Integrator environment. You can then refer to the encrypted secrets from anywhere in your server configurations or synapse artifacts. Also, you might be having separate APP IDs for different environments(Dev, Staging, Prod). Hard coding the values into the synapse configuration will lead you to maintain separate artifacts for each environment.

For the purpose of explanation, I will be using an example that I used in one of my previous blog. There we used a OpenWeather APP ID and we stored the APP ID in a Property mediator. Since the deployed API can be viewed via the Monitoring dashboard, it is not advisable to store the APP ID as plain-text. I will be using this example to demonstrate how you can encrypt your secret in different scenarios.

Please note that the below examples are illustrated with Micro Integrator 1.2.0 and Integration Studio 7.1.0. It might be different for other versions.

1) Storing the secret in the deployment.toml file as a static value

One simple way to handle secrets is to define them in the deployment.toml file and encrypt the secrets using the CipherTool that is available inside MI runtime.

First, you need to define the secret under the secrets section like below. (Please remember to enter the secret inside square brackets)

[secrets] 
APP_ID = "[<YourOwnAppID>]"

Then you need to run the CipherTool by running the following command from the MI root directory. (Use the .bat file if you are on windows). You need to enter the password of the keystore file that you are using. (Default value : wso2carbon)

./bin/ciphertool.sh -Dconfigure 

If you look at the deployment.toml file, you can see that the secret you entered has now been encrypted. Also, the APP_ID would have been stored in the secure vault. (After this step, the server will prompt you for the keystore password during server startup)

You can now access this APP_ID by doing a vault-lookup in the API like below.

<property name="uri.var.appid" scope="default" expression="wso2:vault-lookup('APP_ID')"/>

On a further note, if you want to use the secret inside deployment.toml file itself, you can use it like below.

[keystore.tls]
file_name = "wso2carbon.jks"
password = "$secret{keystore_pass}"
[secrets]
keystore_pass   = "<encrypted_password>"

2) Configuring the Embedded Micro Integrator with secrets inside Integration Studio

Integration Studio is the primary development tool for Micro Integrator. There is an embedded Micro Integrator packed into the Studio which helps you to test the solutions then and there without deploying it into a Micro Integrator runtime. Normally we do not edit the deployment.toml file of the Integration studio directly. We use the wizard that comes up with the Studio. If you want to test a solution with a secret, then you can use the Cipher tool that is packed with the Studio.

For example, if you want to encrypt the APP ID key that I mentioned previously in my blog, you have to add the following section in the deployment.toml and then click on Encrypt Secrets, which will enable the Secure vault inside the embedded Micro Integrator and encrypt the secret.

[secrets] 
APP_ID = "[<YourOwnAppID>]"

Using Integration Studio wizard to encrypt secrets

You can then refer to it inside the API as we did in section 1.

Another primary use case of Integration Studio is to create Micro Integrator docker images with your Integration artifacts packed. It is possible to encrypt the secret that is defined inside deployment.toml while creating the docker image.

Steps to pack a deployment.toml with encrypted secrets

1. Define the secret in the deployment.toml file inside the DockerExporterProject.
2. In the pom.xml, place the tick on ‘Required for secrets encryption and decryption’.
3. After that, click on the Build Image in the top right corner.

Now the image would contain the encrypted secret and you can use it in the configurations and synapse artifacts as mentioned in Section 1 of this blog. (You do not need to provide the Keystore password during server startup as we did previously in a standalone runtime)

Encrypting secrets while building the docker image

Note: The above steps would apply to the Kubernetes Exporter project as well.

3) Using Dynamic Secrets with different environments

Dynamic secrets are specified in configurations and artifacts as environment variables, system properties, Docker secrets, or Kubernetes secrets. If you want to maintain environment-specific secrets, then you can use the Micro Integrator CLI tool to encrypt the secret and store it appropriately in your environment.

In all of the below examples, I will be using Micro Integrator CLI to encrypt the secret. You can download it from here under other resources.

You need to initialize the MI CLI with the below command to encryptt the secrets.

mi secret init

This will ask for the Keystore file location, type, and password. You need to use this initialized CLI in the following scenarios.

• Using dynamic secrets in VM based deployments

In a VM based deployment, you can store the secrets in the instance by exporting them as an environment variable or passing them as a system variable during server startup. Either way, you need to encrypt the secret first. To encrypt the secret you can use the MI CLI tool in the following way.

# To encrypt secret and get output to console 
mi secret create

When you enter the above command, it will ask for the alias and the secret and it will print out the encrypted secret to the console.

➜  wso2mi-cli-1.2.0 ./bin/mi secret create
Enter plain alias for secret:APP_ID
Enter plain text secret:
Repeat plain text secret:
APP_ID : pu2vkcFW6LgLns869pUzT9xnYEJqGsUmvucwKgoCH5689EkVQkDYAXwtXxUooz58BmozDZ1A876RIiVyVrE9wcmheSJq14NPMl72QCDUuS5P1FqbiaGqQOqPoGqGJVOx0ldI+RY/vXvVaOqDHaFbQfEyz6hTXhsC3fk4S/b8r0qLdCt1Z8OlzaLc01BW06G02y5HMLqHIGTDxy7VVcolkoNCaREP4bViI9AmYnm2+t9IHsKTgfui+xon5/yUHBXYmB85Ue9T8lIwTlRfPInvWwKnX9DCq5KwUFxanTrKt4UxifX3gVzYCki9PUYEJBDqTya1zBzIaQuVIeqH/xK36A==
Encryption completed successfully...

Furthermore, if you want to output the password to a file, or you have got a load of secrets to encrypt, then you can use the following commands.

# To encrypt secret and get output to file 
mi secret create file 
 
# To bulk encrypt secrets defined in a properties file 
mi secret create -f=</file_path>

Once you have encrypted the secrets, you can store them in your environment in a suitable manner. You can refer to them in the deployment.toml as below.

# If it is an environment variable
[secrets] 
APP_ID = "$env{env_var_key}"
# If it is a system variable
[secrets]
APP_ID = "$sys{sys_var_key}"

This way you can ensure that your MI synapse configuration remains the same across different environments and the secrets can be provided dynamically.

• Using dynamic secrets in Docker-based deployments

In docker deployments, we can maintain different docker secrets in each environment. In Micro Integrator, you can directly refer to the docker secret from the synapse configuration.

Below is the command to create the docker secret in Docker environment.
echo “<YourOwnAPPIDencryptedwithCLI>” | docker secret create weather_appid -

Once you have created docker secrets, you can refer to them directly in your synapse configurations.

<property expression="wso2:vault-lookup('weather_appid', 'DOCKER', 'true')" name="uri.var.appid"/>

Here, the third argument ‘true’ indicates that you passed an encrypted value to the docker secret. If the value is not encrypted using the Ciphertool/CLI, you can make it to ‘ false’.

• Using dynamic secrets in Kubernetes deployments

Similar to Docker secrets, we can use Kubernetes secrets to maintain different secrets in different environments. We can directly use the MI CLI to generate the kubernetes secret file.

# To encrypt secret and get output as a .yaml file 
mi secret create k8

Once you have generated the secret file, you can apply it in your kubernetes environment. After you apply it, you need to refer to it inside your integration_cr.yaml file.

apiVersion: "integration.wso2.com/v1alpha1" 
kind: "Integration" 
metadata:     
   name: "weathermiintegration" 
   spec:  replicas: 1     
   image: "arunans23/testmiweather:1.0.1"     
   env:         
       -  name: APP_ID            
          valueFrom:                 
          secretKeyRef:                     
            name: wso2misecret                    
            key: APP_ID

This will add the secret as an environment variable to the Kubernetes deployment. You can then refer to it in your synapse artifacts as below.

<property expression=”wso2:vault-lookup(‘APP_ID’, ‘ENV’, ‘true’)” name=”uri.var.appid”/>

If you are not familiar with EI Kubernetes Operator please refer to the documentation.

References

WSO2 Enterprise Integrator

WSO2 Integration Studio

Encrypting Secrets

The Love story of Integration Studio and Micro Integrator

Life of MI : How an ESB went on to serve Microservices?