Managing Secrets in Ansible Playbooks: A Comprehensive Guide

When you first begin your Ansible journey, you might be tempted to hardcode secrets directly into your playbooks for simplicity. However, as your automation practices mature, properly securing sensitive information becomes critical.

Fortunately, Ansible offers several built-in mechanisms for handling secrets, as well as integration options with external secret management systems.

This guide will walk you through progressively sophisticated approaches to secret management in Ansible:

1. Using interactive prompts for manual playbook execution
2. Employing Ansible Vault for encrypted storage
3. Integrating with dedicated secret management platforms

We'll also cover important security best practices to ensure your secrets remain protected throughout the automation lifecycle.

Using interactive prompts

If you're just beginning with Ansible or for playbooks that are always run manually, interactive prompts provide a straightforward solution.

This approach causes Ansible to request sensitive information from the user at runtime, avoiding the need to store secrets within playbook files.

Implementing vars_prompt

To implement an interactive prompt, you'll use the vars_prompt directive in your playbook. Here's a basic example that prompts for database credentials:

- name: Configure database connection
  hosts: database_servers
  gather_facts: false
  vars_prompt:
    - name: db_user
      prompt: Enter database username
      private: false

    - name: db_password
      prompt: Enter database password
      private: true
      confirm: true

  tasks:
    - name: Update database configuration file
      template:
        src: database.conf.j2
        dest: /etc/application/database.conf
        owner: app_user
        group: app_group
        mode: '0640'

The private: true parameter ensures that the entered text isn't displayed on screen while typing, which is important for passwords. The confirm: true parameter will ask the user to enter the password twice to avoid mistakes.

When you run this playbook, Ansible will prompt you for the required information:

ansible-playbook database_config.yml

Enter database username: admin
Enter database password:
Confirm Enter database password:

PLAY [Configure database connection] ***************************

TASK [Update database configuration file] **********************
changed: [db1.example.com]

PLAY RECAP ***************************************************
db1.example.com           : ok=1    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

The template file referenced in the task might look something like this:

# Database connection configuration
# Generated by Ansible on {{ ansible_date_time.date }}

DB_HOST={{ db_host | default('localhost') }}
DB_PORT={{ db_port | default(5432) }}
DB_USER={{ db_user }}
DB_PASSWORD={{ db_password }}

Limitations of interactive prompts

While interactive prompts are easy to implement, they have significant limitations:

1. They require manual input, making them unsuitable for automated pipelines or scheduled runs
2. They don't provide a way to securely store the secrets for future use
3. They become cumbersome when managing multiple secrets across numerous playbooks

For these reasons, interactive prompts are best suited for:

• Initial setup procedures
• Rarely run maintenance tasks
• Development and testing environments
• Playbooks that are always executed manually

As your Ansible usage matures, you'll likely need a more robust solution for secret management.

Using Ansible Vault for native encryption

Ansible Vault provides built-in encryption capabilities that allow you to secure sensitive data while still keeping it alongside your playbooks. This approach strikes a good balance between security and convenience, making it a popular choice for many Ansible users.

Let's start with creating an encrypted file that contains database credentials:

db_user: admin
db_password: complex_password_123
api_key: a72bd98c66e4f5432bb90c6ad1b
ssl_private_key: |
  -----BEGIN PRIVATE KEY-----
  MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQC7VJTUt9Us8cKj
  MzEfYyjiWA4R4/M2bS1GB4t7NXp98C3SC6dVMvDuictGeurT8jNbvJZHtCSuYEvu
  Nmg2fFnTYUjQAlt4aUNl5jDWtLZVfMcZLu0XzXGo9T7HVWKgH95g3Gxz6EZtXmY0
  ...
  -----END PRIVATE KEY-----

To encrypt this file, use the ansible-vault encrypt command:

ansible-vault encrypt secrets.yml

You'll be prompted to create and confirm a password. Once completed, the file contents will be transformed into encrypted data:

$ANSIBLE_VAULT;1.1;AES256
7839616262613373935663839666463306231653861336630613938303662633538633836656465
3637353766613339663032363538626430316135623665340a653961303730353962386134393162
62343936366265353935346336643865643833353737613962643539373230616239346133653464
6435353361373263640a376632613336366430663761363339333737386637383961363833303830
34336535623736313031313162353831666139343662653665366134633832646661...

Using encrypted files in playbooks

With the secrets now encrypted, you can reference them in your playbook using the vars_files directive:

- name: Deploy application with secure credentials
  hosts: app_servers
  gather_facts: true
  vars_files:
    - secrets.yml

  tasks:
    - name: Create application configuration file
      template:
        src: app_config.j2
        dest: /opt/myapp/config.json
        owner: app_user
        group: app_group
        mode: '0600'

When running this playbook, you'll need to provide the vault password:

ansible-playbook deploy_application.yml --ask-vault-pass

The system will prompt you for the password, decrypt the secrets, and make them available as variables during playbook execution.

Working with encrypted files

There are several ways to work with encrypted files:

In-place editing

To edit an encrypted file without having to manually decrypt and re-encrypt it:

ansible-vault edit secrets.yml

This will prompt for the vault password, decrypt the file temporarily in memory, open it in your default editor, and re-encrypt it when you save and exit.

View encrypted content

If you just need to view the contents without editing:

[command]
ansible-vault view secrets.yml

Decrypt-modify-encrypt workflow

Alternatively, you can fully decrypt the file, make your changes, and then re-encrypt it:

ansible-vault decrypt secrets.yml

vi secrets.yml # Edit the file with your favorite editor

ansible-vault encrypt secrets.yml # Re-encrypt when finished

This approach is useful when you need to process the file with other tools or make extensive changes.

Password management options

Ansible Vault offers several options for providing the encryption password:

Command-line prompts

The most basic approach is to have Ansible prompt for the password on each run:

ansible-playbook deploy_application.yml --ask-vault-pass

Password files

For automated runs, you can store the password in a file and reference it:

ansible-playbook deploy_application.yml --vault-password-file ~/.vault_pass

The vault password file should contain just the password on a single line:

my_secure_vault_password

Make sure to restrict access to this file:

[command]
chmod 600 ~/.vault_pass

Script-based password retrieval

For more sophisticated setups, you can use a script that retrieves the password from a secure location:

#!/usr/bin/env python3

import os
import subprocess
import sys

# Example: retrieve password from environment variable
password = os.environ.get('ANSIBLE_VAULT_PASSWORD')

if password:
    print(password)
    sys.exit(0)
else:
    # Or from a secure password manager
    result = subprocess.run(['pass', 'show', 'ansible/vault'],
                            capture_output=True, text=True)
    if result.returncode == 0:
        print(result.stdout.strip())
        sys.exit(0)

    sys.exit(1)

Make the script executable and use it:

chmod +x get_vault_pass.py

ansible-playbook deploy_application.yml --vault-password-file ./get_vault_pass.py

Multiple vault IDs

For more complex environments, Ansible supports multiple vault IDs, allowing different passwords for different contexts:

ansible-vault encrypt --vault-id production@prompt secrets_prod.yml

ansible-vault encrypt --vault-id development@~/dev_vault_pass secrets_dev.yml

When running playbooks, you can specify which vault IDs to use:

ansible-playbook site.yml --vault-id production@prompt --vault-id development@~/dev_vault_pass

This is particularly useful for managing different environments or separating secrets with different sensitivity levels.

Encrypting specific variables

Instead of encrypting an entire file, you can encrypt just specific variables:

ansible-vault encrypt_string 'sensitive_data_here' --name 'encrypted_variable'

This produces output you can paste directly into a YAML file:

regular_variable: visible_value
encrypted_variable: !vault |
          $ANSIBLE_VAULT;1.1;AES256
          31613338383765633932316333623839336165386439313864643663323435303566643537353833
          3539633735383661323432366462613364643263663439350a303837636236316662646566363438
          34623162376164643030363635366664323630323864643132323333623337313664623132323461
          6530313164383264340a623934393038393930616130633339333937643139323039656132653562
          3039

Integration with external secret management systems

For organizations with existing secret management infrastructure, Ansible can integrate with these systems rather than implementing its own storage. This approach leverages specialized tools designed specifically for secret management while maintaining the automation capabilities of Ansible.

HashiCorp Vault integration

HashiCorp Vault is a popular secret management tool that can be used with Ansible through the hashi_vault lookup plugin.

First, ensure you have the required Python package:

pip install hvac

Then, in your playbook, you can retrieve secrets directly from Vault:

- name: Deploy application using HashiCorp Vault secrets
  hosts: web_servers
  gather_facts: false
  vars:
    vault_url: "https://vault.example.com:8200"
    vault_token: "{{ lookup('env', 'VAULT_TOKEN') }}"

  tasks:
    - name: Retrieve database credentials from Vault
      set_fact:
        db_credentials: "{{ lookup('hashi_vault', 'secret=database/creds/webapp token=' + vault_token + ' url=' + vault_url) }}"

    - name: Create database configuration
      template:
        src: db_config.j2
        dest: /etc/webapp/database.conf
        owner: webapp
        group: webapp
        mode: '0640'
      vars:
        db_user: "{{ db_credentials.data.username }}"
        db_password: "{{ db_credentials.data.password }}"
      no_log: true

The template for the database configuration might look like this:

# Database configuration
# Generated by Ansible

DATABASE_URL=postgresql://{{ db_user }}:{{ db_password }}@db.example.com:5432/webapp

AWS Secrets Manager integration

For AWS environments, you can integrate with AWS Secrets Manager:

- name: Configure application with AWS secrets
  hosts: application_servers
  gather_facts: false

  tasks:
    - name: Retrieve API key from AWS Secrets Manager
      set_fact:
        api_secrets: "{{ lookup('aws_secret', 'api/keys', region='us-east-1') }}"
      no_log: true

    - name: Configure application with API key
      lineinfile:
        path: /opt/application/config.ini
        regexp: '^API_KEY='
        line: "API_KEY={{ api_secrets.api_key }}"
      no_log: true

CyberArk integration

For organizations using CyberArk, Ansible provides a dedicated lookup plugin:

- name: Deploy with CyberArk credentials
  hosts: all
  gather_facts: false

  tasks:
    - name: Retrieve service account password
      set_fact:
        service_password: "{{ lookup('cyberarkpassword', appid='ansible', query='Safe=MySecretSafe;Object=ServiceAccount') }}"
      no_log: true

    - name: Configure service with retrieved password
      template:
        src: service_config.j2
        dest: /etc/service/config.yml
        owner: service_user
        group: service_group
        mode: '0600'
      vars:
        password: "{{ service_password }}"
      no_log: true

Custom integrations

If you're using a secret management system that doesn't have a dedicated Ansible plugin, you can create a custom integration using the shell or command module in combination with a script:

- name: Configure with custom secret retrieval
  hosts: application_servers
  gather_facts: false

  tasks:
    - name: Retrieve secret from custom system
      command: /usr/local/bin/get_secret.sh --name "database_password"
      register: secret_result
      changed_when: false
      no_log: true

    - name: Use retrieved secret
      lineinfile:
        path: /etc/app/config.ini
        regexp: '^DB_PASSWORD='
        line: "DB_PASSWORD={{ secret_result.stdout | trim }}"
      no_log: true

Security best practices

Proper secret management extends beyond just storage. Here are critical best practices to ensure your secrets remain secure throughout your Ansible workflow.

Protecting secrets in logs

By default, Ansible logs task parameters, which can inadvertently expose secrets. The no_log parameter is crucial for preventing this:

---
- name: Deploy application
  hosts: web_servers
  gather_facts: false
  vars_files:
    - secrets.yml

  tasks:
    - name: Configure application with API key
      lineinfile:
        path: /etc/app/config.ini
        regexp: '^API_KEY='
        line: "API_KEY={{ api_key }}"
      no_log: true  # This prevents the API key from appearing in logs

Without no_log: true, a failed task might expose the secret in error output:

TASK [Configure application with API key] ***************************
fatal: [webserver]: FAILED! => {
    "changed": false,
    "msg": "Could not find file: /etc/app/config.ini",
    "rc": 1,
    "cmd": "lineinfile path=/etc/app/config.ini line='API_KEY=s3cr3t4p1k3y'"
}

With no_log: true, the output is sanitized:

TASK [Configure application with API key] ***************************
fatal: [webserver]: FAILED! => {
    "censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result"
}

Understanding debug levels

Be aware that increasing Ansible's verbosity with -v, -vv, or -vvv can reveal more information, potentially including secrets. Always review debug output carefully before sharing logs.

Securing credential files

If using password files for Vault encryption:

chmod 600 ~/.vault_pass # Restrict file permissions

Regular password rotation

Implement a process for regularly rotating Vault passwords and updating encrypted files:

ansible-vault rekey secrets.yml --new-vault-password-file ~/.new_vault_pass

File permissions for playbooks

Since playbooks might contain paths to sensitive files or hints about security architecture:

chmod 750 ~/ansible-projects

Limit secret access scope

Structure your Ansible projects to limit the scope of secrets:

# Encrypted vault file with database-specific secrets
db_root_password: encrypted_password_here
db_backup_key: encrypted_key_here

# Encrypted vault file with web-specific secrets
ssl_private_key: encrypted_key_here
api_tokens: encrypted_tokens_here

This prevents unnecessary exposure of secrets to hosts that don't need them.

Final thoughts

As your Ansible usage evolves, you may find yourself combining these approaches—perhaps using Vault for some secrets while integrating with external systems for others. The key is to develop a consistent, documented approach that works for your environment while maintaining the appropriate security controls.

Thanks for reading!

Got an article suggestion? Let us know

Next article

Understanding VPN Virtual Locations and Their Impact on Privacy

A comprehensive analysis reveals that most VPN providers route your traffic through countries you never selected. Explore the science behind server geolocation, understand virtual locations, and learn what this means for your privacy and security.

→

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.