# Managing Secrets in Ansible Playbooks: A Comprehensive Guide

When you first begin your Ansible journey, you might be tempted to hardcode
secrets directly into your playbooks for simplicity. However, as your automation
practices mature, properly securing sensitive information becomes critical.

Fortunately, Ansible offers several built-in mechanisms for handling secrets, as
well as integration options with external secret management systems.

This guide will walk you through progressively sophisticated approaches to
secret management in Ansible:

1. Using interactive prompts for manual playbook execution
2. Employing Ansible Vault for encrypted storage
3. Integrating with dedicated secret management platforms

We'll also cover important security best practices to ensure your secrets remain
protected throughout the automation lifecycle.

[ad-logs]

## Using interactive prompts

If you're just beginning with Ansible or for playbooks that are always run
manually, interactive prompts provide a straightforward solution.

This approach causes Ansible to request sensitive information from the user at
runtime, avoiding the need to store secrets within playbook files.

### Implementing vars_prompt

To implement an interactive prompt, you'll use the `vars_prompt` directive in
your playbook. Here's a basic example that prompts for database credentials:

```yaml
[label database_config.yml]
- name: Configure database connection
  hosts: database_servers
  gather_facts: false
  vars_prompt:
    - name: db_user
      prompt: Enter database username
      private: false

    - name: db_password
      prompt: Enter database password
      private: true
      confirm: true

  tasks:
    - name: Update database configuration file
      template:
        src: database.conf.j2
        dest: /etc/application/database.conf
        owner: app_user
        group: app_group
        mode: '0640'
```

The `private: true` parameter ensures that the entered text isn't displayed on
screen while typing, which is important for passwords. The `confirm: true`
parameter will ask the user to enter the password twice to avoid mistakes.

When you run this playbook, Ansible will prompt you for the required
information:

```command
ansible-playbook database_config.yml
```

```text
[output]
Enter database username: admin
Enter database password:
Confirm Enter database password:

PLAY [Configure database connection] ***************************

TASK [Update database configuration file] **********************
changed: [db1.example.com]

PLAY RECAP ***************************************************
db1.example.com           : ok=1    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
```

[The template file](https://betterstack.com/community/guides/linux/ansible-templates/) referenced in the task might look
something like this:

```text
[label database.conf.j2]
# Database connection configuration
# Generated by Ansible on {{ ansible_date_time.date }}

DB_HOST={{ db_host | default('localhost') }}
DB_PORT={{ db_port | default(5432) }}
DB_USER={{ db_user }}
DB_PASSWORD={{ db_password }}
```

### Limitations of interactive prompts

While interactive prompts are easy to implement, they have significant
limitations:

1. They require manual input, making them unsuitable for automated pipelines or
   scheduled runs
2. They don't provide a way to securely store the secrets for future use
3. They become cumbersome when managing multiple secrets across numerous
   playbooks

For these reasons, interactive prompts are best suited for:

- Initial setup procedures
- Rarely run maintenance tasks
- Development and testing environments
- Playbooks that are always executed manually

As your Ansible usage matures, you'll likely need a more robust solution for
secret management.

## Using Ansible Vault for native encryption

[Ansible Vault](https://betterstack.com/community/guides/linux/ansible-vault/) provides built-in encryption capabilities that
allow you to secure sensitive data while still keeping it alongside your
playbooks. This approach strikes a good balance between security and
convenience, making it a popular choice for many Ansible users.

Let's start with creating an encrypted file that contains database credentials:

```yaml
db_user: admin
db_password: complex_password_123
api_key: a72bd98c66e4f5432bb90c6ad1b
ssl_private_key: |
  -----BEGIN PRIVATE KEY-----
  MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQC7VJTUt9Us8cKj
  MzEfYyjiWA4R4/M2bS1GB4t7NXp98C3SC6dVMvDuictGeurT8jNbvJZHtCSuYEvu
  Nmg2fFnTYUjQAlt4aUNl5jDWtLZVfMcZLu0XzXGo9T7HVWKgH95g3Gxz6EZtXmY0
  ...
  -----END PRIVATE KEY-----
```

To encrypt this file, use the `ansible-vault encrypt` command:

```command
ansible-vault encrypt secrets.yml
```

You'll be prompted to create and confirm a password. Once completed, the file
contents will be transformed into encrypted data:

```text
[output]
$ANSIBLE_VAULT;1.1;AES256
7839616262613373935663839666463306231653861336630613938303662633538633836656465
3637353766613339663032363538626430316135623665340a653961303730353962386134393162
62343936366265353935346336643865643833353737613962643539373230616239346133653464
6435353361373263640a376632613336366430663761363339333737386637383961363833303830
34336535623736313031313162353831666139343662653665366134633832646661...
```

### Using encrypted files in playbooks

With the secrets now encrypted, you can reference them in your playbook using
the `vars_files` directive:

```yaml
- name: Deploy application with secure credentials
  hosts: app_servers
  gather_facts: true
  vars_files:
    - secrets.yml

  tasks:
    - name: Create application configuration file
      template:
        src: app_config.j2
        dest: /opt/myapp/config.json
        owner: app_user
        group: app_group
        mode: '0600'
```

When running this playbook, you'll need to provide the vault password:

```command
ansible-playbook deploy_application.yml --ask-vault-pass
```

The system will prompt you for the password, decrypt the secrets, and make them
available as variables during playbook execution.

### Working with encrypted files

There are several ways to work with encrypted files:

#### In-place editing

To edit an encrypted file without having to manually decrypt and re-encrypt it:

```command
ansible-vault edit secrets.yml
```

This will prompt for the vault password, decrypt the file temporarily in memory,
open it in your default editor, and re-encrypt it when you save and exit.

#### View encrypted content

If you just need to view the contents without editing:

```text
[command]
ansible-vault view secrets.yml
```

#### Decrypt-modify-encrypt workflow

Alternatively, you can fully decrypt the file, make your changes, and then
re-encrypt it:

```command
ansible-vault decrypt secrets.yml
```

```command
vi secrets.yml # Edit the file with your favorite editor
```

```command
ansible-vault encrypt secrets.yml # Re-encrypt when finished
```

This approach is useful when you need to process the file with other tools or
make extensive changes.

### Password management options

Ansible Vault offers several options for providing the encryption password:

#### Command-line prompts

The most basic approach is to have Ansible prompt for the password on each run:

```command
ansible-playbook deploy_application.yml --ask-vault-pass
```

#### Password files

For automated runs, you can store the password in a file and reference it:

```command
ansible-playbook deploy_application.yml --vault-password-file ~/.vault_pass
```

The vault password file should contain just the password on a single line:

```text
[label .vault_pass]
my_secure_vault_password
```

Make sure to restrict access to this file:

```text
[command]
chmod 600 ~/.vault_pass
```

#### Script-based password retrieval

For more sophisticated setups, you can use a script that retrieves the password
from a secure location:

```python
#!/usr/bin/env python3

import os
import subprocess
import sys

# Example: retrieve password from environment variable
password = os.environ.get('ANSIBLE_VAULT_PASSWORD')

if password:
    print(password)
    sys.exit(0)
else:
    # Or from a secure password manager
    result = subprocess.run(['pass', 'show', 'ansible/vault'],
                            capture_output=True, text=True)
    if result.returncode == 0:
        print(result.stdout.strip())
        sys.exit(0)

    sys.exit(1)
```

Make the script executable and use it:

```command
chmod +x get_vault_pass.py
```

```command
ansible-playbook deploy_application.yml --vault-password-file ./get_vault_pass.py
```

### Multiple vault IDs

For more complex environments, Ansible supports multiple vault IDs, allowing
different passwords for different contexts:

```command
ansible-vault encrypt --vault-id production@prompt secrets_prod.yml
```

```command
ansible-vault encrypt --vault-id development@~/dev_vault_pass secrets_dev.yml
```

When running playbooks, you can specify which vault IDs to use:

```command
ansible-playbook site.yml --vault-id production@prompt --vault-id development@~/dev_vault_pass
```

This is particularly useful for managing different environments or separating
secrets with different sensitivity levels.

### Encrypting specific variables

Instead of encrypting an entire file, you can encrypt just specific variables:

```command
ansible-vault encrypt_string 'sensitive_data_here' --name 'encrypted_variable'
```

This produces output you can paste directly into a YAML file:

```yaml
regular_variable: visible_value
encrypted_variable: !vault |
          $ANSIBLE_VAULT;1.1;AES256
          31613338383765633932316333623839336165386439313864643663323435303566643537353833
          3539633735383661323432366462613364643263663439350a303837636236316662646566363438
          34623162376164643030363635366664323630323864643132323333623337313664623132323461
          6530313164383264340a623934393038393930616130633339333937643139323039656132653562
          3039
```

## Integration with external secret management systems

For organizations with existing secret management infrastructure, Ansible can
integrate with these systems rather than implementing its own storage. This
approach leverages specialized tools designed specifically for secret management
while maintaining the automation capabilities of Ansible.

### HashiCorp Vault integration

[HashiCorp Vault](https://www.vaultproject.io/) is a popular secret management
tool that can be used with Ansible through the `hashi_vault` lookup plugin.

First, ensure you have the required Python package:

```command
pip install hvac
```

Then, in your playbook, you can retrieve secrets directly from Vault:

```yaml
- name: Deploy application using HashiCorp Vault secrets
  hosts: web_servers
  gather_facts: false
  vars:
    vault_url: "https://vault.example.com:8200"
    vault_token: "{{ lookup('env', 'VAULT_TOKEN') }}"

  tasks:
    - name: Retrieve database credentials from Vault
      set_fact:
        db_credentials: "{{ lookup('hashi_vault', 'secret=database/creds/webapp token=' + vault_token + ' url=' + vault_url) }}"

    - name: Create database configuration
      template:
        src: db_config.j2
        dest: /etc/webapp/database.conf
        owner: webapp
        group: webapp
        mode: '0640'
      vars:
        db_user: "{{ db_credentials.data.username }}"
        db_password: "{{ db_credentials.data.password }}"
      no_log: true
```

The template for the database configuration might look like this:

```text
[label db_config.j2]
# Database configuration
# Generated by Ansible

DATABASE_URL=postgresql://{{ db_user }}:{{ db_password }}@db.example.com:5432/webapp
```

### AWS Secrets Manager integration

For AWS environments, you can integrate with
[AWS Secrets Manager](https://aws.amazon.com/secrets-manager/):

```yaml
- name: Configure application with AWS secrets
  hosts: application_servers
  gather_facts: false

  tasks:
    - name: Retrieve API key from AWS Secrets Manager
      set_fact:
        api_secrets: "{{ lookup('aws_secret', 'api/keys', region='us-east-1') }}"
      no_log: true

    - name: Configure application with API key
      lineinfile:
        path: /opt/application/config.ini
        regexp: '^API_KEY='
        line: "API_KEY={{ api_secrets.api_key }}"
      no_log: true
```

### CyberArk integration

For organizations using [CyberArk](https://www.cyberark.com/), Ansible provides
a dedicated lookup plugin:

```yaml
- name: Deploy with CyberArk credentials
  hosts: all
  gather_facts: false

  tasks:
    - name: Retrieve service account password
      set_fact:
        service_password: "{{ lookup('cyberarkpassword', appid='ansible', query='Safe=MySecretSafe;Object=ServiceAccount') }}"
      no_log: true

    - name: Configure service with retrieved password
      template:
        src: service_config.j2
        dest: /etc/service/config.yml
        owner: service_user
        group: service_group
        mode: '0600'
      vars:
        password: "{{ service_password }}"
      no_log: true
```

### Custom integrations

If you're using a secret management system that doesn't have a dedicated Ansible
plugin, you can create a custom integration using the `shell` or `command`
module in combination with a script:

```yaml
- name: Configure with custom secret retrieval
  hosts: application_servers
  gather_facts: false

  tasks:
    - name: Retrieve secret from custom system
      command: /usr/local/bin/get_secret.sh --name "database_password"
      register: secret_result
      changed_when: false
      no_log: true

    - name: Use retrieved secret
      lineinfile:
        path: /etc/app/config.ini
        regexp: '^DB_PASSWORD='
        line: "DB_PASSWORD={{ secret_result.stdout | trim }}"
      no_log: true
```

## Security best practices

Proper secret management extends beyond just storage. Here are critical best
practices to ensure your secrets remain secure throughout your Ansible workflow.

### Protecting secrets in logs

By default, Ansible logs task parameters, which can inadvertently expose
secrets. The `no_log` parameter is crucial for preventing this:

```yaml
[label secure_logging.yml]
---
- name: Deploy application
  hosts: web_servers
  gather_facts: false
  vars_files:
    - secrets.yml

  tasks:
    - name: Configure application with API key
      lineinfile:
        path: /etc/app/config.ini
        regexp: '^API_KEY='
        line: "API_KEY={{ api_key }}"
      no_log: true  # This prevents the API key from appearing in logs
```

Without `no_log: true`, a failed task might expose the secret in error output:

```text
[output]
TASK [Configure application with API key] ***************************
fatal: [webserver]: FAILED! => {
    "changed": false,
    "msg": "Could not find file: /etc/app/config.ini",
    "rc": 1,
    "cmd": "lineinfile path=/etc/app/config.ini line='API_KEY=s3cr3t4p1k3y'"
}
```

With `no_log: true`, the output is sanitized:

```text
[output]
TASK [Configure application with API key] ***************************
fatal: [webserver]: FAILED! => {
    "censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result"
}
```

### Understanding debug levels

Be aware that increasing Ansible's verbosity with `-v`, `-vv`, or `-vvv` can
reveal more information, potentially including secrets. Always review debug
output carefully before sharing logs.

### Securing credential files

If using password files for Vault encryption:

```command
chmod 600 ~/.vault_pass # Restrict file permissions
```

### Regular password rotation

Implement a process for regularly rotating Vault passwords and updating
encrypted files:

```command
ansible-vault rekey secrets.yml --new-vault-password-file ~/.new_vault_pass
```

### File permissions for playbooks

Since playbooks might contain paths to sensitive files or hints about security
architecture:

```command
chmod 750 ~/ansible-projects
```

### Limit secret access scope

Structure your Ansible projects to limit the scope of secrets:

```yaml
# Encrypted vault file with database-specific secrets
db_root_password: encrypted_password_here
db_backup_key: encrypted_key_here
```

```yaml
# Encrypted vault file with web-specific secrets
ssl_private_key: encrypted_key_here
api_tokens: encrypted_tokens_here
```

This prevents unnecessary exposure of secrets to hosts that don't need them.

## Final thoughts

As your Ansible usage evolves, you may find yourself combining these
approaches—perhaps using Vault for some secrets while integrating with external
systems for others. The key is to develop a consistent, documented approach that
works for your environment while maintaining the appropriate security controls.

Thanks for reading!