Anas Anjaria
Anas Anjaria's blog

Anas Anjaria's blog

Monitor Ansible Playbook Executions

Photo by Markus Winkler on Unsplash

Monitor Ansible Playbook Executions

Anas Anjaria's photo
Anas Anjaria
·Sep 4, 2022·

2 min read

Subscribe to my newsletter and never miss my upcoming articles

I planned to use an ansible, a configuration management tool, in a production system. When it comes to the production system, it is important to know which configurations are applied successfully and which aren’t.

After spending some time on the internet, I couldn’t find an easy way to monitor failed execution. Hence I come up with this solution.

Basic workflow

  1. Dump playbook output on a host-specific file using log_plays.
  2. Ship these logs to elasticsearch using filebeat.
  3. Finally, visualize information using kibana.
Basic workflow for monitoring ansible playbook executions.
The basic workflow for monitoring playbook executions

Proof of concept

Source code: monitor-ansible-playbook-executions

Ansible offers a callback that writes playbook output to a file per host by using the following configuration.

-- ansible.cfg

[defaults]     
stdout_callback = log_plays

This will dump playbook execution logs in the following format.

time - path-to-playbook - task-name - task-command - task-status - task-output

time - path-to-playbook - task-name - task-command - task-status - task-output

These logs are parsed using filebeat via the dissect processor. Since there is an empty line between the two messages hence parsing is done conditionally.

processors:    
- dissect:    
    when:    
      not:    
        equals:    
          message: ""    
    tokenizer: "%{time} - %{playbook_path} - %{task_name} - %{task_cmd} - %{task_status} - %{task_output}"    
    field: "message"    
    target_prefix: "ansible"

In order to enrich logs with host information, the script processor is used.

- script:    
    lang: javascript    
    source: >    
      function process(event) {    
          var ansible_path = event.Get("log.file.path");    
          var host = ansible_path.split("/").pop();    
          event.Put('ansible.host', host)    
      }

Finally, logs are published to an elasticsearch.

output.elasticsearch:    
  enabled: true    
  hosts: ["elasticsearch-1:9200"]

One could now visualize this information via kibana and add watchers to inform the team when failures occur.

Ansible execution status — success vs failure
Ansible execution status — success vs failure

Thanks for reading.


Want to connect?

anasanjaria.bio.link

 
Share this