Behavior Analysis and Solutions for Using set_facts with with_items in Ansible

Background and Phenomenon

In Ansible automation tool, the set_facts module is used to set host facts, while the with_items loop allows iteration over list items. Users often attempt to combine these to dynamically construct lists, such as starting from an initial list and incrementally adding new elements within a loop. However, in practice, especially in early versions of Ansible 1.x and 2.x, this behavior may not work as expected.

Consider a typical scenario: a user defines an initial list foo containing the element 'zero'. Subsequently, multiple set_facts tasks append elements, with the first three tasks using static values (e.g., 'one', 'two', 'three') successfully extending the list. But when the fourth task uses a with_items loop to iterate over dynamic items (e.g., 'four', 'five', 'six'), the output shows the list contains only the value from the last iteration ('six'), rather than accumulating all iterated items. This results in a final list of ['zero', 'one', 'two', 'three', 'six'], missing 'four' and 'five'.

This behavior was observed in both Ansible 1.7.2 and 1.8, raising user questions: Is this a bug in Ansible? In fact, based on community feedback and official documentation, this reflects specific mechanisms of Ansible's variable scoping and loop handling.

Root Cause Analysis

The core issue lies in Ansible's variable scoping and how set_facts executes within loops. In Ansible, facts are host-level variables with global scope, but they can be influenced by loop context during task execution. When set_facts is combined with with_items, the expression is re-evaluated in each iteration, but variable references may not accumulate correctly across iterations.

Specifically, in a loop, an expression like set_fact: foo="{{ foo }} + [ '{{ item }}' ]" attempts to add the current item to the existing foo list in each iteration. However, due to Ansible's variable resolution mechanism, foo might be reset or overwritten in each iteration, leading to only the last iteration's result being retained. This is not a bug but a design behavior stemming from how loops and variable interactions were handled in early Ansible versions.

The community has extensively discussed this issue, and a pull request (e.g., PR #8019 on GitHub) aims to improve this functionality to support list accumulation in loops. This PR proposes modifying set_facts behavior to correctly append elements within with_items, but as of this writing, it may not be fully integrated into all Ansible versions. Therefore, users need to rely on workarounds to achieve the desired outcome.

Solutions and Best Practices

Based on community answers and practical testing, here are several effective solutions applicable to different Ansible versions and scenarios.

First, use the register keyword to capture loop results, then extract the needed data via filters. This method works in both Ansible 1.x and 2.x, offering more controlled variable management. For example:

- name: set fact with items
  set_fact: foo_item="{{ item }}"
  with_items:
    - four
    - five
    - six
  register: foo_result

- name: make a list from registered results
  set_fact: foo="{{ foo_result.results | map(attribute='ansible_facts.foo_item') | list }}"

In this example, register: foo_result captures the set_facts result from each iteration, storing it in the foo_result.results list. Then, the map filter extracts the ansible_facts.foo_item value from each result, and the list filter converts it into a list. Ultimately, foo will contain all iterated items ['four', 'five', 'six']. This approach, while more verbose, ensures data integrity and traceability.

Second, adjust the reference syntax in the set_facts expression to avoid nested Jinja2 interpolation. In Ansible 2.2 and later, the following syntax may be more reliable:

- set_fact: something="{{ something | default([]) + [ item ] }}"
  with_items:
    - one
    - two
    - three

Here, item is referenced directly as a list element, without extra quotes. Additionally, the default([]) filter handles the initial case where the something variable is undefined, ensuring the operation does not fail. This method simplifies the code and leverages Ansible's improved variable handling. If conditional checks are needed within the loop, combine with a when statement, e.g., when: item.name in allowed_things.item_list, to filter specific items.

Furthermore, for more complex scenarios, consider using other Ansible modules or custom filters. For instance, the loop keyword, as a modern replacement for with_items in Ansible 2.5 and above, may offer more consistent loop behavior. However, note that set_facts combined with loop might have similar limitations, so testing is key.

Conclusion and Recommendations

In Ansible, the variable accumulation issue when using set_facts with with_items stems from Ansible's variable scoping and loop execution mechanisms. While community improvements are planned, users can currently achieve dynamic list construction through workarounds.

Key recommendations include: prioritize the register and filter method for compatibility; experiment with simplified syntax in newer versions; always test Playbooks to verify behavior. Understanding these underlying mechanisms aids in writing more robust and maintainable Ansible code, enhancing automation efficiency. As Ansible continues to evolve, future versions are expected to more seamlessly support such common use cases.