cloudsoft.io

Groups

The group resource dynamically collects as “members” all entities that match a given condition. This allows alternate views and conditionally applying behavior to resources.

As a simple example, consider:

resource maeztro web1 {
  config { 
    is_web = true
  }
}
resource maeztro web2 {
  config { 
    is_web = true
  }
}
resource maeztro db {
}

resource group web_items {
  members {
    condition = try(item.is_web==true, false)
  }
}

In this example, all resources with is_web set to true will be collected under the group.web_items node, as a member of that group, in addition to their usual location. This can be used to collect any resource on any condition, such as Terraform resources based on item.config["mz.resource_type"].

Groups simplify working with many resources when there are different logical ways to organize them, such as location or type, or problem states, or load. Once defined, a group can be used to:

  • write effectors or sensors that iterate over members, using foreach x in members – for example to apply an effector at each of the is_web resources selected above

  • apply additional definitions for things like effectors and policies to the members

  • pivot the members into intermediate partition sub-groups dynamically

Each of these is included below.

The resource group block must contain a members { ... } block defining the treatment of members, and can in addition take any of the usual resource-definition blocks and attributes applicable to any resource, such as effectors and policies declared at the group. Within the members block, the following can be supplied:

  • a condition attribute or block (required) indicating the condition required for candidate entities to be added (or removed) as a member of the group, and as a member of the applicable partition and sub-partitions if specified; the variable item is typically used in the condition, often with the can or try function to catch errors if the sub-field of the item is not present

  • a search block (optional) allowing customization of which resources should be considered as candidates for members in the group, tested against the condition; by default all resources in the current module are considered
    • an ancestor attribute (optional) indicating the root element to search from; only descendants of this entity will be searched; defaults to the root of the current module, or application if not in a module
    • a modules attribute (optional), either true or false, defaulting to false, indicating whether descendant modules should be included in the search

  • an apply block (optional) containing resource-definition blocks and attributes applicable to any resource, which are applied to members (and unapplied if an entity is no longer a member); the variable group can be used in this block to refer to the group

  • one or more partition blocka (optional) defining the scheme for partitioning members; this block should be given a name including an expression in terms of ${item.<field>} for some <field> on a candidate member item; a partition is created as a child for each different evaluated value of that name, and the corresponding items are placed as members there; this block can be further configured with:
    • an on (optional) block or map defining key-value pairs with the values also typically expressions in terms of the candidate member item; they keys here can be references in the paritition name directly and in any of the other blocks and as self.<field> on the partition
    • a condition (optional) in addition to the parent group’s condition, to further qualify membership of the partition
    • an apply block (optional) to apply additional resource-definition blocks and attributes to members of the partition
    • a resource block (optional), optionally with one of the resource types indicated, but not supplying an identifier (because that comes from the partition name), containing any of the usual resource-definition blocks and attributes which should be applied to each instance of the partition
    • further partition blocks which can further sub-partition the accepted entities (in the same way as this partition block) Template variables – e.g. group and partition, item references in the partition name, values in the on block – are resolved on an “apply” or “reapply” cycle, and should not be elements that change frequently. These variables can exceptionally be used in block names.

Membership is recomputed on any “apply” cycle, and it can be triggered manually or by a policy (periodically and/or on a sensor event) by invoking the rescanEntities effector on the group.

There is no restriction on placing resources into multiple groups. In fact when partitions are used, resources are placed into the base group as well as the first matching partition. This allows the use of foreach x in members can be used at the group to loop over all members. Partitions are added as children resources, so are excluded from the members list.

Partition specifications are applied in the order they are defined in the code, and members are placed in the first partition whose condition matches. Thus there is no need to exclude previous partition conditions, and a catch-all partition "default" { ... } block without a condition can be used to pick up anything which doesn’t match other partitions. If there is no matching partition, the resource is not added to any partition, but will still be added to the parent group. An apply block in a partition is applied to all resources which are added to the partition.

When nested partitions are used, the same process is used for the nested partitions, and resources are added as members to the parent partition as well as the first matching nested partition if there is one. This allows the use of foreach x in members to loop over all resources added to the partition, including nested partitions, or foreach x in children to loop over the child partition resoruces only, excluding members.

A rich example illustrating many of capabilities above is as follows, gathering “pods” (which could be Kubernetes pods, though here they are statically defined), and attaching effectors to them:

resource group "my_colored_pods" {
  name = "Colored Pods Group"
  members {
    # only accept nodes that set is_pod to true and have a color label or initial color
    condition = can(try(item.is_pod)) && can(try(item.labels.color, item.initial_color))
    search {
      ancestor = maeztro.all_pods  # restrict the search to this node and descendants (optional, by default entire module)
    }

    apply "add_change_color_effector" {
      effector "change_color" {
        parameter "new_color" {}
        steps = [ "set-config labels['color'] = ${new_color}" ]
      }
    }

    partition "${color}_pods" {
      on {
        color = try(item.labels.color, item.initial_color)
      }
      resource {
        effector "change_color_from_${color}" {
          parameter "new_color" {}
          steps [
            step "foreach child in members" {
              input = { new_color = new_color }
              steps [ "invoke-effector change_color with new_color=${new_color}" ]
            }
          ]
        }
      }
    }
  }
  effector "reset_all_colors" {
    steps [
      step "foreach item in members do invoke-effector change_color with new_color=${self.initial_color}" {}
    ]
  }
  effector "change_all_colors_and_regroup" {
    parameter "old_color_regex" {}
    parameter "new_color" {}
    steps [
      step "foreach child in children" {
        input = { // make these vars available to each child loop
          old_color_regex = old_color_regex
          new_color = new_color
        }
        condition = can(regex("^${old_color_regex}$", child.color))  // only run for partition if regex matches
        reducing = { // keep a count, and expose it afterwards
          count = 0
        }
        steps [
          "invoke-effector change_color_from_${child.color} with new_color=${new_color}"
          "let count = ${count+1}"
        ]
      }
      "let partitions_affected_count = ${count}"
      "invoke-effector rescanEntities"
      step "return" {
        value = {
          partitions_affected: partitions_affected_count
          partition_count_now: length(self.children)
        }
      }
    ]
  }
}

resource maeztro "all_pods" {}
resource maeztro "blue_pod_1" {
  parent = maeztro.all_pods
  config {
    is_pod = true
    initial_color = "blue"
  }
}
resource maeztro "green_pod_1" {
  parent = maeztro.all_pods
  config {
    is_pod = true
    initial_color = "green"
  }
}
resource maeztro "yellow_pod_1" {
  parent = maeztro.all_pods
  config {
    is_pod = true
    initial_color = "yellow"
  }
}
resource maeztro "yellow_pod_2" {
  parent = maeztro.all_pods
  config {
    is_pod = true
    initial_color = "yellow"
  }
}

AMP and Maeztro are developed and supported by Cloudsoft Corporation.