cloudsoft.io

Tips and Tricks for Visual Composer

Setting Tags

Many resources within AWS have a ‘Tags’ property, such as AWS::EC2::Instance. The long-form in CloudFormation is:

Properties:
  Tags:
    - Key: key1
      Value: val1
    - Key: key2
      Value: val2

Visual Composer also accepts a short-hand for this: each tag can be entered using either the format key1=val1 or key1: val1.

Configuring tags

If you have a key or value containing these special characters, put the key and value in quotes such as "key:1": "val:1"

Each tag can also be entered as JSON (see below), e.g. {"Key": "key1", "Value": "val1"}.

JSON Values

Some properties in CloudFormation have complex types. Additional support is being added in future versions of Visual Composer for setting such values. However, it is always possible to provide a JSON value.

For example, the AWS::EC2::Instance Volumes property is a list of Device-VolumeId tuples, such as:

Properties:
  Volumes:
    - Device: "/dev/sdf"
      VolumeId: vol-xxxxxxxx
    - Device: "/dev/sdg"
      VolumeId: vol-yyyyyyyy

This can be achieved by using the first entry {"Device": "/dev/sdf", "VolumeId": "vol-xxxxxxxx"}, and similarly for the second entry.

Configuring json values

Where a textbox is annotated with the “</>” icon, it indicates that JSON is recognised by the graphical editor at composition time - click on this icon to have the value recognised as JSON rather than a string.

However, where a string (in JSON format) is used and a complex type is expected, it will attempt to automatically convert it to the complex type when ‘Export CFN’ is clicked.

If the JSON is invalid (e.g. an incorrect field), the ‘Export CFN’ will return an error. Messages are more specific when using JSON rather than a string. For example, “Invalid blueprint: Type io.cloudsoft.aws.cfn.raw.ec2.instance.Volume has no apparent setter for Dveyez”, versus “Invalid blueprint: Cannot coerce type class java.lang.String to io.cloudsoft.aws.cfn.raw.ec2.instance.Volume”.

References

CloudFormation supports references to other resources in the template. This includes use of Ref and Fn::GetAtt.

For example, a template could create a AWS::Route53::RecordSet and a AWS::ElasticLoadBalancingV2::LoadBalancer, and could configure the RecordSet with an AliasTarget to point at the ELB.

Such a trimmed-down CloudFormation template is shown below.

Resources:
  AWSRoute53RecordSet:
    Type: "AWS::Route53::RecordSet"
    Properties:
      AliasTarget:
        HostedZoneId:
          Fn::GetAtt:
          - "Elb"
          - "CanonicalHostedZoneID"
        DNSName:
          Fn::GetAtt:
          - "Elb"
          - "DNSName"
      ..
  Elb:
    Type: "AWS::ElasticLoadBalancingV2::LoadBalancer"
    Properties:
      ...

To write this in Visual Composer, you can set the ‘Target’ value to point at the ELB, which will auto-generate this cross-reference configuration.

First add the desired resources from the palette. Then select the property whose value is to be set (e.g. ‘Target’ in this case). Click the lightning-bolt icon to choose the target reference.

Configuring lightning-bolt

This opens a wizard for choosing the target value. Use the search box to filter the targets, and click on the desired one (in this case the Load Balancer). The target must have a name - if the ‘Entity ID’ text box at the bottom of the wizard is blank, then give it a name such as ‘elb’ (otherwise a random ID will be generated). Then Click ‘Done’.

Configuring using the DSL editor

Custom CloudFormation

This Composer does not yet natively support all the fields available in CloudFormation. However you can add arbitrary CFN YAML easily in the Composer’s YAML blueprint, to support things like Mappings and Outputs. This is done by adding a extraCloudFormationYaml key in brooklyn.config at root (or on a given entity to set e.g. Metadata or a CreationPolicy there) and adding the YAML map data to be merged. For example:

services:
- type: AWS::Sample
  id: sample
  brooklyn.config:
    extraCloudFormationYaml:
      Metadata: YOUR_METADATA_HERE
brooklyn.config:
  extraCloudFormationYaml:
    Mappings: YOUR_MAPPINGS_HERE
    Outputs: YOUR_OUTPUT_HERE

will generate:

AWSTemplateFormatVersion: "2010-09-09"
Description: "Application CloudFormation exported from Cloudsoft AMP"
Parameters: {}
Mappings: YOUR_MAPPINGS_HERE
Resources:
  sample:
    Type: AWS::Sample
    Metadata: YOUR_METADATA_HERE
Outputs: YOUR_OUTPUT_HERE

YAML Model

When composing a CloudFormation template, the visual composer constructs a model in YAML. There is a bi-directional mapping to this YAML, which you can seen by clicking the two-way arrows in the top-right of the composer (click it again to switch back to the graphical view).

This YAML model (referred to as a YAML blueprint) is a simpler, portable description of applications. It uses the Cloudsoft AMP format. The CloudFormation is auto-generated from this model (i.e. the model can be thought of as an intermediate representation).

AMI Auto-Discovery

The property AWS::EC2::Instance ImageId is the AMI id within a given region, of the form ami-xxxxxxxxxxxxxxxxx.

Hard-coding these IDs within a CloudFormation template has advantages: it is the most straight-forward configuration. However, there are disadvantages: it is hard to read (requiring looking up the AMI to check what it is); it only works in one region; it will not pick up the newest AMI (e.g. of a hardened certified private image within your organisation).

There are many approaches to handle this within CloudFormation, such as using Mappings to specify different AMIs per region, or passing the AMI id as a parameter to the template, or referencing the named output from another CloudFormation stack.

Another way to handle this within Visual Composer is to use an ‘Image Name Regex’. When ‘Export CFN’ is run, it will lookup the matching AMI(s) to get the ID, and will include the ID in the generated CloudFormation. This approach may be appropriate for some use-cases, such as if using the Visual Composer REST API as part of a build pipeline to generate separate templates for each region.

Parameter Store

The Parameter Store, part of the AWS Systems Manager, is a key-value store which can hold strings, string lists or secure strings. These values can then be referenced in templates or in Visual Composer. This provides several advantages:

  • Values that are used in many places can be kept and updated in one place
  • If the value changes, all references to it can be updated
  • Sensitive information does not need to be placed directly into the template

In short, this makes your templates much easier to work with, even cross-region or cross-account. There is an AWS document covering the creation and usage of parameters and an AWS blog describing the use of secure strings to store sensitive information.

There are two popular ways to reference a parameter store value from a template.

“Resolve” Keyword for Parameter Store Values

The simpler and more powerful mechanism is simply to writing:

within your template, where <key> is the relevant key in the parameter store. This same expression can be used in the Visual Composer. Note that whenever this is written in YAML it may need to be wrapped quotes.

The SSM Parameter Store supports versions, starting at 1 and increading by 1 on each edit; a specific version can be forced by appending :<version> after <key>.

SSM “Secure Strings” can be accessed by replacing ssm in the expression with ssm-secure:
this is recommended for credentials and other highly sensitive information.

This syntax can be used anywhere in Visual Composer where a string value is expected. (To use it in other cases, the DSL syntax $brooklyn:literal("" can be used. Note that if editing YAML directly the expression should be wrapped in either single quotes or double quotes.)

As an example, set a key GoldenImage with the value for your preferred image ID in the parameter store for the account and region you wish to target. Then in Visual Composer, drag an EC2 Instance to the canvas and enter `` as the Image ID (and something, such as t3.micro, as the instance type). When the resulting template is deployed, it will use the most recent value for the GoldenImage key.

Define a “Parameter” for the Parameter Store Value

An alternative way to reference a parameter value is to define an input parameter for your template with type:

AWS::SSM::Parameter::Value<String>

The value of this CFN parameter should be the SSM Parameter Store key name. CFN will then replace references to this parameter (using standard !Ref syntax) with the value in the store.

This has some advantages:

  • The value is resolved once and used consistently throughout
  • You are not limited to 60 references (as you are with the resolve:ssm:... dynamic reference)

However it also has significant disadvantages:

  • You cannot specify a specific version of a value
  • You cannot access secure strings (i.e. credentials)

This is not currently supported in the Visual Composer.