cloudsoft.io

Command-line scripting with the REST API

The generated CloudFormation can be used in all the ways you’re used to, with no obligation to consider the options presented on this page. However, some of the techniques presented here may make it easier for you to use templates in agile ways and maintain them through your system’s lifecycle.

Visual Composer is more than just a graphical editor. It has a library where you can save and reuse ‘blueprint’; it allows CloudFormation templates to be composed, using blueprints from the library as building blocks; the library supports versioning; and there is a REST API for managing all of this.

This section of the docs focuses on the REST API, enabling you to use this in your automation toolchain and build pipelines.

The Source Blueprint: Cloudsoft AMP YAML

First, let’s look at how Visual Composer works.

When composing a CloudFormation template, it constructs a model in YAML. There is a bi-directional mapping to this YAML, which you can seen by clicking the two-way arrows in the top-right of the composer (click it again to switch back to the graphical view).

This YAML model (referred to as a YAML blueprint) is a simpler, portable description of applications. It uses the Cloudsoft AMP format. The CloudFormation is auto-generated from this model (i.e. the model can be thought of as an intermediate representation).

The REST API works with these YAML blueprints.

Working with YAML Blueprints

It is possible to use these YAML Blueprints as the version-controlled artifacts, and for a build pipeline to then auto-generate the CloudFormation templates. This can open up other use-cases, such as generating different flavours of the CloudFormation template for different regions, accounts, and modes of operation (e.g. dev and prod).

It is also possible to refer to AMIs within the YAML blueprints using naming conventions, and to resolve the AMI id at the time the CloudFormation template is generated. This is one way to allow a single source-controlled artifact to generate different CloudFormation templates for different regions or users. See tips and tricks for more information.

(Do you have other resolution and lookup strategies that you need to support? If so, please let us know and we can evolve how strategies are supported.)

The REST API

The next concept you will need to know is the REST API.

By default, it uses basic-auth - the credentials are the same as for logging into the web-console (i.e. user admin and password is the VM’s instance id).

The REST API for Export

A YAML blueprints can be POST’ed to the server, to export it to CloudFormation.

For example, given the following input file named /path/to/blueprint.yaml:

services:
  - type: AWS-EC2-Instance
    brooklyn.config:
      ImageId: ami-0bdb1d6c15a40392c
      InstanceType: t3.micro

The curl command below would return the auto-generated CloudFormation:

curl -k -u "admin:${EC2_INSTANCE_ID}" \
    --header "Content-Type: text/plain" \
    --data-binary @/path/to/blueprint.yaml \
    "https://ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com/v1/export/cloudformation"

Working with the Catalog

YAML Blueprints can be added to the catalog (i.e. the library), to be used as building blocks when writing CloudFormation templates. This can be done through the web-console, or via the REST API.

The catalog consists of ‘bundles’, which are versioned zip files containing these blueprints.

The bundles can be listed by running:

curl -k -u "admin:${EC2_INSTANCE_ID}" \
    "https://ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com/v1/catalog/bundles"
Tip: pipe the output in to a tool such as `jq` to pretty-print the returned json.

The blueprints (aka ‘types’) in a given bundle can be listed by running a command like:

curl -k -u "admin:${EC2_INSTANCE_ID}" \
    "https://ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com/v1/catalog/bundles/io.cloudsoft.aws.cfn-export-entities/0.2.0.SNAPSHOT/types"

Or all types across all bundles can be listed by running:

curl -k -u "admin:${EC2_INSTANCE_ID}" \
    "https://ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com/v1/catalog/types"

To add a bundle to the catalog, run:

curl -k -u "admin:${EC2_INSTANCE_ID}" \
    --header "Content-Type: application/x-zip" \
    --data-binary @/path/to/bundle.zip \
    "https://ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com/v1/catalog/bundles"

For a fuller description of the catalog and its functionality, see the Clousoft AMP documentation.

Using with Version Control and a Pipeline

One possible way of using Visual Composer is to create your blueprint in the graphical composer, but instead of exporting, get the YAML blueprint. This can be stored in version control.

The REST API can be used to auto-generate the CloudFormation templates, and this can be used in the downstream steps of the pipeline (e.g. validation, updating stacks, etc).