AMP Cluster

AMP Cluster is a scalability solution that is designed to increase the number of applications that can simultaneously be managed by AMP from the hundreds of a single AMP node, to many tens of thousands.

It does so by deploying a horizontally scalable worker pool of AMP nodes that can be interacted with via a Cluster API. This allows a user to easily deploy and manage their applications via the cluster.

An AMP Cluster itself can effortlessly be deployed and managed by an AMP using the AMP Cluster blueprint.


AMP Cluster consists of three main components:

  • A rest based cluster API which provides the access point to the cluster. The API for this is documented here.

  • A CouchDB database which holds information about the structure of the AMP Cluster.

  • A pool of AMP nodes from which, the managed applications are orchestrated.

For further information please refer to the AMP Cluster structure documentation.

Getting Started

For a tutorial on the basics of AMP Cluster, see the tutorial section of this guide.

AMP Cluster is included in AMP from version 4.7 onwards, please ensure you are using at least this version of AMP. Further details for deploying a blueprint can be found here.

Deployment of this should create three new scalable clusters for the API, Databases and AMPs.


AMP Cluster provides a rest API endpoint which can be queried using any standard rest client. To list all managed apps using curl for example, you can use run the following bash script :

curl -u username:password $AMP_CLUSTER_ENDPOINT/v1/cluster/applications/

Where $AMP_CLUSTER_ENDPOINT is the main.uri of your AMP Cluster. For more detailed information on using a deployed AMP cluster please consult the cluster API documentation here.


The following config keys are available for the AMP Cluster amp-deployment entity:

Config Key Default Description   A link to the version of AMP to install   A link to the jar file for the cluster-api server.
amp.username admin The AMP username
amp.password password The AMP password
brooklyn.cfg.additions   Any text included here will be appended to the brooklyn.cfg file.
brooklyn.osgilauncher.cfg.additions   Any text included here will be appended to the end of the org.apache.brooklyn.osgilauncher.cfg file. This parameter can conflict with amps.persistence.location and amps.persistence.dir so do not set these in both params. /var/lib/amp/worker The persistence directory that the worker cluster will use (with a suffix appended automatically so workers will have different paths)
amp.https.enabled false Set to true if amp should be deployed with https enabled
amp.port 8081 Port to be used if amp.https.enabled is set to false
amp.https.port 8443 Port to be used if amp.https.enabled is set to true
amp.https.cert   The tls certificate to use, must be set if amp.https.enabled
amp.https.key   The tls private key, must be set if amp.https.enabled   The tls ca cert to append to the certificate amp-cluster The name of the CouchDB database amp-cluster will use
couchdb.username admin The default CouchDB admin user
couchdb.password 1p4s5w0rd The default CouchDB admin password
couchdb.sharedsecuritygroup.create true Enable a shared security group customizer
clusterapi.username user The api-server admin user
clusterapi.password password The api-server admin password
clusterapi.ssl   SSLProxy configuration that sets the api cluster loadbalancer up with ssl certificates
workerAmps.initialSize 1 Number of Worker AMPs in the initial deployment
clusterApi.initialSize 1 Number of Cluster API servers in the initial deployment
couchdb.initialSize 3 Number of CouchDB servers in the initial deployment
amp.maxMem   The ‘Xmx’ value to use when starting AMP
clusterapi.maxMem   The ‘Xmx’ value to use when starting the API Server

Object Store Persistence

It is recommended to use an object store for the persisted state of the AMP Workers. Each AMP Worker should write its state to a different bucket, or a separate sub-folder of a bucket.

The simplest way to do this is to set the AMP will then use this to dynamically create bucket(s) for each amp-worker and will handle failover when a node is replaced. AMP generates a unique path for each AMP Worker by appending a suffix to this configuration value. If the configuration value contains a forward slash (/), then the part before the slash is the bucket name and the part after is the path prefix. If there is no forward slash, then the entire value (including the auto-generated suffix) is the bucket name.

Below is an example for AWS-S3:

- type: amp-deployment
    aws.identity: <identity>
    aws.credential: <credential>
    brooklyn.cfg.additions: |
    brooklyn.osgilauncher.cfg.additions: |
      persistenceLocation=aws-loc my-amp-cluster-bucket/my-path-prefix

An alternative to an object store is to use a shared NFS mount so that the replacement VM for a failed AMP Worker can see the persisted state directory of the previous AMP Worker.

Externalized Configuration

For production blueprints running on the AMP Workers, it is important that they use externalized configuration, rather than credentials appearing in plain text.

This requires configuring brooklyn.cfg on each AMP Worker, for example with a Vault endpoint and credential.

This is done by specifying the configuration in brooklyn.cfg.additions. For example: services: - type: amp-deployment brooklyn.config:
brooklyn.cfg.additions: | $brooklyn:formatString(“ brooklyn.external.supplierName = org.apache.brooklyn.core.config.external.vault.VaultTokenExternalConfigSupplier brooklyn.external.credentials.endpoint = brooklyn.external.credentials.token = %s brooklyn.external.credentials.path = secret/amp-worker”, $brooklyn:external(“amp-master-creds”, “worker-amp-vault-token”))

Indeed, it is also strongly recommended to use externalized configuration for the amp-deployment blueprint, when specifying the vault token to be injected (as shown above).


AMP Cluster is currently design to enable the mass deployment of many small to medium applications (< 200 VMs). It is currently not designed to enable the sharding and deployment of large applications. In Addition the following developments are planned but not yet available:

  • Auto Scaling of the AMP Worker Cluster

  • Reservation of space on an AMP for an application which is expected to grow

  • Sophisticated selection strategies for placement of applications, such as AMP with least load, regional placements etc.

  • Advanced authentication mechanisms such as single sign-on