Location Aware Elasticity in OpenNebula

In the context of the Beacon Project, OpenNebula needs to be able to define Virtual Machine placement policies with geographical location constraints. 

Moreover, Beacon use cases call for location-aware elasticity (also called auto-scaling). This post describes the framework defined to achieve it.

First, a OneFlow Service needs to be defined. OneFlow is an OpenNebula component that allows users and administrators to define, execute and manage multi-tier applications, or services, composed of interconnected Virtual Machines with deployment dependencies between them. Each group of Virtual Machines is deployed and managed as a single entity. This component also provides auto-scaling capabilities based on configurable policies, using performance metrics and scheduled times.

A service contains Roles. Each role is a group of identical Virtual Machines, with a certain initial cardinality (number of Virtual Machines). Following the auto-scaling elasticity policies, OpenNebula adjusts the cardinality of each Role individually.

To implement location-aware scalability, a Service can be defined with a Role for each geographical location. The following figure represents a Service that holds a simple application with a front-end machine, and several worker nodes. The worker nodes are separated into three Roles: one for the local infrastructure, one for the EC2 US region, and one for EC2 Europe region.

 

Representation of a OneFlow Service with 3 geographical locations

Each role defines a set of elasticity rules, that will act individually for each role. The metric used to control the elasticity will be called CLIENT_REQ_MIN, and it is suggested as a monitoring value that the application will report to OpenNebula, periodically, with the number of client requests per minute.

In Sunstone, the OpenNebula web interface, that OneFlow service template is defined as follows:

 

OneFlow Template. Note: the elasticity policies are individual for each Role

To implement the location-aware placement, each Role uses different VM Templates, that in turn has unique placement requirements, expressed as:

SCHED_REQUIREMENTS = “HOSTNAME = \"ec2-us-east-1\"”

 

Virtual Machines deployed in the correct location by the OpenNebula scheduler

In order to trigger elasticity rules, OpenNebula VMs must be able to collect relevant information from the remote cloud providers.

The metrics retrieved by the provider’s API are limited, and in some cases they may not be relevant to the performance definition for the application running inside the Service. For this reason, OpenNebula allows VMs to report their own internal metrics using the OneGate component. OneGate exposes a REST interface that, combined with an automated authentication token management, allows the Virtual Machines to push and pull information about the VMs themselves, and the OneFlow service they are part of.

High-level architecture of OneGate monitoring

The application running inside the VMs will periodically report the application performance in each one of the worker nodes. This could be done with a command such as:

$ curl -X "PUT" "${ONEGATE_ENDPOINT}/vm" \
--header "X-ONEGATE-TOKEN: `cat token.txt`" \
--header "X-ONEGATE-VMID: $VMID" \
-d "CLIENT_REQ_MIN = 12"

That attribute, stored in the OpenNebula database, will be accessible by the OneFlow component, in charge of the Service scalability.

 

OneFlow Roles cardinality after the metric is pushed to the VM metadata

The previous image shows the OneFlow service with the cardinality adjusted only for the ‘eu’ role. The expression “CLIENT_REQ_MIN > 10” is responsible for this scale-up action.