OpenStack: The Heat Orchestration Module

We’re going to look at how we can build an infrastructure template from a virtual device using the OpenStack orchestration module, Heat.

Heat requires the python-heat package, which is already available in the repositories of newer Linux distributions. It can be installed from the PyPI repository using the PIP utility. Installation instructions can be found under Access in the Projects menu of the Virtual Private Cloud control panel.

Key Concepts

Before getting into the specifics of working with Heat, we want to clarify what exactly “stacks” and “templates” are.

A stack is a set of cloud resources (machines, logical volumes, networks, etc.) that are interconnected to make up a single structure.

A template is a stack description and is usually a specially formatted text file. The template contains a description of the resources and their connections. Resources can be listed in any order: stacks are assembled automatically. Previously assembled stacks can be used as descriptions for other templates, letting you create nested stacks.

Let’s look a at an example of a template structure and how it’s written. We’ll create a stack made up of two servers, a local network, and a router that connects to a public network.

Template Format

Templates come in several formats. We’ll be using HOT format, which was created specially for Heat and uses a fairly simple and easy-to-understand syntax. The format is based on YAML, so it’s important to mind spaces in indents and hierarchy when editing texts.

The CFN format (AWS CloudFormation) is also supported to provide compatibility with Amazon EC2 templates.

Template Structure

We’ll use the following template to create a stack:

heat_template_version: 2013-05-23

description: Basic template for two servers, one network and one router

parameters:
  key_name:
    type: string
    description: Name of keypair to assign to servers for ssh authentication
  public_net_id:
    type: string
    description: UUID of public network to outer world
  server_flavor:
    type: string
    description: UUID of virtual hardware configurations that are called flavors in openstack
  private_net_name:
    type: string
    description: Name of private network (L2 level)
  private_subnet_name:
    type: string
    description: Name of private network subnet (L3 level)
  router_name:
    type: string
    description: Name of router that connects private and public networks
  server1_name:
    type: string
    description: Custom name of server1 virtual machine
  server2_name:
    type: string
    description: Custom name of server2 virtual machine
  image_centos7:
    type: string
    description: UUID of glance image with centos 7 distro
  image_debian7:
    type: string
    description: UUID of glance image with debian 7 distro

resources:

  private_net:
    type: OS::Neutron::Net
    properties:
      name: { get_param: private_net_name }

  private_subnet:
    type: OS::Neutron::Subnet
    properties:
      name: { get_param: private_subnet_name }
      network_id: { get_resource: private_net }
      allocation_pools:
        - start: "192.168.0.10"
          end: "192.168.0.254"
      cidr: "192.168.0.0/24"
      enable_dhcp: True
      gateway_ip: "192.168.0.1"

  router:
    type: OS::Neutron::Router
    properties:
      name: { get_param: router_name }
      external_gateway_info: { "enable_snat": True, "network": { get_param: public_net_id }}

  router_interface:
    type: OS::Neutron::RouterInterface
    properties:
      router_id: { get_resource: router }
      subnet_id: { get_resource: private_subnet }

  server1:
    type: OS::Nova::Server
    properties:
      name: { get_param: server1_name }
      block_device_mapping:
        - volume_size: 5
          volume_id: { get_resource: "server1_disk" }
          device_name: "/dev/vda"
      config_drive: "False"
      flavor: { get_param: server_flavor }
      image: { get_param: image_centos7 }
      key_name: { get_param: key_name }
      networks:
        - port: { get_resource: server1_port }

  server1_disk:
    type: OS::Cinder::Volume
    properties:
      name: server1_disk
      image: { get_param: image_centos7 }
      size: 5

  server1_port:
    type: OS::Neutron::Port
    properties:
      network_id: { get_resource: private_net }
      fixed_ips:
        - subnet_id: { get_resource: private_subnet }

  server1_floating_ip:
    type: OS::Neutron::FloatingIP
    properties:
      floating_network_id: { get_param: public_net_id }
      port_id: { get_resource: server1_port }
    depends_on: router_interface

  server2:
    type: OS::Nova::Server
    properties:
      name: { get_param: server2_name }
      block_device_mapping:
        - volume_size: 5
          volume_id: { get_resource: "server2_disk" }
          device_name: "/dev/vda"
      config_drive: "False"
      flavor: { get_param: server_flavor }
      image: { get_param: image_debian7 }
      key_name: { get_param: key_name }
      networks:
        - port: { get_resource: server2_port }

  server2_disk:
    type: OS::Cinder::Volume
    properties:
      name: server2_disk
      image: { get_param: image_debian7 }
      size: 5

  server2_port:
    type: OS::Neutron::Port
    properties:
      network_id: { get_resource: private_net }
      fixed_ips:
        - subnet_id: { get_resource: private_subnet }

outputs:
  server1_private_ip:
    description: private ip within local subnet of server1 with installed Centos 7 distro
    value: { get_attr: [ server1_port, fixed_ips, 0, ip_address ] }
  server1_public_ip:
    description: floating_ip that is assigned to server1 server
    value: { get_attr: [ server1_floating_ip, floating_ip_address ] }
  server2_private_ip:
    description: private ip within local subnet of server2 with installed Debian 7 distro
    value: { get_attr: [ server2, first_address ] }

Let’s take a closer look at this structure.

The template is made up of several blocks. The first tells us the template version and description format. Every new release of OpenStack supports its own set of properties and values, which are gradually changing. Our example uses version 2013-05-23. This supports all the features implemented in Icehouse.

heat_template_version: 2013-05-23

           description: >
  Basic template of two servers, one network and one router

The second block gives us a general description of the template and its values:

parameters:
  key_name:
    type: string
    description: Name of keypair to assign to servers for ssh authentication
  public_net_id:
    type: string
    description: UUID of public network to outer world
  server_flavor:
    type: string
    description: UUID of virtual hardware configurations that are called flavors in openstack
  private_net_name:
    type: string
    description: Name of private network (L2 level)
  private_subnet_name:
    type: string
    description: Name of private network subnet (L3 level)
  router_name:
    type: string
    description: Name of router that connects private and public networks
  server1_name:
    type: string
    description: Custom name of server1 virtual machine
  server2_name:
    type: string
    description: Custom name of server2 virtual machine
  image_centos7:
    type: string
    description: UUID of glance image with centos 7 distro
  image_debian7:
    type: string
    description: UUID of glance image with debian 7 distro

Then we list a few additional parameters which will be sent to Heat when we create the stack. We can set the keys for SSH connections to our new server in the key_name parameter. The server_flavor and public_net_id parameters act as identifiers (UUID) for the “hardware” configuration of the virtual machine and public network.

We can also assign names to the new devices and machines.

resources:

  private_net:
    type: OS::Neutron::Net
    properties:
      name: { get_param: private_net_name }


  private_subnet:
    type: OS::Neutron::Subnet
    properties:
      name: { get_param: private_subnet_name }
      network_id: { get_resource: private_net }
      allocation_pools:
        - start: "192.168.0.10"
          end: "192.168.0.254"
      cidr: "192.168.0.0/24"
      enable_dhcp: True
      gateway_ip: "192.168.0.1"


  router:
    type: OS::Neutron::Router
    properties:
      name: { get_param: router_name }
      external_gateway_info: { "enable_snat": True, "network": { get_param: public_net_id }}


  router_interface:
    type: OS::Neutron::RouterInterface
    properties:
      router_id: { get_resource: router }
      subnet_id: { get_resource: private_subnet }


  server1:
    type: OS::Nova::Server
    properties:
      name: { get_param: server1_name }
      block_device_mapping:
        - volume_size: 5
          volume_id: { get_resource: "server1_disk" }
          device_name: "/dev/vda"
      config_drive: "False"
      flavor: { get_param: server_flavor }
      image: { get_param: image_centos7 }
      key_name: { get_param: key_name }
      networks:
        - port: { get_resource: server1_port }


  server1_disk:
    type: OS::Cinder::Volume
    properties:
      name: server1_disk
      image: { get_param: image_centos7 }
      size: 5


  server1_port:
    type: OS::Neutron::Port
    properties:
      network_id: { get_resource: private_net }
      fixed_ips:
        - subnet_id: { get_resource: private_subnet }


  server1_floating_ip:
    type: OS::Neutron::FloatingIP
    properties:
      floating_network_id: { get_param: public_net_id }
      port_id: { get_resource: server1_port }
    depends_on: router_interface


  server2:
    type: OS::Nova::Server
    properties:
      name: { get_param: server2_name }
      block_device_mapping:
        - volume_size: 5
          volume_id: { get_resource: "server2_disk" }
          device_name: "/dev/vda"
      config_drive: "False"
      flavor: { get_param: server_flavor }
      image: { get_param: image_debian7 }
      key_name: { get_param: key_name }
      networks:
        - port: { get_resource: server2_port }


  server2_disk:
    type: OS::Cinder::Volume
    properties:
      name: server2_disk
      image: { get_param: image_debian7 }
      size: 5


  server2_port:
    type: OS::Neutron::Port
    properties:
      network_id: { get_resource: private_net }
      fixed_ips:
        - subnet_id: { get_resource: private_subnet }

The next block describes the resources we’ve created: networks, router, servers, etc. In this part of the template, we describe the local network (private_net), the subnet, its range of addresses, and enable DHCP support.

The next step is to create a router and its interface; the router will connect to the local network we’ve created via this interface. Then we see a list of servers. Each server should have a port and disk. The first server should also have a floating IP address (floting_ip) set, which an outside address from the public network can use to associate with “grey” addresses from the local network. Subsequent servers don’t need this.

  server1_floating_ip:
    type: OS::Neutron::FloatingIP
    properties:
      floating_network_id: { get_param: public_net_id }
      port_id: { get_resource: server1_port }
    depends_on: router_interface

Pay close attention to how parameters and resources are used when describing new devices. Above, we showed the description of a floating IP address for the first server. We need to define the UUID of the public network where it will get the floating IP address (floating_network_id) and the UUID for the server port (port_id) that will connect to the address. For the get_param function, we indicate that the value should be taken from the public_net_id (we’ll discuss how to use this parameter below). We still don’t have an identifier for the first server’s port though; this will only become available after the server has been created.

The get_resource function indicates that the value of server1_port will be used as the port_id UUID as soon as it’s created.

Resource DELETE failed: Conflict: Router interface for subnet 8958ffad-7622-4d98-9fd9-6f4423937b59 on router 7ee9754b-beba-4301-9bdd-166117c5e5a6 cannot be deleted, as it is required by one or more floating IPs.

According to this message, the router can’t be deleted because there is a floating IP address attached to the network the router connects to. As one would expect, before you can delete a stack, the IP address must first be deleted, then the router and network connected to it. The problem is that all of these components’ (Neutron, Cinder, Nova, Glance) resources are independent from one another; they are separate entities for which a connection and dependencies are created.

When creating a stack, Heat usually defines the order in which resources need to be created and the connections between them need to be established. These connections are also taken into consideration when a stack is deleted: they define the order resources are deleted. However, just like in our example above, errors do sometimes occur. We clearly defined that the floating IP address is attached to the router and its interface using the depends_on directive. Because of this connection, the IP address will now be assigned after the router and interface have been created. The order will be reversed when resources are deleted: first the floating IP address will be deleted, then the router and its interface.

In this last template fragment, we describe the parameters we need for our virtual devices. This way their values are set only after the stack has been created.

outputs:
  server1_private_ip:
    description: private ip address within local subnet of server 1 with installed Centos7 distro
    value: { get_attr: [ server1_port, fixed_ips, 0, ip_address]}
  server1_public_ip:
    description: floating ip that is assigned to server1 server
    value: { get_attr: [ server1_floating_ip, floating_ip_address ]}
  server2_private_ip:
    description: private ip address within local subnet of server2 with installed Debian7 distro
    value: { get_attr: [ server2, first_address ]}

In this fragment, we specify that we want the following values for resources that were created while the stack was being assembled: the address of the first server on the local network, the first server’s public address (floating IP address), and the address of the second server on the local network. We’ve entered a short description and the desired value for each parameter. We did this using the get_attr function, which requires two values: the first being the name of the resource, the second its attributes.

Pay attention to the different ways we can get a local network address from the first and second server. They are both perfectly acceptable. The difference is that in the first case, we refer to Neutron (remember: server1_port is set to OS::Neutron::Port) and the first IP address is taken from the fixed_ips attribute. In the second case, we refer to Nova (server2 is set to OS::Nova::Server) and the first_address attribute.

OpenStack components like Neutron and Cinder showed up after Nova. This is why Nova used to perform many more functions, including disk and network management. As Neutron and Cinder developed, these functions were no longer necessary, but were left in for consistency. Nova’s policies are slowly being revised, and several functions are being advertised as obsolete. It’s possible the first_address attribute will no longer be supported in the near future.

value: { get_attr: [ server1_port, fixed_ips, 0, ip_address]}
value: { get_attr: [ server2, first_address ]}

More information on templates and their makeup can be found in the official manual.

Creating a Stack

Now that we’ve prepared a template, we’ll check it for any syntax errors and against the standard:

$ heat template-validate -f publication.yml

If the template was properly compiled, then we’ll get the following response in json format:

{
  "Description": "Basic template of two servers, one network and one router\n", 
  "Parameters": {
    "server2_name": {
      "NoEcho": "false", 
      "Type": "String", 
      "Description": "", 
      "Label": "server2_name"
    }, 
    "private_subnet_name": {
      "NoEcho": "false", 
      "Type": "String", 
      "Description": "the Name of private subnet", 
      "Label": "private_subnet_name"
    }, 
    "key_name": {
      "NoEcho": "false", 
...

We need the following to create the stack:

$ heat stack-create TESTA -f testa.yml -P key_name="testa" \
-P public_net_id="ab2264dd-bde8-4a97-b0da-5fea63191019" \
-P server_flavor="1406718579611-8007733592" \
-P private_net_name=localnet -P private_subnet_name="192.168.0.0/24" \
-P router_name=router -P server1_name=Centos7 -P server2_name=Debian7 \
-P image_server1="CentOS 7 64-bit" \
-P image_server2="ba78ce9b-f800-4fb2-ad85-a68ca0f19cb8"

Manually transferring parameters to Heat is far from ideal; mistakes can easily be made. To get around this, we create an additional file that follows the format of the primary template, but only contains the main parameters.

parameters:
  key_name: testa
  public_net_id: ab2264dd-bde8-4a97-b0da-5fea63191019
  server_flavor: myflavor
  private_net_name: localnet
  private_subnet_name: 192.168.0.0/24
  router_name: router
  server1_name: Centos7
  server2_name: Debian7
  image_server1: CentOS 7 64-bit
  image_server2: ba78ce9b-f800-4fb2-ad85-a68ca0f19cb8

This makes it easier to create a stack using the Heat console utility.

$ heat stack-create TESTA -f testa.yml -e testa_env.yml
+--------------------------------------+------------+--------------------+----------------------+
| id | stack_name | stack_status | creation_time |
+--------------------------------------+------------+--------------------+----------------------+
| 96d37fd2-52e8-4b59-bf42-2ce72566e03e | TESTA | CREATE_IN_PROGRESS | 2014-12-17T15:17:17Z |
+--------------------------------------+------------+--------------------+----------------------+

We can use the standard set of OpenStack utilities to find the values of the requested Heat parameters. For example, we can use Neutron to find the public network identifier public_net_id:

$ neutron net-list
+--------------------------------------+------------------+-----------------------------------------------------+
| id | name | subnets |
+--------------------------------------+------------------+-----------------------------------------------------+
| 168bb122-a00a-4e34-bcc9-3bd0b417ee2b | localnet | 256647b7-7b73-4534-8a79-1901c9b25527 192.168.0.0/24 |
| ab2264dd-bde8-4a97-b0da-5fea63191019 | external-network | 102a9263-2d84-4335-acfb-6583ac8e70aa |
| | | aa9e4fc4-63b0-432e-bcbd-82a613310acb |
+--------------------------------------+------------------+-----------------------------------------------------+

Using this same method and the right utilities, we can find the names or identifiers for server_flavor, image_server1 and image_server2.

Stack Operations

After we’ve created a stack, we need to be sure that there haven’t been any errors and find out which IP addresses have been assigned to the server (starting with the public IP address of the first server).

A list of all created stacks can be retrieved using the heat-list command. The printout will include information on the status of each stack:

$ heat stack-list
+--------------------------------------+------------+-----------------+----------------------+
| id | stack_name | stack_status | creation_time |
+--------------------------------------+------------+-----------------+----------------------+
| e7ad8ef1-921d-4e70-a203-20dbc32d4a02 | TESTA | CREATE_COMPLETE | 2014-12-17T18:30:54Z |
| ab5159d2-08ad-47a2-a964-a2c3425eca8f | TESTNODE | CREATE_FAILED | 2014-12-17T18:39:38Z |
+--------------------------------------+------------+-----------------+----------------------+

As you can see from the printout, we didn’t properly set the UUID of the local network that our server port connects to. This is why we got an error. Errors usually occur due to a lack of available resources (every project has a limit to the number of cores, RAM, etc.).

If a stack is successfully created, then the “outputs” section of the stack-show printout will give us the parameters we’re interested in.

+----------------------+----------------------------------------------------------------------------------------------------------------------------------+
| Property | Value |
+----------------------+----------------------------------------------------------------------------------------------------------------------------------+
| capabilities | [] |
| creation_time | 2014-12-17T15:17:17Z |
| description | Basic template of two servers, one network and one |
| | router |
| disable_rollback | True |
| id | 96d37fd2-52e8-4b59-bf42-2ce72566e03e |
| links | https://api.selvpc.ru/orchestration/v1/58ad5a5408ad4ad5864f260308884539/stacks/TESTA/96d37fd2-52e8-4b59-bf42-2ce72566e03e (self) |
| notification_topics | [] |
| outputs | [ |
| | { |
| | "output_value": "192.168.0.10", |
| | "description": "private ip within local subnet of server2 with installed Debian 7 distro", |
| | "output_key": "server2_private_ip" |
| | }, |
| | { |
| | "output_value": "192.168.0.13", |
| | "description": "private ip within local subnet of server1 with installed Centos 7 distro", |
| | "output_key": "server1_private_ip" |
| | }, |
| | { |
| | "output_value": "95.213.154.134", |
| | "description": "floating_ip that is assigned to server1 server", |
| | "output_key": "server1_public_ip" |
| | } |
| | ] |
| parameters | { |
| | "server2_name": "Debian7", |
| | "image_centos7": "CentOS 7 64-bit", |
| | "OS::stack_id": "96d37fd2-52e8-4b59-bf42-2ce72566e03e", |
| | "OS::stack_name": "TESTA", |
| | "private_subnet_name": "192.168.0.0/24", |
| | "key_name": "testa", |
| | "server1_name": "Centos7", |
| | "public_net_id": "ab2264dd-bde8-4a97-b0da-5fea63191019", |
| | "private_net_name": "localnet", |
| | "router_name": "router", |
| | "server_flavor": "myflavor", |
| | "image_debian7": "d3e1be2a-e0fc-4cfc-ac07-35c9706f02cc" |
| | } |
| stack_name | TESTA |
| stack_status | CREATE_COMPLETE |
| stack_status_reason | Stack CREATE completed successfully |
| template_description | Basic template of two servers, one network and one |
| | router |
| timeout_mins | None |
| updated_time | None |
+----------------------+----------------------------------------------------------------------------------------------------------------------------------+

In most cases, the heat stack-show printout is extremely detailed and just plain huge. Finding a minute yet important detail (like the IP address of the first server) in the printout can be incredibly tedious. If we’re only interested in the first server’s floating IP address, then we can retrieve it using the following command, where we specify the stack name and public IP address:

$ heat output-show TESTA server1_public_ip
"95.213.154.192"

Stacks can easily be deleted using the heat stack-delete command:

$ heat stack-delete TESTA
+--------------------------------------+------------+--------------------+----------------------+
| id | stack_name | stack_status | creation_time |
+--------------------------------------+------------+--------------------+----------------------+
| e7ad8ef1-921d-4e70-a203-20dbc32d4a02 | TESTA | DELETE_IN_PROGRESS | 2014-12-17T18:30:54Z |
+--------------------------------------+------------+--------------------+----------------------+

In situations where you have to temporarily free up system resources, you can suspend a stack using the heat action-suspend command and resume it later with the heat action-resume command.

More detailed information can be found in the official documentation or using the command heat help.