Getting Started with Hashicorp Nomad and Consul

Why write another learning guide?

The official Nomad learning guides provide excellent examples for deploying workloads with Docker but very few showcase orchestration of non-containerized workloads. As a result of this I actually found it a little tough to get some of my first jobs spun-up. When it came time to on-board some of my co-workers, I decided to provided some examples and eventually it became a series of workshops that I was able to teach every other week.

Install Consul and Nomad

Get Consul and Nomad started in dev mode

Both the nomad and consul binaries will run in the foreground by default. This is great because you can watch log lines as they come in and troubleshoot any issues encountered while working through these workshops. However, this also means you'll want to use tmux or using some kind of native tab or window management to keep these running in a session other than the one you'll be using to edit, plan, and run Nomad jobs.

Start the Consul server in dev mode:

$ consul agent -dev -datacenter dev-general -log-level ERROR

Start the Nomad server in dev mode:

$ sudo nomad agent -dev -bind 0.0.0.0 -log-level ERROR -dc dev-general

Ensure you can access the Consul and Nomad web UIs

Open the Consul web UI: http://localhost:8500/ui
Open the Nomad web UI: http://localhost:4646/ui

Clone the letsencrypt/hashicorp-lessons repository

This repository contains both the starting job specification and examples of how your specification should look after we complete each of the workshops.

Scan through this commented job specification

Don't worry if it feels like a lot, it is. It took me a few days of working with Nomad and Consul to get a good sense of what each of these stanzas was actually doing. Feel free to move forward even if you feel a little lost. You can always come back and reference it as needed.

// 'variable' stanzas are used to declare variables that a job specification can
// have passed to it via '-var' and '-var-file' options on the nomad command
// line.
//
// https://www.nomadproject.io/docs/job-specification/hcl2/variables
variable "config-yml-template" {
  type = string
}

// 'job' is the top-most configuration option in the job specification.
//
// https://www.nomadproject.io/docs/job-specification/job
job "hello-world" {

  // datacenters where you would like the hello-world to be deployed.
  //
  // https://www.nomadproject.io/docs/job-specification/job#datacenters
  datacenters = ["dev-general"]

  // 'type' of Scheduler that Nomad will use to run and update the 'hello-world'
  // job. We're using 'service' in this example because we want the
  // 'hello-world' job to be run persistently and restarted if it becomes
  // unhealthy or stops unexpectedly.
  //
  // https://www.nomadproject.io/docs/job-specification/job#type
  type = "service"

  // 'group' is a series of tasks that should be co-located (deployed) on the
  // same Nomad client. 
  //
  //https://www.nomadproject.io/docs/job-specification/group
  group "greeter" {
    // 'count' is the number of allocations (instances) of the 'hello-world'
    // 'greeter' tasks you want to be deployed.
    //
    // https://www.nomadproject.io/docs/job-specification/group#count
    count = 1

    // 'network' declares which ports need to be available on a given Nomad
    // client before it can allocate (deploy an instance of) the 'hello-world'
    // 'greeter'
    //
    // https://www.nomadproject.io/docs/job-specification/network
    network {
      // https://www.nomadproject.io/docs/job-specification/network#port-parameters
      port "http" {
        static = 1234
      }
    }

    // 'service' tells Nomad how the 'hello-world' 'greeter' allocations should be
    // advertised (as a service) in Consul and how Consul should determine that
    // each hello-world greeter allocation is healthy enough to advertise as
    // part of the Service Catalog.
    //
    // https://www.nomadproject.io/docs/job-specification/service
    service {
      name = "hello-world-greeter"
      port = "http"

      // 'check' is the check used by Consul to assess the health or readiness
      // of an individual 'hello-world' 'greeter' allocation.
      //
      // https://www.nomadproject.io/docs/job-specification/service#check
      check {
        name     = "ready-tcp"
        type     = "tcp"
        port     = "http"
        interval = "3s"
        timeout  = "2s"
      }

      // 'check' same as the above except the status of the service depends on
      // the HTTP response code: any 2xx code is considered passing, a 429 Too
      // ManyRequests is warning, and anything else is a failure.
      check {
        name     = "ready-http"
        type     = "http"
        port     = "http"
        path     = "/"
        interval = "3s"
        timeout  = "2s"
      }
    }

    // 'task' defines an individual unit of work for Nomad to schedule and
    // supervise (e.g. a web server, a database server, etc).
    //
    // https://www.nomadproject.io/docs/job-specification/task
    task "greet" {
      // 'driver' is the Task Driver that Nomad should use to execute our
      // 'task'. For shell commands and scripts there are two options:
      //
      // 1. 'raw_exec' is used to execute a command for a task without any
      //    isolation. The task is started as the same user as the Nomad
      //    process. Ensure the Nomad process user is sufficiently restricted in
      //    Production settings.
      // 2. 'exec' uses the underlying isolation primitives of the operating
      //    system to limit the task's access to resources
      // 3. There are many more here: https://www.nomadproject.io/docs/drivers
      //
      // https://www.nomadproject.io/docs/job-specification/task#driver 
      driver = "raw_exec"

      config {
        // 'command' is the binary or script that will be called with /bin/sh.
        command = "greet"
        
        // 'args' is a list of arguments passed to the 'greet' binary.
        args = [
          "-c", "${NOMAD_ALLOC_DIR}/config.yml"
        ]
      }

      // 'template' instructs the Nomad Client to use 'consul-template' to
      // template a given file into the allocation at a specified path. Note:
      // that while 'consul-template' has 'consul' in the name, 'consul' is not
      // required to use it.
      // https://www.nomadproject.io/docs/job-specification/task#template
      // https://www.nomadproject.io/docs/job-specification/template
      template {
        // 'data' is a string containing the contents of the consul template.
        // Here we're passing a variable instead of defining it inline but both
        // are perfectly valid.
        data        = var.config-yml-template
        destination = "${NOMAD_ALLOC_DIR}/config.yml"
        change_mode = "restart"
      }

      // 'env' allows us to pass environment variables to our task. Since
      // consul-template is run locally inside of allocation, if we needed to
      // pass a variable from our job specification we would need to pass them
      // in this stanza.
      // https://www.nomadproject.io/docs/job-specification/task#env
      //
      env {
        // 'foo' is just an example and not actually used in these lessons.
        foo = "${var.foo}"
      }
    }
  }
}

Ready to deploy your first job?

Continue on to Nomad Workshop 1 - Hello World.