New deployment methods for the OtterTune Agent

The OtterTune Agent is an open-source Python program that collects monitoring data from your database for OtterTune to analyze. Many of our customers opt to install the OtterTune Agent instead of using Guarded Public Access because it offers a secure, auditable application that can run in any environment.

At OtterTune, we want installing the agent to be as easy as possible. Toward that end, we’ve recently expanded our guided onboarding experience to include two new ways to install the agent in addition to the two existing approaches, totaling four guided onboarding experiences available on our website:

• CloudFormation Stack

• Docker

• Kubernetes Manifest (New!)

• Helm Chart (New!)

In this post, we’ll describe each of the four guided onboarding approaches and identify the different use cases for each.

CloudFormation Stack

OtterTune has a CloudFormation (CF) Stack template to deploy the agent on Fargate. You can fill out a form directly in the AWS console with your credentials and API key, and CloudFormation will take care of configuring, permissioning, and deploying the agent as a Docker container in AWS Fargate. This is by far the fastest way to get up and running and it’s especially useful for organizations that already rely heavily on CloudFormation to manage their infrastructure. However, organizations with less CF familiarity might feel the process has too much magic.

Pros

• The fastest approach to get up-and-running.

• Cheap and effective. Using a Fargate container lets you right-size the agent so you don’t pay for more resources than you need.

• Fire and forget. Fargate is responsible for restarting failed containers.

• Predictable pricing, if you’re tracking it.

Cons

• Harder to track cloud spending. Many organizations prefer tracking their infrastructure in GitHub?. Resources created in the AWS console using this onboarding guide may need to be added to Source Control at a later time.

• Easy to lose track of the cost. If you don’t track your costs in Source Control, you can find your budget slowly draining over time. It’s easy to create a resource in a far-off region with CF and forget to shut it down later. AWS’s billing information, though extensive, isn’t always easy to parse.

Docker

The OG reliable method of software packaging and delivery. You can’t go wrong with a Docker image. OtterTune distributes a public Docker image via Amazon Elastic Container Registry Public. Docker allows users to install and deploy the image in seconds on most major operating systems. Additionally, most operators are familiar with Docker, and using the OtterTune Agent is no different than any other Docker image!

The biggest weakness with using Docker is database connectivity. Since the agent needs to connect to your AWS database, the Docker container must be running on a host with network access to the database. This could mean using a VPC and running the Docker container locally, or running the container on an EC2 node on an internal network.

Additionally, Docker containers aren’t automatically restarted if they fail, so if the container crashes due to a system reboot, your monitoring data will look lumpy and OtterTune won’t be able to generate new recommendations for knob settings. One quick solution to this is to run boot your Docker container with your operating system’s init system. For example, on Linux distros that use systemd, you can get away with a simple unit file like this:

[Unit] Description = "OtterTuneAgent"  
[Service] KillSignal=INT 
ExecStart=/usr/local/bin/docker run public.ecr.aws/ottertune/agent 
Restart=always 
ExecStopPost=sleep 5

Pros

• Super familiar for many developers, making knowledge transfer lightweight.

• Configure the agent using environment variables, following 12-factor methodology.

• Best choice for anyone running the agent on a VM or bare metal.

• Best choice for anyone running a Containers-As-A-Service platform like Google Cloud Run or Docker Swarm.

Cons

• No local network access to the database.

• Requires tweaking a supervisor to restart containers to prevent possible service disruption.

• Most production platforms use a service orchestrator on top of Docker containers, requiring extra configuration.

Kubernetes Manifest

As mentioned in the last section, most cloud-native production architectures use a service orchestrator to supervise Docker containers. Far and away the most popular service orchestrator is Kubernetes (K8s). If you’re already a K8s shop, then the Docker onboarding guide just isn’t enough to get you up and running.

The Kubernetes manifest brings you one step closer by providing an example Deployment resource for the OtterTune agent. Just like any Kubernetes Deployment, the agent will self-heal if the host node goes down, avoiding the problems discussed in the Docker section above.

The K8s manifest is especially useful for organizations whose RDS instances are only accessible from applications inside of the cluster. Using the manifest, you can deploy the agent alongside your applications, connecting the agent to RDS just like they do.

However, K8s clusters are highly variable, and you might have to edit the manifest quite heavily to make it fit your cluster. The manifest we provide is the simplest possible, to make it as comprehensible as possible to those with little K8s exposure. If you have more sophisticated requirements for applications in your cluster, you may find it easier to pull code from existing Deployment manifests in your cluster than using this bare-bones example.

For example, if you want to use a ConfigMap or Secret to configure the OtterTune agent, you’ll likely be writing twice as much YAML than you get in this manifest.

Pros

• Cluster networking allows you to securely connect to RDS instances.

• Fastest approach if you already have a cluster.

• Best choice for simple clusters.

Cons

• Overly simplistic manifest for most production scenarios.

• Only applicable for organizations with a Kubernetes cluster.

• Can’t check the manifest into version control without storing sensitive information.

Helm Chart

Continuing with the theme of increasingly sophisticated container setups, OtterTune offers an open source Helm Chart to handle more advanced Kubernetes setups. With the Helm Chart, you can perform more complex administrative tasks, like creating a ServiceAccount, attaching annotation labels, node affinities, and resource constraints to the Agent Deployment, and storing credentials in Secrets. The Helm Chart is the most production-ready option for organizations using Kubernetes. If you’re already using Helm, installing the Agent is a breeze:

1. Run this command: helm repo add ottertune https://ottertune.github.io/agent-helm-chart/

2. Create your values.yaml file.

3. Lastly, run this command: helm install

If you’re not using Helm, consider giving it a try! Helm is the most widely used tool for Kubernetes package installation. We don’t have any plans to add guided onboarding experiences for Helm alternatives like Cue, Dhall, or Kustomize, but let us know if that’s something you’d like to see. Helm can be a lot to learn, so if you want to go this route, there are several things to keep in mind:

Pros

• Most production-ready approach for Kubernetes clusters.

• Solves the problems present in the Docker and Kubernetes Manifest guides.

• Easy support for Secrets – keep your credentials private!

• Configuration is extremely flexible and pragmatic.

Cons

• Helm, and Kubernetes more broadly, have a very high learning curve.

• The Helm Chart would benefit from supporting a wider set of features, like volume mounts.

• By default Helm still store values in plaintext in etcd, so sensitive information like API keys and AWS credentials should be provided using an existing Kubernetes Secret. Check out the documentation for more.

• Helm isn’t a perfect fit for everyone. If your organization is avoiding it, you might have to fall back on the Kubernetes manifest onboarding guide, and write a lot of YAML by hand.

We want to make OtterTune set-up as simple and secure as possible, so we’re providing you with several options to deploy the OtterTune Agent deploy within your virtual private cloud (VPC) as a Kubernetes pod, on an EC2 instance, or anywhere else it can talk to your database.

If you’re a curious Otter, then come talk to us on Slack. Our team is more than happy to help.