Nov 4, 2021
How OtterTune securely collects database metrics
Tuning databases requires information collection, and that means access to schema, indexes, queries, and application data, right? And tuning via a public service means you need to expose your database for external access, right?
Not in OtterTune’s case. Like their cousin the honeybadger, Otters don’t care*. OtterTune automates database configuration tuning by analyzing knob settings and metrics–not application data, queries, or schema.
We designed OtterTune to collect metrics from your database securely, via an open source Agent. The OtterTune Agent solves the problem of having to expose your database — it’s software you can deploy within your virtual private cloud (VPC) as a Kubernetes pod, on an EC2 instance, or anywhere else it can talk to your database. With the OtterTune Agent, you can connect to the database privately and securely, and send its knob/metric data back to the OtterTune tuning service.
The Agent reports metrics back to the OtterTune tuning service via a secure protocol, but the OtterTune service does not need to talk back to the agent. This means that you can set up an OtterTune Agent without allowing any inbound traffic to your network, helping to keep your system safe from any outside attacks. The only data from your systems that will be sent out by the agent are internal database metrics and AWS Cloudwatch information.
When setting up a database that is monitored by the OtterTune Agent, you will be directed to an Agent Setup page. Note that the OtterTune Agent needs valid AWS credentials for a user or role to inspect the database and collect CloudWatch metrics for the database you wish to optimize (see the agent’s AWS access here). You then deploy the Agent as an AWS Fargate stack or launch a Docker container in your network. The OtterTune service generates a secret API Key that is used to authenticate Agent information being passed to the OtterTune service. See the OtterTune documentation for recommended approaches.
Fewer security worries
The OtterTune Agent runs in your internal environment, so there’s no need to expose your database to external traffic. Nor is there any requirement to allow inbound access to the database to the OtterTune service. The Agent only pushes metrics and knob settings information out; it does not accept data in.
The OtterTune service also does not look at any sensitive user data or queries. OtterTune only collects runtime metrics from the database (e.g., InnoDB stats, pg_stat_database) and CloudWatch. Check out the relevant code here. These performance measures are enough of a signal to tell how your application uses the database and how to optimize the system accordingly.
You may wonder why you would need the Agent if you are already using a monitoring system that collects metrics.
MySQL and PostgreSQL expose many internal (native) runtime metrics that are not available in CloudWatch but are useful for OtterTune’s machine learning-based configuration recommendations. The OtterTune Agent is necessary for collecting these metrics because it requires connecting to the customer database (CloudWatch metrics can be collected through AWS’s APIs).
The OtterTune Agent is also able to collect the database’s internal runtime metrics more frequently than CloudWatch metrics. Additionally the metrics OtterTune gathers are useful for performance monitoring too, and customers can view them in the Performance Charts in OtterTune’s UI.
We open sourced the OtterTune Agent so you can see exactly how it connects to your database and what information it collects. We also welcome pull requests from the community if you’re interested in contributing!. See the repository on GitHub.
*Members of the Mustelidae family are viciously cute and uninterested in the contents of any database.