VLDB 2021 video: Applying machine learning-based database tuning in production

During yesterday’s VLDB conference, I presented “An Inquiry into Machine Learning-based Automatic Configuration Tuning Services on Real-World Database Management Systems.” The paper was a collaborative effort among the Carnegie Mellon Database Group, Princeton University, Societe Generale, and OtterTune.

Autonomous database operation and tuning, driven by machine learning (ML), is becoming more prevalent. In this VLDB talk, I presented learnings that help DBMS users better understand how to benefit from ML more productively and safely.

You can read the research paper here.

Talk highlights

Tuning configuration knobs for modern database management systems (DBMSs) is essential, especially for workload-intensive applications, yet tuning is a complex process that typically requires an experienced database administrator (DBA). There are past and current efforts to automate the process, starting with heuristics tools in the early 2000s and leading up to today’s more advanced ML-based tools, including OtterTune. Evaluations have shown that ML-based tuners can achieve better performance compared to DBAs and other tuning tools, but what happens when ML-based tuning tools are used in real-world deployments?

This study discusses three “mismatches” between previous research studies and real-world deployments:

1. Studies only consider open-source database systems.

2. They use synthetic benchmarks that don’t consider real workloads.

3. They run their experiments on dedicated local storage.

Automatic DBMS tuning in production at Societe Generale

Given these issues, my collaborators and I were eager to complete a field study of automatic knob configuration on a commercial DBMS (Oracle) with a real-world workload in a production environment at Societe Generale (SG).

In 2020, SG agreed to work with our team to test out the OtterTune ML-based tuning framework. The goal was to see whether automated tuning could improve a DBMS’s performance beyond what their DBAs achieve through manual tuning. In this talk, I discuss the set-up at SG, as well as the methodology for tuning using OtterTune. The OtterTune and SG teams ran into several unexpected hurdles along the way—but in the end, the results were impressive. The ML algorithms generated knob configurations that improved performance by up to 45% over ones generated by a human expert.