OpenSearch diagnostics and capacity forecasting tooling
Built so the team stops firefighting the OpenSearch clusters and starts planning for them.
In plain terms
The OpenSearch clusters I manage are groups of servers that store and search client logs. When one has a problem, someone has to work out why, and that meant checking a list of things by hand, one after another.
Clusters also have limits, like how much they can store. There was no warning before a cluster reached them. And when a new compliance cluster was needed, deciding how big to make it was guesswork.
The problem
Three gaps, in short: slow diagnosis, no early warning and guesswork sizing.
What I did
I wrote Python tooling for the OpenSearch clusters. It cuts routine diagnosis from about 45 minutes to about two minutes, collects ingestion statistics per source, and forecasts when each cluster will reach its capacity limits.
It can also provision and configure new compliance clusters, sized from the ingestion they'll actually receive.
The outcome
Routine checks no longer tie up an engineer, and the team sees capacity limits coming instead of running into them.