feat: add alerts for cost monitoring system#8238
Conversation
|
Merging this PR will trigger the following deployment actions. Support deployments
Staging deploymentsNo staging hub upgrades will be triggered Production deploymentsNo production hub upgrades will be triggered |
6498e7e to
41988cb
Compare
525365c to
ab6687e
Compare
|
@jnywong what is needed to drive this to completion? |
|
The image bumps is so that the cost monitoring services exports Prometheus metrics for every HTTP request made to the system plus updating the helm chart to add pod annotations to tell Prometheus to scrape from the cost-monitoring pod. The remaining work in this PR is to come up with a sensible alert(s) for the cost monitoring application that I want to be alerted on via PagerDuty. I'll add this to the top card so that's clearer. I've not been able to push this work to completion because I've been focused on other things, but I hope we can pick this up again as goal-driven work for this quarter. |
Ref #8123
To Do
support/values.jsonnetwith grouped cost-monitoring alerts