Skip to main content

Everyone wants to be data driven but wielding data effectively and consistently is difficult. Only a few businesses achieve the level of data-driven sophistication they aspire to. There are a number of hurdles that one encounters along the way to building a successful data-driven fly-wheel.

In this two part series, we will cover what it takes to achieve metrics monitoring maturity. Here we will first cover the building blocks that are prerequisites for success. In the next follow-up blog post, we will briefly describe the 5 stages of analytics monitoring maturity and discuss how you can accelerate your progression.

Fig 1. The journey to achieve monitoring maturity can be long and challenging.

What you need to achieve analytics monitoring maturity

When we look at DevOps (or even MLOps), we see that there is a strong understanding of what excellence in workflows and incident management looks like. However, when it comes to analytics we find that there is poor understanding of what maturity entails. This is partly because engineering is upstream and user-facing, therefore monitoring server and software performance is imperative, while monitoring downstream data and business performance was seen as a nice-to-have. However, we believe that the time has come to apply the same rigor and discipline that exists for monitoring application performance and uptime to monitoring metrics and your overall analytics.

We identify the three following attributes that you need to achieve analytics monitoring maturity:

1

Infrastructure Quality

Wherever your data comes from it needs to be reliable. This means that it needs to be well-collected (adequately tagged), consistent (available on cadence), and interpretable (transformed and unambiguous).
2

Metric Observability

No business or product leader wants to query a SQL table or create a Mixpanel report. Metric observability means that decision-makers have access to the metrics they need when they need it. There are two parts to observability:
a) Quality reporting via dashboards that present the data in and easy to consume format.
b) Tools that proactively alert you to changes in metrics that require your attention.
3

Root Cause Analysis (RCA)

Flagging high level issues in performance is never enough. Root Cause Analysis (RCA) means you can pinpoint the drivers responsible for fluctuations in your key metric performance so that you can fix it. Typically, today this is achieved with more (deep-dive) dashboards or ad-hoc manual analyses.

In our next post we describe how businesses sequentially address these attributes and reach maturity or where many get stuck. We’ll also show you how to leapfrog ahead to maturity without getting mired in the problems that many before you have faced.

Out of the Blue™ is building a solution to address the shortcomings of today’s existing pull-based tools and to empower data-driven professionals to give them what they need when they need it. Stay tuned for more.

Learn how we do it, Get in Touch

Author

  • Mohit is a former product manager turned data scientist. He led analytics at Opower, Nest, and Google, successfully using data to build products, launch features, and grow users. He has an MBA from the University of Chicago, Booth School of Business and a BS in Engineering from University of California, Berkeley.

Leave a Reply