Five Levels of Analytics

How well does your business intelligence system enable your employees?

I will, from time to time talk about my profession as a data engineer with 30 plus years in the business. I think what qualifies me to write as a stoic and as a social critic has much to do with my daily discipline of testing to see whether or not the systems I designed and built yesterday are still working today. I am a strong advocate of this kind of critical thinking. Ever since 9/11 I would wake up with the question: what broke while I was sleeping? This is one of the things that lead me to stoicism and how I keep my head on straight.

Bear with me as I use some terms that might not be familiar with you but I brought this together with an aim to give a good overview of how to think about growing an established system for the data-driven business. As always, I encourage you to use the comments section.

—-

As 'Chief Iteration Officer' my mantra is 'iterate to quality'. That means preserve what you know and get a little bit better every day. I have always been from the Deming School and don't often advocate the Hammer approach. I like John Boyd's OODA loop and I think it works wonders for business processes. As systems integrators, we don't always get greenfield projects. Most of the time we are given to take over systems that are broken, incomplete, poorly performing or out of date. Just having 'analytics' in place is not sufficient. The general idea is always good, but how well are you doing?

This is a guide to help describe the level of thoroughness of a business intelligence system.

Discovery - Level One

• I can use my system to know what information is available to me.

This is the broadest and most basic level. It means that as a user, you know which systems are available to you, how to access those systems and what you can expect to find in those systems. The cash flow reports are here. The current reports are here. The historical reports are there. The systems that identify operations are there. You can navigate to all of the systems relevant to your area of responsibility; you know their names, functions and update timeliness.

Clarity - Level Two

• The information available to me is clear and unambiguous.

This means that master data issues are resolved. Codes and spellings of descriptions are complete. There is no question that data is in the proper range. It is validated and audited so that there is little doubt that you are dealing with the right information at the right time. You are aware of the units of measurement and these units are converted into standardized, consistent, useable form.

Purpose - Level Three

• I understand how the information relates to the business at hand.

Purpose is achieved when data becomes information and you can describe the business in terms of that information. It might only describe the costs or staffing levels of the business, but it is specifically applicable to the operational function of the particular business unit. You may have unambiguous clarity about 'gallons of gas available' but it serves no purpose to the legal affairs office. What is measured should match what is known about the operations, whether or not the information is managed. Also, proportionality of the numbers themselves are understood at this level. One familiar with the business should know how various measures and metrics relate to each other, if at all.

Predictability - Level Four

• I can determine with precision what specific information describes business performance.

At this level of analytic depth, system users are capable of isolating that information that is a primary indicator of business performance as contrasted to those that might be trailing indicators or otherwise not driving the business. It is the difference between knowing when a small fluctuation or a big fluctuation is material to business performance. It might be something like a regulatory ratio that must be maintained, a quality control statistic for pass/fail of a manufactured lot of goods. It might be a stated performance objective like a tight budget or schedule variance. Industry standard metrics would be here.

Responsibility - Level Five

• I can accurately predict and plan business performance with my system.

The final level of analytic depth is responsibility. At this point reasonable budgets and forecasts are be made on key performance indicators. Variances are taken against actuals and not only business performance, but prediction performance is tracked. A well-managed business is data driven at this level of analysis and planning, and novel KPIs and metrics might be generated whose meaning and value are quickly evident.