Key Performance Indicators

To repurpose the quote above, what if the portions were really large? I could see that being a place that some people might go to frequently. The food stinks, but man do you get a lot of it. That is how many teams operate without realizing it. It is so easy to produce a lot of work without any of it actually being valuable to customers.

So what can we do to avoid producing a lot of food that stinks?

We have to measure what we're doing.

KPIs are essential for understanding the impact of our work. Without them, we really have no idea if what we're working on has any value. And focusing on the wrong KPIs is a great way to serve up a lot of food, but it may not be the kind of food that our target audience wants to eat.

For a while when I first started in my current role, we didn't have well-defined KPIs. And we got whiplash because of it. It was incredibly easy for anyone to come to our team and request new products and features under the guise of "improving satisfaction" and "better experience". Those things are incredibly important. But how will this feature improve satisfaction or create a better experience? How do we know for sure? How will we measure it? Too often those questions went unanswered and we were left having certain decisions made for us by whoever could yell the loudest or get the most attention from higher-ups (not to say that isn't still an ongoing struggle, but let's save that for another post).

So establishing meaningful KPIs was crucial in beginning to wrangle stakeholders and partners into legitimate discussions around outcomes. If we believed a new feature would improve efficiency or satisfaction or outcomes, how were we going to measure? What did we expect and how could we validate it?

Performance indicators also tie the overall company strategy back to our products. At WGU, the mission is to provide affordable, competency-based education to underserved groups. The measures of that all revolve around student outcomes: making progress toward a degree, overall satisfaction, keeping students enrolled and engaged, etc. So the measures for products need to stay  focused on those overall goals and the university strategy.

The key performance indicators I created fall into four groups: student outcomes, student satisfaction, student issues, and staff efficiency (you can replace "student" with "customer" in your own KPIs if this is useful to you). In order for new features to pass muster, they had to move the needle in one or more of those areas.

For example, within student satisfaction, we measure that through a variety of surveys. Since my team's focus is on the assessment experience, and student's are given surveys after they take an online assessment, we are able to very accurately measure if changes we make impact their overall satisfaction. 

Another example is within the employee efficiency. We know very accurately the number of issues our support staff deal with and the time it takes to deal with them. So we can measure the decrease in issues as well as equate that to time spent troubleshooting and solving problems. If our features are truly impactful, we can measure the decrease in the number of issues and how many man-hours that saves.

Establishing the right KPIs and measuring against them is difficult and can be time-consuming. Sometimes it can be incredibly difficult to find the right measure. On my team, we deliver assessments to students and evaluate them. But I don't think the number of assessments we deliver is necessarily a good KPI. It is easy to measure, yes, but would delivering more assessments mean anything? Possibly, but because that would be an easy rabbit hole to go down without adding value, it's not currently one of our KPIs. 

Establishing the most important indicators up front is one of the keys to knowing what to work on and how well you've done solving underlying issues. It isn't easy. It takes time and effort. And patience. We may not always see immediate pay-offs. But having created the right KPIs initially, and staying focused on measuring and improving, we can avoid making a banquet of terrible food.