Happy Friday,

Earlier this week, a friend showed me a video in which a CIO explained how they view setting up analytics for an organization. In a nutshell, they said that while people mostly understand it as five high-level steps, he cautioned that creating analytics should be treated more like a process problem.

Treating it like a process problem means that you need to understand existing business processes, work with folks to modify them to incorporate the required data collection, and, most importantly, examine the analytics to make new decisions.

While I agree with this entirely, I wanted to raise two additional points that will help with any analytics or reporting effort.

Asking the Right Question

The video's description of the 5 basic steps are:

1. Understand problem/conext
2. Data collection
3. Data analysis
4. Visualization
5. Support/Maintain

I want to focus on steps two and three. Now, the person in the video cautions that a circular loop can happen with steps two and three as you collect data, analyze, refine, and collect again. It can be hard to know when you're done.

Now, mentally, I reverse these steps and say you start with analysis and then collect data. It sounds strange, but let me explain.

When folks ask for reports or analytics, there is a question I ask, which is, "What would you need to see that would make you stop, change course, or double down?"

This question is about taking the idea of analytics being good to a place of knowing what specific indicator is needed to prompt action. Far too often, dashboards like this are a swath of data that sparks debate but not action. That's because nobody knew what signals were important, so building analytics dashboards becomes an "I'll know it when I see it" problem.

By asking the question above, we wind up with a clearer idea of exactly what we need in our dashboard, which informs what data we need to collect. In my experience taking this approach, I'll also say that you almost always wind up with just a few simple metrics instead of a broad selection of non-impactful numbers.

Weaponized Reporting

The next topic I wanted to address is the actual use of analytics. I've only ever seen two uses for reports and analytics:

• Guiding group decisions
• Convincing an individual

I want to point out the last case. This case often comes in the form of a request for data so they can show something. Now, if you probe further and use the question I mentioned, you'll tease out that the requester needs to convince people of something, and they want data to be their smoking gun.

If you think this is the case, you can ask, "So, if the data disagrees with you, will you abandon your recommendation?"

While I recommend asking this question, I can tell you now the answer is "No."

So here's the situation: You're going to spend time getting data for no reason. Even if the data agrees with your requester's position, the person they want to use the report on likely doesn't care.

I view these reports as not a tool to aid decision-making but a weapon to control it. They are best avoided.

Now, As you ask questions about what you'd need to see to change course and whatnot, another thing to pay attention to is who you have this conversation with. Let's say it's a product group. I would recommend having this conversation with everyone in the product group at once so the group reaches an agreement about how these analytics and reports fit into their decision-making. This helps solve the process and weaponized problem.

If you go to each person individually, you'll have a hard time coming up with analytics that work for the group instead of an individual and run the risk of giving someone a weapon.

Here's my weekly update for March 8th, 2024...

🗒️ Superstitions and Piseogs

I bet most of you are wondering what the word piseog is and it is pretty tricky to look up.

I first encountered it in one of my favorite books, “The Once and Future King” by T. H. White.

If you’ve ever watched “The Sword in the Stone” animated movie, you’ve seen the book’s first quarter.

It’s a wonderful and tragic telling of the story of King Arthur.

Click here to read more

🗒️ Poorly Built Racecars Don’t Win

Imagine you are setting off to win an ambitious car race.

You assemble a team of folks with plenty of experience and lay out a plan.

Click here to read more

🗒️ Permission to Choose

“You’re going to what?!” the director asks me.

Click here to read more

Enjoy,

Ryan Latta

PS: Want to talk through what challenges you're having in your organization or with your development teams? Schedule a complimentary call!