Empathy, Innovation, Learning, Pre-Mortem, Testing

Knowing What You’re Seeing

Our minds were young and fresh not that long ago. Sure maybe we were a little naive, but that is because we looked at the world with wide, trusting eyes. And then it happened. We could no longer trust what we saw and lines were drawn between viewpoints that separated brother from sister. It was… The Dress.

theDress

Was it blue and black? Or white and gold? For a few weeks, the internet boiled with heated discussions and color / lens filter analysis. Finally resolved, the world began to repair the bridges burned. Until tragedy struck again weeks ago. I present to you… The Jacket.

theJacket

Blue? White? Brown? Black? We’re still waiting for the first districts to report their votes on this one.

This is a pitfall can trip-up even the experienced innovator at two crucial waypoints.

1. Understanding the Customer and Pain Points.

When researching and listening for pain points, they can often go misinterpreted. It is most common when customer empathy has not been explored enough. The famous quote attributed to Henry Ford applies here.

“If I had asked people what they wanted, they would have said faster horses.”

If you’re only listening to what a customer wants, you’re missing 75% of the picture. You’ve got to observe them, understand how they feel, and try to get inside their head.

A great way to do that is to journey map while interviewing a customer. Ask them to describe the entire pain point; from the earliest decision to well after the pain point. Pay attention to their tone of voice, the words they use, and their gestures. Ask many questions to get at the root cause of the pain point. This often shows possible growth opportunities as well.

2. Analyzing Tested Prototypes

There is nothing more frustrating than looking at test data (for example: 56% conversions) and then looking at yourself and saying “Is that… Is that good?” What you need is a solid feedback loop, and this is something games do very well.

Which final results screen would you rather see?

The one on the left tells you that you won, you completed all the parts, and you should move forward. The one on the right tells you… that progress was made? There is one star, but the empty space makes it feel like there could be more? How many parts did you complete?

You can set your own prototype test to give you all the feedback you need. Before the test illusion (1)ever begins, set the success metrics that will define your test and your prototype. It is beneficial to know the benchmark metrics that you are trying to surpass (if they are available) and how much difference you are trying to make. Use this data to plan for enough testers to be confident in your results. After the test, you will have a clear understanding of your outcome, and whether to pivot or persevere.

Don’t get caught wondering if your prototype was golden, or left you black and blue. Use journey mapping and success metrics to know what your seeing.

 

 

Being Awesome, Brainstorming, Diffusion of Innovation, Ideation, Innovation, Understanding the Customer

Innovating with the Uninterested

My kids send me strong signals all the time. For example, when we have broccoli or sweet potatoes, they respond with very strong signals. Unfortunately their signals are strong AND negative. One way we’ve tried limit these anti-veggie reactions is to get them involved in the meal planning.

Like in meal planning, we should be looking for strong signals with prototype tests. Strong signals validate that solutions are worthy of digging into deeper. Sometimes, you will get responses of “I don’t like this” or “I’m not sure this will work.” These are great strong signals, just not the ones you may have been looking for. Their value though can be immense.

diffusionOfInnovationLooking at the Diffusion of Innovation, these types of strong signals would be achieved from folks in the “Late majority” category. That accounts for 34% of the market, and yet we design by relying on “Early Adopters” or the “Early Majority”. How can we move their timeline of adoption up? How can we use their strong signals, and their personas, to help make our prototype better?

We can design with them!

Wrangle up some of those “This will never work” naysayers and get them in an ideation session. You can often get them to agree by just being honest. No need for trickery or bribes. Just tell them that you’re sorry your one idea didn’t fit for them but you’d like to understand their view better.

“Great! Now I have all the people who hated my prototype in one room. What do I do now?”

It is simple, just understand these three guidelines.
  1. Let the customer drive the conversation
    Strong signals, like kids rebelling over the inclusion of broccoli, can indicate the presence of the "Late Majority". Instead of taking a hit to your momentum, use their energy to design a new, better solution!
    Strong signals, like kids rebelling over the inclusion of broccoli, can indicate the presence of the “Late Majority”. Instead of taking a hit to your momentum, use their energy to design a new, better solution!
    • You must aim for a 80-20 ratio of listening to talking.
    • Listen to understand, not to react.
      • This is a personal pet peeve, but too often we listen with the intent of reacting to what someone says. Especially here where you’ve already show the customer a solution. They will say “Well I need it to do X.” and you’ll want to say “Well what I showed you will already do X, you may have missed it.” AVOID THIS! Internalize that thought but come back to them with something like “Interesting. When it does X, what does that look like to you?”
    • Reiterate what they say if you are unclear.
      • Remove any uncertainty.
    • You may need to set up the ideation session with some easy wins up front to grease the gears of innovation.
  2. Keep their options limited
    • Too many options and they will freeze up. It’s a cousin of the “blank page” syndrome.
    • Constraints can also help people be creative.
      • It’s like squeezing a tube of toothpaste. Applying pressure, with the constraints of the rest of the tube, pushes the paste onto the brush.
  3. Nothing needs to be pretty
    • An “ugly” prototype or a napkin sketch keeps the customer from thinking the idea is set in stone.
    • Thinking everything is “up for change” frees them up to make more suggestions.
    • Encourage them to take a crack at some wireframes.
      • You will hear “I’m not good at drawing” but we’re not looking for art here, just ideas. You can offer to draw for them if this is a sticking point.

I’m not saying you’ll end this process with a market-capturing design. But you will have a better understanding of the needs, the pain points, and the potential gains for your innovation with the Late Majority. Imagine if you can inspire the Late Majority to adopt sooner!

Failure, Going Forth, Innovation

Vanity Metrics & Bathrooms: It’s not the size of your mirror.

Numbers never lie. However numbers can be misleading, even when they don’t mean to be.

One thing that my good friend Joe Greaser has often said “If someone is bragging about their data, it is probably a bunch of vanity metrics.”

Eric Reis (Lean Startup) brought the term “vanity metric” to light. They are the metrics that sound really good, but in the end don’t amount to much. In contrast to vanity metrics are the actionable metrics. These metrics are tied to specific actions or tests and inform you on what to do with them. “My app has had 50,000 downloads.” Vanity metric. “I tested two versions of my app and customers purchased 40% more through the B version.” Actionable metric.

Another one of my good friends, Mike Jarrell, was going to a sporting event where they were promoting stadium upgrades. “87% more female restroom facilities and 20% more male facilities!” At first glance you say “Wow! They really understand that lines for the women’s restroom at sporting events can be painfully long.

But are we falling into the trap of vanity metrics?

Let’s take this to absurdity to prove a point. Assume that Innovation Stadium has 5 female restrooms and 20 male restrooms. Pretty unlikely and unreasonable, I know… but we’re imagining the absurd here. Stick with me. Now let’s agree that Innovation Stadium added 5 more female restrooms and 5 male restrooms. The grand totals are still unbalanced at 10 female restrooms and 25 male restrooms. And yet Innovation Stadium can claim “We’ve added 100% more female restroom facilities and 25% more male facilities.” Sound familiar? Not saying that’s what happened, but sometimes you pick the numbers you want to market.

What sounds better? 5 more female restrooms or 100% more?

Recently, TechCrunch posted about the math behind startup valuations. This is an amazing article, especially for a mathy like me. And I have no doubt that these calculations are reliable within certain parameters. I want to point out my one concern. The article says that without enough customers, a startup has to use estimations. Believe me, I get it. You have to estimate sometimes, especially in a new business. However, if any startup bragged about data calculated from estimates, then they too have been wooed by vanity metrics.

It reminds me about how quark-sized some baseball statistics have become. Almost to the point where a batter can walk to the plate while the announcer tells you his batting average in the month of August, with runners in scoring position, against pitchers with a weekend birthday this year, when they’ve gotten a call from their mom before the game.

Vanity metrics may look good, but they lure you into dangerous assumptions of success.
Vanity metrics may look good, but they lure you into dangerous assumptions of success.

In innovation, we don’t have the kind of time baseball has. They have a 162-game season to see their statistics play out. In innovation, you’re lucky if you have a couple weeks. Build, test, measure, learn… sprint, sprint, sprint. So how do we avoid vanity metrics?

  1. Establish what hypothesis you are testing, before you test.
  2. Identify what metric, or metrics, would absolutely validate your hypothesis.
  3. Now you can test your prototype.
  4. Collect, reflect, observe, and analyze.
  5. Be ok with failing forward.

By setting the hypothesis and success metrics before you test will prevent you from latching on to bright spots. Also, by being alright with failing (as long as you are failing forward) then you feel less pressure for each test to be successful or be validated. It is a tough practice, because you’d like your idea to be a winner, but this is all part of the process in finding the right solution. Remember, you’re refining an idea that will work, not just pushing your favorite to the finish line. So be modest, avoid the vanity metrics, and keep it all actionable.

UPDATE!!!

So here’s something pretty applicable. Early today, another good friend of mine, Adrienne Campbell, and I had a great conversation about this post. I am lucky to have many good friends who:

  1. are deep thinkers
  2. challenge me and make me think deeper

We were talking about vanity metrics and if there really was a good use for them. Maybe they don’t need to be avoided at all costs. Perhaps they could provide some value.

Take a look at this example.

A vanity metric for blogs would be the number of views. This is a great statistic if you are looking for overall exposure and reach. “Should we promote on Blog X? How many views do they get?” A great actionable metric would be “How many excellent conversations came out of a post?” (that’s one from today!) or “When I write about Subject A, how many views do I get compared to when I write about Subject B?” I get more views in WordPress when I write about writing. I get more retweets when I write about innovation processes. This is data I can act on.

Something Adrienne brought up was (and I’m paraphrasing) “Different metrics are appropriate depending on the purpose.”

  • Want to know if your prototype is working? You need to locate actionable metrics to test.
  • Want to promote your solution to an outsider? There may be some vanity metrics that get the conversation going.

It brings it all back to the stadium restroom example. They probably justified the construction based on actionable metrics such as length of wait and restrooms per person, but they promoted vanity metrics by promoting the percentage increase.

That means both can be a welcome tool in your innovative tool box. You just have to know when to use which one. ~GFandBA

Going Forth, Innovation, Persona, Pre-Mortem, Testing

I’m not dead yet!

I enjoy a good scary movie from time to time and zombies are everywhere. Zombies cause a load of difficult situations for folks. Imagine that you’ve just pronounced Steve* deceased, when all of a sudden, Steve sits upright and starts expressing his sudden love of brains. “My bad everyone” you might say sheepishly “I could’ve sworn he was a goner.” Then everyone rolls their eyes at you and it just gets super awkward. If only you had done a little more checking before, you might have a plan for when your “deceased” diagnosis failed.

Whether it is zombies or innovation prototyping, you need to do a pre-mortem.

The whole point of building prototypes and testing them with customers, is to see if you solution succeeds or fails. Yet we often wait to get our “failure folder” before we start thinking of why it went wrong and where the pivot opportunities are. There is no need to wait to make decisions when you conduct a pre-mortem.

Pre-mortems can help you plan if your prototype fails and comes after your brains.
Pre-mortems can help you plan if your prototype fails and comes after your brains.

Pre-mortems occur after your prototype is ready for testing, but before you do any actual testing. They allow you to take a glance at your work from a different angle. To this point, you’ve spent your brain power finding ways to get the prototype to work, how to demonstrate your hypothesis, and how to collect the data points you know are important. A pre-mortem let’s you stop those thoughts, and think “How can this prototype fail during the test and what am I going to do after it does?”

Pre-mortems focus your thoughts of failure to a single test of your prototype, not your end product. By keeping the scope of your vision on just one test, it is way less scary to envision the many ways it can fail.

A simple pre-mortem can be done by yourself as long as your prototype is in its testable state. However, it is very beneficial to have other people look at your prototype during the pre-mortem. These people are not your testers and they may not even be your target customer. Your pre-mortem pals are chosen because they have candor (which means that if it stinks, they’ll tell you… to your face… with no hesitation.) This group isn’t hyper-critical necessarily. They want to see you succeed, so they won’t inflate your hope with false niceties to avoid hurting your feelings.

If a person will tell you that your shirt is ugly, sign them up for your pre-mortem crew.

Executing a pre-mortem can be an informal process, however it is possible to ruin the validity of the pre-mortem. Be cautious to not “lead your witness” by building a worldview where your prototype is the only solution. You are best off not even telling them what your solution is or how it works. It is natural for us to go into pitch mode when we have someone looking at our prototype, but you wont be at the point of sale for your product every time. They will have to use your product by themselves, and “get it”.

Pre-mortems allow you to test your prototype, without any guiding, to see if it stands up on its own. A pre-mortem is like the first few shaky steps of a gangly giraffe; it looks like it will topple at any moment and you want to run over and prop it up, but you shouldn’t interfere if you want it to be running across the savannah someday.

To avoid the “let me just show you how this amazing product works” scenario, I’ve crafted a guide for you. This guide can be used whether you are “pre-morteming” by yourself, or with your group (the ones who are ok saying they don’t like your haircut).

THE GUIDE

  1. Refer to your customer profile so that you or your group can get in the customer’s mindset. Make sure to explain the pain points you identified for the primary persona.
  2. Let your pre-mortem pals interact with your prototype while you sit silently.
  3. Observe and document how they interact.
    • If you are doing this step virtually, have them write down or say everything that comes to mind including “Ok now I am clicking this button because it looks like it needs to be clicked… and now I am going to do this other thing”.
  4. Pretend to be in the future and your prototype has just failed in its most recent test.
    • NO REASON FOR FAILURE IS TO BE GIVEN YET.
  5. Have everyone participating in the pre-mortem write down how your prototype failed. Aim for lists with multiple failure options.
  6. Share and evaluate the responses. If you are doing this in a group, one person may see the prototype failing one way, and that may spark discussion on other ways no one had thought of.
    • I will warn you that this is a tough conversation to be a part of. However, you must encourage the dialogue. “This is great! Tell me more about how my prototype will fall on it’s face during the test!”
  7. Take your list of possible failures and start thinking of ways you can pivot.
    • Take no action now, unless a failure is eminent and will ruin the test. If that’s the case, you have to fix that! Otherwise, the list of failures is not guaranteed. Keep in mind that your pre-mortem pals may not be representative of your testing group.

If you are pre-morteming by yourself, you can skip steps 2 and 3. You should probably also skip any of the open discussion parts unless you want to talk to yourself out loud. And that’s totally fine if you do.

I enjoy pre-mortems. They teach you in so many ways. They serve as mini tests before your real test. They open up your eyes so that you can see your prototype from the customer’s point of view. They help you identify ways your prototype will stumble during the test. They let you plan pivots ahead of time, that you can act on later if your prototype does actually fail. I don’t dread testing prototypes because I know if it fails, I have a plan or two ready to go.

I challenge you to be vulnerable and find some folks who will look at your prototype with a critical eye. Plan for failure now instead of later.

*You may or may not know a Steve but I assure you that the names have been made up and any likeness to someone you know is purely coincidental.