Our minds were young and fresh not that long ago. Sure maybe we were a little naive, but that is because we looked at the world with wide, trusting eyes. And then it happened. We could no longer trust what we saw and lines were drawn between viewpoints that separated brother from sister. It was… The Dress.
Was it blue and black? Or white and gold? For a few weeks, the internet boiled with heated discussions and color / lens filter analysis. Finally resolved, the world began to repair the bridges burned. Until tragedy struck again weeks ago. I present to you… The Jacket.
Blue? White? Brown? Black? We’re still waiting for the first districts to report their votes on this one.
This is a pitfall can trip-up even the experienced innovator at two crucial waypoints.
1. Understanding the Customer and Pain Points.
When researching and listening for pain points, they can often go misinterpreted. It is most common when customer empathy has not been explored enough. The famous quote attributed to Henry Ford applies here.
“If I had asked people what they wanted, they would have said faster horses.”
If you’re only listening to what a customer wants, you’re missing 75% of the picture. You’ve got to observe them, understand how they feel, and try to get inside their head.
A great way to do that is to journey map while interviewing a customer. Ask them to describe the entire pain point; from the earliest decision to well after the pain point. Pay attention to their tone of voice, the words they use, and their gestures. Ask many questions to get at the root cause of the pain point. This often shows possible growth opportunities as well.
2. Analyzing Tested Prototypes
There is nothing more frustrating than looking at test data (for example: 56% conversions) and then looking at yourself and saying “Is that… Is that good?” What you need is a solid feedback loop, and this is something games do very well.
Which final results screen would you rather see?
The one on the left tells you that you won, you completed all the parts, and you should move forward. The one on the right tells you… that progress was made? There is one star, but the empty space makes it feel like there could be more? How many parts did you complete?
You can set your own prototype test to give you all the feedback you need. Before the test ever begins, set the success metrics that will define your test and your prototype. It is beneficial to know the benchmark metrics that you are trying to surpass (if they are available) and how much difference you are trying to make. Use this data to plan for enough testers to be confident in your results. After the test, you will have a clear understanding of your outcome, and whether to pivot or persevere.
Don’t get caught wondering if your prototype was golden, or left you black and blue. Use journey mapping and success metrics to know what your seeing.
I enjoy a good scary movie from time to time and zombies are everywhere. Zombies cause a load of difficult situations for folks. Imagine that you’ve just pronounced Steve* deceased, when all of a sudden, Steve sits upright and starts expressing his sudden love of brains. “My bad everyone” you might say sheepishly “I could’ve sworn he was a goner.” Then everyone rolls their eyes at you and it just gets super awkward. If only you had done a little more checking before, you might have a plan for when your “deceased” diagnosis failed.
Whether it is zombies or innovation prototyping, you need to do a pre-mortem.
The whole point of building prototypes and testing them with customers, is to see if you solution succeeds or fails. Yet we often wait to get our “failure folder” before we start thinking of why it went wrong and where the pivot opportunities are. There is no need to wait to make decisions when you conduct a pre-mortem.
Pre-mortems occur after your prototype is ready for testing, but before you do any actual testing. They allow you to take a glance at your work from a different angle. To this point, you’ve spent your brain power finding ways to get the prototype to work, how to demonstrate your hypothesis, and how to collect the data points you know are important. A pre-mortem let’s you stop those thoughts, and think “How can this prototype fail during the test and what am I going to do after it does?”
Pre-mortems focus your thoughts of failure to a single test of your prototype, not your end product. By keeping the scope of your vision on just one test, it is way less scary to envision the many ways it can fail.
A simple pre-mortem can be done by yourself as long as your prototype is in its testable state. However, it is very beneficial to have other people look at your prototype during the pre-mortem. These people are not your testers and they may not even be your target customer. Your pre-mortem pals are chosen because they have candor (which means that if it stinks, they’ll tell you… to your face… with no hesitation.) This group isn’t hyper-critical necessarily. They want to see you succeed, so they won’t inflate your hope with false niceties to avoid hurting your feelings.
If a person will tell you that your shirt is ugly, sign them up for your pre-mortem crew.
Executing a pre-mortem can be an informal process, however it is possible to ruin the validity of the pre-mortem. Be cautious to not “lead your witness” by building a worldview where your prototype is the only solution. You are best off not even telling them what your solution is or how it works. It is natural for us to go into pitch mode when we have someone looking at our prototype, but you wont be at the point of sale for your product every time. They will have to use your product by themselves, and “get it”.
Pre-mortems allow you to test your prototype, without any guiding, to see if it stands up on its own. A pre-mortem is like the first few shaky steps of a gangly giraffe; it looks like it will topple at any moment and you want to run over and prop it up, but you shouldn’t interfere if you want it to be running across the savannah someday.
To avoid the “let me just show you how this amazing product works” scenario, I’ve crafted a guide for you. This guide can be used whether you are “pre-morteming” by yourself, or with your group (the ones who are ok saying they don’t like your haircut).
Refer to your customer profile so that you or your group can get in the customer’s mindset. Make sure to explain the pain points you identified for the primary persona.
Let your pre-mortem pals interact with your prototype while you sit silently.
Observe and document how they interact.
If you are doing this step virtually, have them write down or say everything that comes to mind including “Ok now I am clicking this button because it looks like it needs to be clicked… and now I am going to do this other thing”.
Pretend to be in the future and your prototype has just failed in its most recent test.
NO REASON FOR FAILURE IS TO BE GIVEN YET.
Have everyone participating in the pre-mortem write down how your prototype failed. Aim for lists with multiple failure options.
Share and evaluate the responses. If you are doing this in a group, one person may see the prototype failing one way, and that may spark discussion on other ways no one had thought of.
I will warn you that this is a tough conversation to be a part of. However, you must encourage the dialogue. “This is great! Tell me more about how my prototype will fall on it’s face during the test!”
Take your list of possible failures and start thinking of ways you can pivot.
Take no action now, unless a failure is eminent and will ruin the test. If that’s the case, you have to fix that! Otherwise, the list of failures is not guaranteed. Keep in mind that your pre-mortem pals may not be representative of your testing group.
If you are pre-morteming by yourself, you can skip steps 2 and 3. You should probably also skip any of the open discussion parts unless you want to talk to yourself out loud. And that’s totally fine if you do.
I enjoy pre-mortems. They teach you in so many ways. They serve as mini tests before your real test. They open up your eyes so that you can see your prototype from the customer’s point of view. They help you identify ways your prototype will stumble during the test. They let you plan pivots ahead of time, that you can act on later if your prototype does actually fail. I don’t dread testing prototypes because I know if it fails, I have a plan or two ready to go.
I challenge you to be vulnerable and find some folks who will look at your prototype with a critical eye. Plan for failure now instead of later.
*You may or may not know a Steve but I assure you that the names have been made up and any likeness to someone you know is purely coincidental.