Job interviews are mostly nonsense. According to Ron Friedman, a psychologist and author of “The Best places to work”, 80% of people lie during their interviews — so if that's the case, the information you’re hearing is likely fiction, or at best, inspired by real-world events. On top of that, there are now huge volumes of research from academic circles, as well as popular journals like Harvard Business Review, INC.com, and Forbes — all pointing to a simple fact. Job interviews mean you hire the best actor on the day — not the best candidate.

So we don’t do them.

Instead, we audition product managers (and Designers too for that matter) in the same way actors are auditioned for a movie. In our specially designed auditions, product managers perform real-world exercises to assess how they think about problems in an environment where nobody from the company knows anything about their experience, their background, or their formal education.

This allows us to strip out as much bias from the room by modifying our entire recruitment approach — and so far it's had good results, especially when it comes to hiring more diverse candidates. Right now, roughly 70% of the team in our division are women, in an industry in which the numbers are essentially inverted.

Given I have stolen bits of this process from various organizations and added my own little parts along the way, I figured I’d share it in case you find it useful. The people who have been through it, however painful, have given me positive feedback, so I share this with the world in the hope it helps others trying to hire great people.

So, without further ado, this is how we are hiring the world's best product team.

The three phases of interviewing for Product Management Roles

There are three stages to interviewing for a product management role at Whispir, but the one I’ll cover in-depth here is stage 2 — the audition. The rest are self-explanatory but I’ve abbreviated them below just in case.

Stage 1. The Phone screening

We do a 30-minute phone or zoom screening which is just to work out things like organizational alignment (do they really want to work on what we’re doing), salary expectations, working rights, and just general housekeeping. We also explain the interview format, why we use it, what they should expect and how to prepare. We do ask one creative question on the phone (I'll explain this shortly), but our expectations of their answers are pretty simple at this stage.

Stage 2. The Audition

Next, we bring them into the office (or do it via zoom if we have to) and have them answer 3 simple questions — a creative question, a strategic question, and an analytical question. That’s the entire interview.

We allocate about 15 minutes per answer, then leave 15 minutes for just general Q&A where we talk about the role and mostly everyone just unwinds after a pretty intense 45 minutes. So all up, a 60 minute round trip.

We also look for two other attributes which we infer from the 3 primary questions. Are they persuasive and are they curious — But more on this later.

Stage 3. An executive interview

We also do a quick executive interview as it helps with buy-in for the role and generally gives the candidate a sense of what kind of stakeholders they might deal with in the organization. This interview is usually focused on company history, alignment, and how they might fit into the company in general, more than anything to do with product management or how to build software.

Q1. The creative question

The creative question is designed to test someone's lateral thinking and creative skills. When working on bold ideas I want people who can think from first principles and come up with ideas that others might not have thought about by using abstract thinking. While creativity is not a trait universally desired in product managers everywhere (common in designers, for sure), it is at Whispir, so we optimize for it.

Here are examples of creative questions we ask.

• Design me a fridge for a blind person (the one I ask most frequently to be honest, but after posting this article, I will likely never ask it again)
• Design a slack competitor for the military.
• Design a new type of exercise equipment for people under the ages of 15.
• Design a better way to clean teeth every day for less than $100 a year.

We tend to ask non-software questions as this fleshes out the process of creativity more than diving into the technology side of it, which we all have a habit of doing in this space.

We ask less ‘technology’ questions in the creative part of the interview and instead, ask candidates to design real-world objects as it highlights the creative process and keeps us away from technical rabbit holes.

What I found was if you ask questions like “design me a Tinder competitor” you will spend a lot of time, after initial discussions about users and research, talking about technology, design, and software, but if you abstract the ideas out and make them more general-purpose products, it’s easier to see the real thinking process work from the software stuff, which feels like a more authentic version of the way people think.

Within these 15 minutes, we coach people to follow the Circles framework — as it will structure their thinking and get them through the process quickly. To ensure things are consistent, we actually send a whole preparation guide that explains the format, but this article is essentially that.

How we use Circles to answer creative questions

Circles is a product framework we use to ideate on solutions within the teams — it’s a simple way to go from having ‘no idea’ of a solution to a fully formed recommendation and you can usually do it in 15 minutes with some practice. This interview pressurizes that activity. To do it well in an interview setting is quite stressful, but the interview is designed that way to see how candidates perform under pressure.

The creative question gets candidates to walk through all 7 phases quickly, logically, and step by step, ending up with a brilliant and well-thought-out idea.

Comprehend

Let's take the fridge for a blind person question as an example. You want to comprehend what the actual situation is. Your first stop should be to ask lots of clarifying questions. Good product managers always ask lots of questions (to the point of exhaustion, really) so spend at least a few minutes doing this. The biggest red flag in product management interviews is people who jump straight from ‘design me a fridge’ into solution mode.

If it were me, I’d ask things like;

• What kind of fridge? Glass front? Electric? How deep?
• How many do we need to design? Is it just one?
• Are we building it or contracting it out to a manufacturer?
• Do we have any goals or targets to hit? What does success look like?
• When will they need to be ready?
• Is there a price we have to keep to?
• What kind of blind? Color-blind? Are they blind from birth? Blind in both eyes?
• How cold does the fridge need to be? Is it just one temperature for the entire unit?
• What kind of food will be stored in it? Drinks too? If so, what kind?
• How big does it need to be?
• Does it need to work in different regions and accommodate different languages?
• What kind of power will be available?

Identify personas

Next, you want to get clarity on the person who is going to use the fridge. Who exactly are we designing this for? How old are they? What are they like? How do they spend their time? Just ask the interviewer to tell you about them. What you want is to build up a clear persona of who this person is and what makes them tick. In the end, you are designing for a specific person. The more narrow the persona the better the solution will likely be.

(Note. there are instances where you can design for more than one user persona but its a bit beyond this activity and requires more explanation)

Also, try and identify who you are in the case study. Are you a fridge-designing company? Are you just some solo entrepreneur? Knowing who you are in the equation might inform how you approach the design process. If you’re pretending to be a company you’ll likely have resources at your disposal. But if you’re a solo manufacturer you’d approach it wildly different.

Write all this down in a clear and simple way.

What does this person really need?

This is the part of the interview most people bleed too much time on. Step back and ask yourself what is this person likely to need. You won’t know them personally, obviously — but speculate on possible needs they might have. One way to do this is to write them up as a job-to-be-done the user is trying to accomplish. In the case of the fridge, the needs might be

• I need to find my food quickly
• I need to know how old my food is in the fridge
• I need to avoid knocking over anything in the fridge.
• I need to know if someone has moved my food in the fridge.

These are all very basic needs and if you quickly list them out, it will help with the next step of the process.

Cut through and prioritize

In the end, you can’t work on everything. There are no solutions in this game, only trade-offs. So you have to work out what needs you’ll focus on. To help, you can put all these needs (or jobs) on a JTBD map. (image below)

JTBD maps plot out how satisfied people are completing this task today along with how important the need is to the customer. What you want to focus on (for the sake of the exercise at least) is one or two jobs only, and usually, things that are highly important but also underserved in the market will give you a product people really want with lower acquisition friction. You rarely have to twist someone's arm if you can solve a problem for them nobody else can — especially if it’s really important.

List solutions

Now the fun part. Fire up the creative engine in your brain and come up with a bunch of solutions for the job at hand. Come up with a range of practical ones and some really out-there ones to demonstrate your lateral thinking ability.

Examples we hear a lot of in the fridge example is;

• Use braille
• Build a mobile app that guides you to the fridge
• Voice activation

Honestly, 90% of the answers fall into these 3 categories, but if you wanted real creative solutions, think way more broadly for extra points. Ask yourself, do you really need a fridge at all? What is the core need here? Is it a fridge, or is it a full belly?

For instance…

• Why do you even need a fridge, why don’t we start a fresh food program for kids where refrigeration is not required and instead we cook everything fresh, and we back the idea as a way to create empathy with communities that lack power in third world countries — which aligns to the purpose of the company. So it’s both a social awareness idea and one that gets fresher food into people's diets. And then we just hand-deliver the food to children as they sit down to eat every day.

Trade-offs

In the end, think critically about your ideas. You want to show you can be commercial when you need to be. Some of your wacky ideas will be too strange or infeasible to build in time or cost-effectively, so talk through the pros and cons of each idea.

An impact vs complexity matrix is the most common way we discuss potential options with teams to make decisions on what to build.

One good way of visualizing this is to use an impact vs complexity matrix. It allows you to orbit your ideas towards impacting a certain objective, which shows you are working towards company goals.

Recommend

In the end, you have to decide and back one idea. Product management is all about making hard choices. Summerise it, why you should do x and be as clear and concise as possible. If you were really pro you could put it all on a lean canvas but that would exceed 15 minutes unless you are really fast. But it's a slick move if you can do it.

Q2. The Strategic questiton

Our strategic questions test two things. How well the candidate understands our mission, our values, and our goals, and how well their ideas align with them. What you don’t want is product teams building stuff they want just because it sounds cool or sparks their interest. All ideas should support the empire.

Examples of strategy questions we ask are things like

• Should Whispir build a native mobile app?
• Should we be concerned about OTT (Over The Top) messaging channels like WhatsApp, WeChat, or Viber?
• Will RCS be something we should invest in?
• What would be our biggest threat in the next 10 years?

To answer this question well, if you don’t know the specifics of the strategy, just ask. The strategic question tends to be more conversational than the creative question. I find the strategic questions the candidate will begin quizzing the company and then asking challenging questions most of the time.

There are no right or wrong answers here and typically we ask questions that are real-life conundrums we face. This way, we have authentic back-and-forth dialogue about a strategic problem we might have going on. As a candidate, you want to think like a management consultant here. How would you help an organization that is at a crossroads of unlimited choices and help them make a key strategic decision?

If you know good product strategy frameworks that can help, walk the interviewer through it. There are too many to list but as an example, we used the Blue Ocean Strategy to work out the product strategy at Whispir, and there are some simple exercises to do that can help frame the conversation.

Q3. The analytical question

Frankly this is is the painful one — and also the one I am not great at myself. The analytical question is essentially a market sizing problem. It is designed to help product managers quickly work out the market size of an idea and calculate large numbers quickly.

Real suggestions come up every day from the business and customers. A stakeholder might come to you and say we really, really need a Slack integration. Sure. Maybe that's a good idea. Market sizing exercises allow you to, in that very moment with the stakeholder, sketch out rough market sizes for the idea using back-of-the-envelope fermi estimations. It’s a quick technique that allows you to see if the market is big enough to work on.

Let’s say the analytical question is as follows

• How much money would we make from launching WeChat as a messaging channel?

This example is actually a question I ask a bit, and the way to answer it goes something like this. You break down the question into more questions first, and using a mindmap is a great way to keep your brain together. This can get really out of hand fast, so staying calm and logical is key to getting to the right answer.

The key here is not perfect math, but it is someone who covers the edge cases and doesn’t leave a lot of unknowns on the table. Try to think of everything you can. You want to get all the way to the end, then gut-check the math. If it feels right, go with it. So this is how I’d answer the question about WeChat

• How many WeChat users are there today in the world?
• How many of those users are likely to use Whispir?
• Within that cohort, what do we expect churn to be?
• How long will it take to build this?
• How quickly will users ramp up?
• How many leads will we need and can we drive that?
• What will the funnel conversion look like?
• How much can we charge?
• How much will it cost to support?

Frankly, you can keep going but you get the idea. Then with each question, you want to break it down even further. Let’s take the how much can we charge question. I’d break this down by

• Will different people in cohorts want different pricing buckets? ie, should we have pricing plans?
• What are the most we can charge while people remain indifferent to the price?
• Should we plan on increasing the price over time or keep it the same?
• Do we care about profit for now?

If you unpack this, you’ll end up with maybe 20–30 questions in 2 or three layers. Then, just quickly go through and speculate on answers. Finally, add it up, and gut-check it.

What I’ve noticed is that candidates with engineering backgrounds or programmers who have moved into product do this really well. I think the task is well suited to more logical / math style thinking patterns.

But it’s also not a useless exercise day today. Frankly, I use these estimations all day with senior executives who are asking me to build all manner of things that are shiny and new. Getting good at fermi estimations means you can leave the conversations in the room, instead of having to turn them into full-on projects for the teams only to uncover the solutions are just not viable enough for the business.

How we score candidates

We typically look for candidates who exhibit 5 traits,

Creative, analytical, strategic, persuasive, and curious. So we write up those on a quick table and ask interviewers to score the candidate on 4 dimensions

• Hire them
• Maybe hire them (This demonstrates they fear they will potentially make a mistake if they pass)
• Probably don't hire them (This demonstrates they fear they will potentially make a mistake if they offer a job)
• Don’t hire them.

Now the curious and persuasive dimensions come out during the first three questions, so while you’re assessing the candidate on creative content, you are trying to work out how persuasive and curious they are too.

I give these sheets to the interviewers and ask them not to share what scores they give with each other. They send them to me, then we compare scores after. What we’re looking for is consistent scores in a positive direction. Getting lots of ‘maybe hires’ is fine, and some interviewers will be tough critics but if the numbers are all over the place, something has gone wrong.

What I like about this process is when the interviews happen, we don’t tell the interviewers who the person is. We don’t say where they’ve worked, what they’ve done, or anything like that. It’s a blind date in many ways. So the person comes in, does their thing, and gets judged on what happens in the room.

What's fascinating about this whole thing is that while it's really stressful, candidates walk away feeling like they’ve been evaluated fairly and often tell me how great the process was.

Written by Brad Dunn