So, what have you tried?

A simple question that works well across user research, Stack Overflow, and leadership hiring.

User research #

When I first started doing user interviews at Microsoft, my mentors reminded me how critical it was to validate that the user’s problem exists and is important to the user. There are too many problems that can exist and do exist, but are not important enough to be solved.

Microsoft is big on codifying such lessons in language, and this was labelled “dents vs. rats”. We are annoyed about the dents on our cars, but we let them be. When we find rats in our houses, we rush to action.

To differentiate rats from dents, product managers must look for the user’s behaviour when faced with the problem. Ask not just whether the problem exists—ask what have they done about it?

This was unintuitive to me. I mean, the solution to their problem was with “me”, and I hadn’t shared it with them yet. What would they even do about their problem given that they did not have a solution?

But, as it turns out, users can do a lot about their problems, even when they don’t have “my solution” (which I know is the best solution!). They can search for solutions online, they can hire people instead of products, they can kludge different products together and incur a cost (time, money) while solving the problem.

When we test for whether a problem is important, we’re testing for whether the user has high intent to solve that problem. We’re doing “intent validation” — which complements “problem validation” (problem exists) and “market validation” (market exists).

Stack Overflow #

If you, like me, frequented Stack Overflow back in the day, you would have run into questions that had zero answers, but instead, had comments that said “what have you tried?”. This comment was even called an “epidemic” and ended up being banned.

Many interpreted the comment as rude and pointless, but I think it served a genuine purpose. It made Stack Overflow work.

Stack Overflow visitors have high intent to solve a burning, specific question that is blocking them at work. It works when an existing question matches the visitor’s question and has answers under it.

This is possible not only because Stack Overflow is full of great answers, but it’s also full of great questions. This is where “what have you tried” helps.

It’s easy to post a broad question (e.g. “how to make API calls”) but without additional context the question has little utility for other visitors. When you expect a “what have you tried” comment, it becomes a nudge to frame better questions: you add more context, share error messages and code snippets. In turn, you make the future visitor feel “yes, this is exactly what I need!”

Leadership hiring #

Most recently, I’ve seen intent validation become useful while interviewing for senior roles at startups. Given the fluidity at startups and the competitive talent market, startups have a tendency to “create roles” that don't exist in their hiring plans for candidates they like.

Testing for the intent behind the newly created role is critical. “Sure, you want to create a role to drive PLG motion, but what have you tried so far?” is a helpful test to evaluate the depth of intent behind wanting a PLG motion.

In the heady moments of hiring-dating, both sides can irrationally optimise towards offer closure. This test eliminates the risk for when the role is optimising for the candidate’s interests and short-term offer closure, instead of optimising for the organisation and long term success. It can also predict whether you will be collaborating or fighting with the organisation while driving your new charter.