User Interviews Considered Harmful
I have now been building products from zero to one for the last four years: some commercially successful, some not. Along the way, I have often been given the advice that it’s important to talk to users before you build, to help ensure that you build something that people actually want.
Four years later, I believe this conventional wisdom does more harm than good. The big problem is that users frequently are not truthful about their wants and needs: whether intentionally or not, they tend to state false preferences. In this blogpost, I want to share some of the failure patterns that I have seen along the way.
Lying About Efficiency
It’s a common pitch to build tools that make people’s lives or workplaces more efficient. In B2B, these are especially common: companies with slow processes want to invest in software to make them faster.
However, unless growth has slowed to a halt, companies are usually better off focusing on growth rather than process-optimizing to shave off a few hours or dollars here or there. I’ve seen many ideas that seem neat and clean, but are too marginal1 to ever win adoption. Usually these are about replacing clunky communications:
Streamline your Figma design reviews, away from email and Slack into a single feed;
Automate legal workflows and case management for law firms, financial advisors, etc.;
Combinations of tools like Airtable, PDFs, forms, and DocuSign into one interface/automation to save an intermediate email step;
The first-glance pitch is always that software can save time and money and reallocate resources to more important work, especially in service-oriented businesses.
But the hard truth is that most people are not paid to be efficient. Their usual day consists of work that is pretty easy, spaced out in a generous way. If they’re much more efficient and they move faster, their day actually gets kind of worse because now they’re tackling harder work for the same amount of time. They have no incentive to move faster. Maybe it’s even dangerous for them to move faster, because that suggests their previous labor was not as useful or valuable.
However, people are incentivized to say that they want to be efficient. This is just the correct social signaling. If you spend your time talking to users dealing with a lot of paperwork: lawyers, financial advisors, data engineers, etc. they will all say they want to be more efficient. But in practice, they often don’t. Efficiency is competition for them, and they will avoid it.
No Attempt to Solve the Problem
My most recent endeavor was building an email inbox for shopping. The app would connect with your Gmail, pull out your purchases, sales, coupons, etc., and then group and filter them for you. Speaking to users had given me a simple thesis:
People are overwhelmed by cluttered inboxes;
People are annoyed with the quantity of shopping emails they get; and
People struggle to find their purchases/tracking numbers, and miss out on sales and coupons.
Our app solved these problems. After launching and marketing it, we saw a flurry of downloads, but the retention was weak. Reflecting on my user interviews, I realized: almost none of the users had made a real effort to use Gmail’s filters. Nor did they systematically unsubscribe from promotions. Their inboxes were cluttered, and they were frustrated by that, but at the end of the day, they muddled through.
The problem was visible enough for users to recognize it and complain about it when prompted. However, it was not urgent enough for them to ever even attempt solving it on their own, which indicated that a solution to this problem would not actually be very valuable. This is a dangerous middle ground: it’s certainly not a need, and only a weak want.
Excuses for Laziness
Many startups have made a pitch that starts like this: “Business Intelligence tooling is broken.” For example, marketers and other non-technical staff are often frustrated with their company’s analytics. The analytics exist in some set of dashboards and tools, and they don’t know how to use that tool, or don’t have the time to look through all the dashboards. They have to wait on data analysts to provide the reports. If only there were an easy-to-use analytics platform that the marketer could use to run all their very important queries in one place to better understand product usage!
But even if they had this tool, the data engineer would still be doing the queries.
The reason is that the business almost always already has some analytics tooling: maybe it’s in Tableau, or Looker, or it requires raw SQL queries. But the business has something, and it’s telling that the user has chosen not to learn how to use the tooling, and is instead asking me for a new solution2. How important can the task be if they haven’t bothered spending four hours trying to pick up the tool?3
The takeaway here is very similar to the story about Gmail clients: people imagine that they need new tools when they’ve actually been too lazy to use the normal tool, which would totally get the job done. If they’ve been able to muddle through the problem despite all this laziness, then it’s probably not a very urgent or valuable problem to solve. The user doesn’t truly care, otherwise they would’ve tried something.
User Thinks Their Problems are Special
Similar to analytics, it is common for operations staff to make pitches for organizational tooling that would help them manage clients, relationships, invoices, and so forth. They want an AI for managing/organizing their sales emails, an API for their invoices, a better system for managing vendors – but what they need is a totally vanilla CRM, payments solution, or ERP respectively.
Oftentimes, the user thinks their problems are special and unique, as if they’re the only firm that has to manage customer relationships. It won’t occur to them that their problems are extremely commonplace and that they don’t need any cutting-edge, flavor-of-the-month technology, but rather, that they need an incredibly boring SAAS product used by millions of other companies: I have had countless conversations that ended with “you are describing a CRM. You don’t need AI. You need a CRM. Have you tried looking for a CRM? No? Ok, try Salesforce.”
You might think it’s fine to build a solution for them – so what if incumbents already exist? – but the problem is that once you’ve built a CRM for them, and they’re actually using a CRM and learning how it’s useful, they will eventually graduate to the incumbent (Salesforce), because there’s a lot of competition. You’d have a customer not because your product is somehow better, but because the customer is ignorant – and eventually they will learn. A fine position for entrepreneurs who want to make a quick buck on a contract that won’t renew.
Conclusion
Building a startup is kind of an Odyssey. It’s very hard to navigate and build good products. Though occasionally guidance from others is impactful, customers are fickle and the stated preferences of users are unreliable. It’s important to have conviction in your own tastes, and the self-determination that makes your products unique: I don’t believe Steve Jobs and Jony Ive really asked for anyone else’s opinions, wants, or needs when building the first iPhone. And you must resist the siren’s song of pitches and opinions from users who are not telling the truth. They will only lead you astray and shipwrecked.
Usually, the other problem is that they are too specific: people like using general-purpose tools. They do not want to learn fifty different maximum-efficiency tools to use for an hour each on average every week. They want to use a few tools all the time.
Chances are that the solution they’re asking for is almost identical to an already existing product in its specification! But they’ve been too lazy to try it. Similar story as with CRMs.
You might argue that picking up an analytics tool in a few hours is difficult, and that’s true. But the key is that the user usually hasn’t even tried. The common failure pattern is that they give it not even a few minutes and then they give up and try to kick the task over to someone else.