As part of the continued series on startup anti-patterns, we look at the battle between conviction and validation.
First, a story. In 2000, Intech technology, a fledgling startup out of Israel, was building a new type of billing software for property managers. Intech had one potential customer- the Israeli government – that shared the founders’ vision of software which could split bills across multiple tenants in a customizable fashion. For example, using this “killer” feature, the property manager could decide that one tenant pays 70% of the gardening bill while another pays the rest.
The excitement at Intech technologies was at its peak. The founders automatically assumed that if they had the vision and one customer wanted it, many others would. Eighteen months and several layoffs later, the truth was unveiled: end-users didn’t really care about the “killer” feature. Other prospective customers showed no interest in the product’s advanced bill-splitting capabilities. They opted for simpler and cheaper systems that generated invoices and connected to building meters.
After building a product that ended up being an overkill, the company shut down. The founders (Itamar was one of them) learned a hard lesson.
What it is
“If you build it, they will come” is the anti-pattern where startups make decisions based on their vision of how a solution should look, ignoring or underemphasizing customer needs and neglecting to collect sufficient product validation from prospective customers.
The origin of this anti-pattern is the allure of “a great idea”. Entrepreneurs, driven by their passion and conviction, tend to assume that their product’s brilliance alone will captivate customers and guarantee success.
Unfortunately, the mere existence of a product doesn’t automatically translate into customers flocking to buy it. The “if you build it, they will come” mentality often leads to a lack of product-market fit, a leading cause of early stage startup failure.
When combined with confirmation bias, another anti-pattern, this problem becomes even more acute. As with ignorance, it’s usually deadly when combined with a big dose of arrogance.
Why it matters
“If you build it, they will come” mentality can kill your company. It results in redundant product development and misalignment, a significant waste of resources, increased technical debt, and challenges in go-to-market. Hoping that a product will resonate with customers is often a recipe for disaster.
Building a product based on conviction as opposed to market validation can harm your startup in multiple ways:
- Increased adoption friction. Instead of iterating and improving the product based on customer feedback, startups who fall into this anti-pattern often lack the features customers want. They find themselves trapped in a vicious cycle of slow growth, small capital raises, financial strain and, ultimately, the demise of the startup.
- Slower product development. Development teams should aim to build what’s most valuable for the business as quickly as possible. Building on conviction without validation is risky because unnecessary features slow down development without creating sufficient business value. Solution complexity and the likelihood of incurring more technical debt, slowing down future development, and shortening a startup’s runway.
- Low morale. Discovering post-launch that a product isn’t well-received can demoralize a team that worked hard on its development. Before then, team members who know that development is happening with insufficient validation may be demoralized by the company’s approach.
Building “in a vacuum” increases the risk of achieving product-market fit. This misalignment can manifest in various ways. The product might solve a problem that customers don’t care enough about or may fail to meet customers’ expectations or needs. Without product-market fit, it’s harder for a startup to build the right brand, launch effective marketing campaigns, and build the right sales playbook.
Diagnosis
Diagnosis requires honest self-reflection. Look at how the company makes product decisions that commit it to significant expenditures of time and money:
- Are you aware of all important decision points? Lack of awareness leads to implicit decision making. System 1 thinking, skewed by cognitive biases, dominates implicit decisions. Make decisions that commit the company to significant resource use explicitly.
- When making important decisions, how much weight do you give to conviction (vision, gut feeling) vs. anecdotal evidence (hearsay, one or few data points collected by an ad hoc process) vs. sufficient evidence collected by a thoughtfully designed validation process? Making big decisions without a responsible amount of evidence is risky.
- Does the evidence supporting decisions come from a sufficiently diverse range of stakeholders, both internal and external ones? Making decisions based on limited/skewed information is risky, especially when decision-makers aren’t aware of the bias and/or variability of the data.
When attempting to diagnose this anti-pattern, make an honest assessment of the extent to which conviction stems from fear. Sim knew a brilliant technical founder who’d rather spend 100 hours writing code than have a validation conversation with a stranger. He thought his product was going to be awesome. It was the only rational way to avoid talking to people who may give him negative feedback.
Fear often deters teams from engaging in validation processes due to a variety of psychological, organizational, and market factors:
- Fear of being wrong. People often intertwine their ideas with their personal identity. They may perceive being wrong as a personal failure. Cognitive dissonance pushes individuals to avoid situations that might challenge their pre-existing beliefs. Confirmation bias pushes them to unconsciously ignore unfavorable feedback.
- Fear of the unknown. If validation feedback suggests that significant changes are necessary, this can lead to an overwhelming feeling of uncertainty. The path forward might not be clear, which can be daunting. Even founders, who typically are comfortable with massive amounts of uncertainty, can fall prey to this.
- Fear of authority. In some hierarchical organizations, when a person of authority has conviction, people lower down in the organization may avoid validation. They fear repercussions if it contradicts the authority figure’s conviction.
- Fear of disclosure. Some entrepreneurs feel their intellectual property (IP) is so valuable that they fear validation processes might leak some of that IP. In his VC days, Sim met with several founders unwilling to talk about the details of their technology before a term sheet. You can imagine how these pitches went.
- Fear of being late. Some teams may skip validation to hasten delivery. They may fear that competitors may beat them to market or feel pressure from stakeholders to deliver by a specific deadline. Discussing time pressure trade-offs honestly and explicitly is good. Replacing validation with conviction implicitly, for fear of being late, is a problem.
- Fear of wasting an investment. Once a team has invested time and money in a particular direction, they might feel that continuing forward is the only option. This is known as the sunk cost fallacy. For fear of creating waste, they will ignore negative evidence. Humans often exhibit loss aversion, where the pain of losing is psychologically about twice as powerful as the pleasure of gaining.
Arrogance and confirmation bias are the most common anti-patterns that make the diagnosis of “if you build it, they will come” difficult.
Misdiagnosis
Misdiagnosis occurs when companies set an unreasonably high bar for the validation required to make product decisions. It may lead to analysis paralysis in organizations, another anti-pattern, reducing the company’s competitiveness in the market and its ability to launch new products in a timely manner.
What is a reasonable, let alone optimal, split between conviction and validation when making decisions? There is no right answer. Context matters. Marissa Mayer famously asked a team at Google to test 41 different shades of blue for the toolbar on Google pages. Was that too much? It’s hardly excessive when considering Google’s scale and Google’s resources. A 41-way test may not have been that much more difficult to execute than a 2-way test. However, the request to test 41 options applied to a product with limited usage would be ridiculous. It’d take too long for the test to produce a valid result.
Refactored solutions
Once diagnosed, the refactoring of this anti-pattern requires changing the organization’s mindset and approach to product development:
- Empower people to make data-driven decisions. Instrument products for data collection with good security and privacy controls. It should be easy to implement A/B and multivariate tests. Clean, machine-readable metadata should be available for data enhancement. Manage data consistently in a unified platform. Give key stakeholders self-service access to the analytics that matter. Operational dashboards that answer known questions aren’t enough: optimize for ad hoc analytics aimed at answering new questions quickly and precisely. Distribute organizational authority, responsibility, and accountability for making decisions based on data.
- Embrace market research and customer feedback. Starting at the top, foster a culture of listening to markets by implementing methodologies such as customer development. Engage with potential customers through surveys, interviews, or beta testing to gather valuable feedback that shapes the product roadmap and the entire company. Pay special attention to statistical validity.
- Share the voice of customers. Broadly distribute customer and market feedback within the organization. Spend time in all-hands and other company-wide communication channels to highlight customers. Empower your customer support/success team to work more closely with product teams and rotating engineers and product managers to support duty.
Your ability to make well-validated product decisions is like a muscle: the more you exercise it, the stronger it gets. Getting good at validation isn’t easy. It requires significant investments in culture, systems, and processes. It also requires overcoming fears.
To overcome fear and foster an environment that encourages validation, organizations and teams can foster a culture of learning and experimentation; encourage collaboration and open communication; and incorporate iterative processes with smart feedback cycles. By addressing fear, organizations can improve the likelihood of developing products that meet market and customer needs, ultimately enhancing their chances of success.
When it could help
This anti-pattern can help in two cases: when an excess of conviction can be useful and when the expected value of validation is low.
As with ignorance, an excess of conviction can be useful in very special circumstances:
- Entrepreneurs vary wildly in their ability to predict the future. On average, they’re very wrong, but there are outliers. If you have solid evidence, without ego-boosting revisionist history, that you are such an outlier, it may be smart to put relatively more weight on your convictions.
- If resources and timeframes are very tight, there truly may be no room for doubt or validation, and it may be worth taking on significant validation risk. It’s time for a Hail Mary pass. Startups often live or die by these decisions.
- There’s a saying in venture capital that a little bit of data is a dangerous thing. Sometimes the presence of data that isn’t great is worse than having no data at all. This is especially true in tough fundraising climates, when investors who are slowing down their investment pace are looking for even more reasons to reject deals. Since hiding bad data is unethical, entrepreneurs sometimes make the decision to avoid or reduce validation instead of risking having to disclose unfavorable data. However, the absence of validation data may lead to fundraising failure.
All these strategies follow the strategy paradox: while they can be extremely successful, they can also lead to extreme failure. Even Steve Jobs, the quintessential product visionary, came up with Macintosh Portable and the Newton.
There are some cases where market validation has lower expected value because it produces fuzzy and/or biased results:
- Highly disruptive products. One example is Uber/Lyft in the early days. When surveyed, early prospective customers were concerned about getting a ride with an unknown, unlicensed driver. However, after consumers got used to the convenience and cost efficiencies with ride-hailing, they became comfortable with it. Strong network effects compound this early on and it is difficult to imagine the value at scale.
- Groundbreaking technology. It’s sometimes hard to articulate technology that works like magic to customers. When Steve Jobs introduced the iPhone, many didn’t understand why touch screens would matter so much. Previous smartphones had keyboards and regular touchscreens, and it wasn’t immediately apparent that capacitive touchscreens would change the world.
- Category creation. In blue ocean scenarios, there are no (or almost no) prospective customers to talk with. The market or category doesn’t exist yet and will only unfold in the (hopefully near-term) future. For example, when Life360 first launched, investors, advisors, and even parents consistently said they don’t believe kids will have smartphones. Smartphones back then were business tools, not replacements for cell phones, and the general audience didn’t think kids would need them. They were clearly wrong (easily said in hindsight).
Some ideas are much harder to validate than others. Smart startups focus a lot of effort on validation to reduce the risk of achieving product-market fit.
Co-authored with Simeon Simeonov. More startup anti-patterns here.