Hi all —
After talking with approximately 2.4 billion B2B founders, I’ve realized one of the main root causes of everyone’s pain and frustration.
“Experimentation.”
We’ve been told since the dawn of the Lean Startup that we need to generate hypotheses, experiment, measure, learn, and pivot.
But the Lean Startup’s experimentation orthodoxy is wrong and counterproductive for founders and GTM teams. Why? It assumes you can generate testable hypotheses and learn real things. Generally, you can’t. Every time you try to do this in a startup setting, you realize your hypothesis was dumb and wrong, but can never figure out why.
Worse, the Lean Startup approach (along with OKRs and a bunch of other “best practices”) has contributed to an obnoxious culture of startup LARPing where teams say fancy words and make nice slides, but accomplish nothing. The word “experimentation” sounds scientific and professional, just like the word “platform,” so we say it a lot. We half-ass experiments, decide it’s time to “pivot” because we’re not a billion-dollar company immediately, and wonder why we never make progress.
See this through a few times and you realize that the Lean Startup is bullshit. Most successful founders realize this… it just hasn’t been said aloud.
Once you realize the Lean Startup is bullshit, you tend to take the anti-scientific approach I lovingly refer to as “throw shit at the wall and hope something works.” This approach has downsides – you can wind up in “indefinite experimentation mode” – but it usually generates better results than the Lean Startup approach.
There IS an approach to GTM experimentation that avoids the Lean Startup’s idiocy and the anarchy of “throwing shit at the wall.” I’ll explain that after I dig into why Lean Startup-style experiments never work.
The scientific method ain’t for business
Traditional experiments lead to these kinds of conversations:
- “We sent 1,000 email sequences and got no useful responses other than ‘unsubscribe.’ Time to pivot?”
- “We ran a Google Ads test with a few thousand bucks, our cost per click was $25 and we got one website conversion, but they didn’t show up to the demo. Time to pivot?”
That is: You’ve learned nothing, and have no clue what to do.
Here’s why.
Any sort of “experiment” you run tests a gazillion variables (way more than you realize ex ante). When the experiment doesn’t work, you start to realize how many things could have gone wrong, and you’re not sure which of the gazillion variables fucked you over.
Let’s take a common example: Experimenting with outbound email.
When your outbound email experiment doesn’t work, you realize it could have been because:
- You’re emailing the wrong kinds of people
- (Are you targeting the wrong title? Wrong kind of company? Wrong size organization? Wrong geography? Wrong seniority level? Or something else?)
- Something about your messaging is wrong
- (There are a million different templates out there by “gurus” – why aren’t people responding?)
- Something about your operations is wrong
- (Are you sending from the right domains? Getting spam-blocked? Sending enough volume? Sending at the right time?)
- Something else is wrong
- (Is email even the right channel? Or should you be using a multi-channel approach?)
You don’t know if you’re off by an inch or a mile. Is the problem one variable or all of them? Impossible to know. And so you throw your hands up in the air, decide it’s time to pivot, and never leave the starting line.
This is true of all kinds of “business experiments” – it is nearly impossible to get ANY data back. If you’re lucky, you do get data back – but those data are often mixed and difficult to interpret. (No, customer interviews aren’t the solution, because you always hear what you want to hear in customer interviews. That’s for another post.)
So WTF do you do?
Get something kinda-working, fast.
My experimentation approach is best described as: “Get something kinda-working, fast.”
After an MBA, years in consulting, and watching multiple startups succeed/fail, the ones that win are the ones that get some sort of sales momentum… and figure out stuff along the way.
The ones that fail are the ones that spend a lot of time theorizing, hypothesizing, synthesizing, analyzing.
Here’s why “getting something kinda-working” is a better strategy than all the fancy-sounding theories from books and school:
- It’s much easier to get something that kinda-works to work better… than to figure out something new that works well. We under-estimate how hard it is to get ANYTHING to work.
- You will NEVER know ex ante how “something that kinda-works” will unfold. Maybe it will lead to a billion-dollar-revenue opportunity, maybe it will lead to a ten-million-dollar-revenue opportunity. You can’t imagine what you’ll learn as you execute. But “something that kinda-works” is more likely to lead to ANY positive outcome than “something that makes logical sense in theory, but hasn’t been proven in practice.”
- Nobody is smart enough to think through all the implications of an idea, a hypothesis, or an experiment. You’ll learn more by doing than by thinking.
“Get something kinda-working, fast” means “SELL TO LEARN.” Implications:
- Design “experiments” to get responses first
- Using the outbound email example, you want to design emails to get responses first… once you get responses, then you can then tweak your messaging to book meetings.
- Don’t bother with marketing or PLG in the early stages, because you need the pain of trying to sell something to a real human, live, to actually learn things.
- (You can sometimes get lucky with marketing / PLG in early stages, and that’s great, but generally the best probability-weighted approach is to sell)
- Sell something, even if it’s not what you ultimately want to sell
- Yes, we all want to sell our category-creating software platform. In the early days, sell whatever the buyer wants to buy — being able to work directly with customers is priceless and will teach you what’s really worth building + selling.
I’ll leave you with this: Any business can find product-market fit. Any opportunity can become big. You don’t figure this out via “experimentation” and startup LARPing. You figure your business out by selling and iterating.
