How to refine your product idea PREVEW

How to refine your business idea

• Hey everyone, welcome. My name is Erik and I think you’re all tired of hearing my intro but just for completeness: I worked as a software engineer at places like Apple and Dropbox before transitioning to product management and entrepreneurship. I met Ali when he recruited me to Dropbox, where I started the Dropbox Paper team and scaled that from three of us to about 85 folks, then I cofounded Vanta and grew that from two to a few hundred employees and just shy of a couple billion dollar valuation with Sequoia leading our last round, and I helped scale Panther Labs from a team of 18 employees to about 175 and a $1.5b valuation.
  • That all sounds cool when I say it now but trust me that it felt meandering and confusing while I was doing all of that so if your life feels like that then you’re probably doing it right. I also mostly tried and failed to do things before age 30 but today’s theme is that it doesn’t matter where you start, only where you end up
• If you have any questions or thoughts while I’m talking, please post them to the chat! I have a few points where I’ll stop and check anything that’s been written for some Q&A so please don’t be shy. I left time for questions!
  • Also, I’m not a very good artist but I wanted slide visuals so all images today are brought to you by Dall-E!
• There’s a lot to talk about at a startup, from recruiting to fundraising to sales. But you’ll find that basically everything is easy when you have product-market fit, and everything is hard when you don’t. Today, we’re going to talk about refining your business idea and specifically about iterating your way to product/market fit. Here are a few problems/observations about this:
  • First, PMF involves a tremendous amount of luck. It’s very, very hard to find great business ideas. Most people would see that and say, well, if it’s all luck then there’s no way to deterministically win.
    • (slide) But that’s not right. If the game is a blindfolded dart throw, then the optimal way to play is to throw as many darts as possible before your funding or your motivation runs out. Today, we’ll learn how to do that.
  • Second it’s very difficult to feel like you’re making progress in the early days. You’re probably used to a post-PMF company where there are tickets, user feedback, outages, etc. and there’s no shortage of stuff to do. In the beginning you’ll have one spike of work where you name the company, set up your email addresses, etc. but then you just show up to a room and it’s like… what do we do today? This can be demoralizing and, at worst, push teams to start building a product before they’re ready because of motivational problems vs. because they’ve found PMF
    • Your two most important resources are funding and motivation. The latter is actually more important.
  • Third, everyone tells you to talk to customers but doesn’t teach you how to do that. This leads to a lot of bad customer discovery that burns your time and eats up your warm intros but doesn’t get you closer to PMF. I call this “tell me your problems” customer discovery because for some reason people think that phrase will work (it doesn’t)
    • “Tell me your problems” → crossed out
• Let’s talk about how to solve those
• [question break]
• First, I want to give you the most abstract framework to guide your thinking about product development. That framework is: “customers → problems → solutions → implementations.” This is the order in which you should “lock in” a decision, and you need to be very careful about not skipping steps. This is because decisions for anything on the right side of the arrow are dependent on decisions already made on the left side of the arrow.
  • When you skip steps, you bundle multiple assumptions together — you’re validating a problem, solution, and implementation all at once. So quick math quiz: if the probability of being right about one of those is 1% and you guess a problem, solution, and implementation — what’s the probability that you’re right about all three simultaneously? Hit me up in the chat. Yeah, it’s 1%^3, or a one-in-a-million chance of success. If you take those problems in sequence, you can attempt each stage 100 times to expect success once for a total of 300 total attempts. That’s not only better, it’s the difference between a chance of success and deterministically setting yourself up to fail.
    • “I’m going to tokenize real estate and put it on the blockchain”
  • One question I get is: why start with customers?
    • If you talk to Datadog customers about the one feature they want from their security tools, they’ll say “better integration with Datadog.” If you talk to the Facebook security team, it might be “scalability”
    • What happened? Different customers, different core problems
  • Here’s a trap that founders fall into. When you work in a traditional role at a company, your customers are pre-selected (they’re your employer’s customers) and you “home base” in either the problem, solution, or implementation
    • PMs tend to live in problem land, designers in solution land, engineers in implementation land
  • You have to play all three roles but will likely be tempted to retreat to your area of expertise.
    • I think many of you are engineers, so this means you’re going to be tempted to start writing code of some sort. Code is implementation.
      • [SLIDE]
      • If this happens to you, ask yourself: are you acting like a first engineer or a founder? If you want to start a successful startup, you have to stop defining yourself as an engineer first. That’s behind you. Your job is not to write code or program computers anymore. Your new job is derisking your business by any means necessary and doing absolutely nothing else. If that’s not what you want to do, I’d recommend being a first engineer at a startup instead.
        • Doordash founders drove orders in the beginning
        • I worked as a SOC 2 consultant and manually created and filed documents to get companies their SOC 2s
          • I wasn’t thinking, I’m doing this in order to eventually be able to code. What I was doing was the job, coding was my old job, before I was a founder
  • Since most of you are starting at the beginning of your journey, we have to start with customer and then choose problem
  • (move to customer → problem → solution → implementation slide)
• [pause for questions] [~18 min mark]
• So let’s talk about choosing your customer
  • Business ecosystems are like biological ecosystems. The herbivores co-evolve with the plant life, the carnivores evolve to eat those herbivores, etc. There isn’t much “missing” from a stable ecosystem — it’s relatively self-contained, the apex predators fit their environments, etc.
    • (nature picture?)
  • But then… boom! Something changes. Maybe a volcano explodes or something and kills all of the plant life, it creates a caldera and now it’s Lake Tahoe, whatever. The whole system is in disarray and there are different ecosystems and nutrients that can support entirely different life forms altogether. The system scrambles to get back into equilibrium and then stays there until something happens again.
  • Systems that are changing are most ripe for startups because the rules of the game flip upside-down.
    • When software was on-prem for however many years, there were lots of successful businesses that sold security products to secure your company’s in-house servers. Like, you’d run a server in your closet and need to secure it, sometimes physically
    • When everything moves to the cloud, it’s like that volcanic eruption. No one has any software to deal with this because why would they? It wasn’t a thing, so no one could start a business around it. Just like a fish couldn’t live in Lake Tahoe when it was a volcano and not a lake.
    • Now it’s a footrace to build the missing products. Startups win footraces surprisingly well against incumbents.
  • So what am I trying to say? Mostly: look for what’s changing. Change creates opportunity.
    • It could be that voice transcription is getting better
    • That “customer success” is now a role but it didn’t used to be
    • That startups, for the first time ever, had to start caring about security to get early sales deals done (!)
    • etc.
  • Don’t try to first-principles reason too hard about this. Sometimes people sit in a dark room and think really really hard to figure out what’s changing. Knowing and believing that this thing is changing is your secret sauce — it’s the key insight for your business. If it’s obvious without hands-on and time-consuming research, it’s probably obvious to everyone. If it’s obvious to everyone, the foot race has already started and you’re late. Find something new and unique that’s obvious to you but mostly not other people.
  • The final important concept for customer selection is: find customers that you feel passionate about. They don’t have to be your life’s mission right now but don’t make it someone that you actively dislike. You’re going to spend a lot of time thinking about these people so make sure you at least meet the bar of “legitimately curious” before going too far down any road
  • (slide that says “Customer selection”)
• [questions pause] (~23 minute mark or so)
• Alright, so we’ve picked our customers. Now let’s talk problem discovery
  • Here’s the worst question you can ask during customer development: “what are your problems?”
    • Everyone who has never started a successful startup tells you that this is all you need to ask
    • It would be awesome if that worked! This Neo thing could be a lot shorter and startup success rates would be nearly 100%
  • Why is this bad?
    • First, if someone knew a multi-billion dollar opportunity and actively had it in their head, they probably would just start that company
    • More importantly, most people just aren’t that introspective about their jobs. It could be for lots of reasons but mostly to survive you have to focus on the day-to-day. It’s hard to see industry trend lines or even zoom out and see the year-to-year or quarter-to-quarter problems
    • So you’re going to hear a lot of short-term answers like “recruiting!” and you’ll either start a recruiting startup (don’t do this) or just get frustrated and stuck
  • So that’s what not to do. So what should you do?
    • We talked before about the dart game, and how the goal is to throw as many darts as possible before time expires. This means very tight feedback loops.
      • A slow feedback loop is scheduling a one-hour call a week from today and not learning anything interesting from it
      • An even slower feedback loop is spending six weeks coding something and then finding out that you weren’t solving a worthwhile problem
      • The best feedback loop is an oracle that just told you if your idea would work or not!
        • You could just shout 10,000 ideas at it and eventually one might be right! Then you just go build that (and you already mostly know how to build stuff) and you’re done. Taadaa!
    • The rest of this talk is about building that oracle, and specifically we’re going to turn you into one. We’re going to shift focus away from “talk to customers to find a multi-billion dollar idea” and we’re going to instead move to “learn how to simulate your customers so you can answer any question correctly without having to leave the building.” By taking an indirect route to our goal, we can actually get there faster and more reliably than by beelining for it
• [question pause] (~28-30 min)
• So, the core concept that I’d like to introduce today are what I call “hypothesis sheets.” A hypothesis sheet is a list of everything that you believe to be true about your customers or your industry of choice. Your job, every day, is to take hypotheses from your list and either validate or invalidate them. You should do that validation or invalidation via the cheapest and fastest means possible and then move on and throw your next dart. When you’re done, you’ll either have a business idea or you’ll decide that there isn’t anything obvious to build, in which case you can pivot or focus on a different customer or problem.
  • At Vanta, we did about four of these — voice transcription, data science/ML tooling, drones, and cybersecurity. The last one worked and then we came up with a bullshit origin story because that makes it easier to recruit and raise money
• What I want to do now is to put together an example hypothesis sheet for cybersecurity so you can see how this works
  • We’re going to start with a blank doc and our first step is to write down everything we think we know about security. Now, if you were like a gifted kid or something growing up then maybe you’re used to people telling you that you’re smart and know a lot or whatever, well, this can be a painful step. You’re going to confidently write down a lot of things that aren’t correct and are sometimes downright embarrassing. Shed your identity, embrace being wrong, don’t worry about how people think about you. That’s going to be a recurring theme in entrepreneurship. We’re not trying to sound smart or impress people, we’re trying to start a successful company — when those are in conflict, we pick the latter.
  • So let’s say I don’t know anything about cybersecurity, which I didn’t when I started Vanta.
    • “Security teams spend the majority of their time chasing hackers who are trying to get into their infrastructure”
    • “Strong encryption is the most important defense against hackers”
  • We actually want to draw out the stuff that’s wrong! Because those are the assumptions that will lead you down a bad path. Think hard about all of your biases and assumptions!
  • The next step is to do a quick “sort by highest risk” (don’t think that hard about this) and start to validate
• Ok, so we have our initial hypotheses listed and know that a lot of them are probably wrong. In a second we’ll validate them together, but first a quick pause for questions.
• [questions so far?]
• Let’s walk through validating a few hypotheses
  • “Security teams spend the majority of their time chasing hackers who are trying to get into their infrastructure”
    • How can we validate that? Type answers into the chat!
    • I would go chat with security personnel for this one. It should be easy to find out how they spend their time.
    • Bad question: “do you spend the majority of your time chasing hackers trying to get into your infrastructure?”
      • It’s leading and doesn’t allow for sufficient exploration. People are inclined to answer leading questions with “yes” or “sort of” because they want to make you happy
    • Better questions
      • “How do you spend your time in an average day?”
      • “What’s the most important thing you got done this week? Why? What didn’t you get done but you wish you had?”
      • “Can you tell me about the subteams in your security organization? How large are they relative to each other and what do they each focus on?”
      • “What was a big win for your team recently?”
    • What did I learn? Lots of paperwork, involvement with the sales team, time spent with customers. Hard to focus on actual security.
  • “It is possible to automate the process of preparing for a SOC 2” [a SOC 2 is a type of security certification — that’s the Vanta product]
  • “Every company is different, so their SOC 2 process will be different”
  • [questions?] [45 min or so?]
  • “Customers would use this tool if I built it”
• Ok, last tactical tip — when you end an interview, try to get an intro to the next person so you have a continuous stream of calls
• So I know that I covered a lot of material today. I’ll give a quick summary and then take questions: