04. Balancing user feedback

Estimated Time

  • Reading: ~8 minutes
  • Video: ~17 minutes
  • Activities: to be completed prior to the next week


  • Focus on developing and improving your core features
  • Not all customer feedback is equal — find your high-value users
  • Customer requested feature development should not be used as a sales tactic
  • Find a balance that works for you and continue to make daily progress

Episode Date: September 3, 2021 -- External Link to video

Jason Calacanis | TWiST | Twitter | LinkedIn

Customer feedback

  • Founders need to get to know who the users are in order to better understand the feedback
    • Why did they pay to use your product?
    • How they are using the product/service?
    • What do your users need?
    • Are you meeting the needs of the user?
  • Collecting feedback from your users is critical to the adoption and growth of your product
  • As we've discussed in detail, you should be prioritizing customer interactions as a founder
  • But keep in mind, not all customer feedback should be weighted equally
  • It is essential to understand which customers you are talking to and which input to consider for your roadmap
    • Create a group of your top customers
      • It is likely your top users understand your product as well as (or better than) some of your employees
        • This isn't ideal, but it is a reality
        • Does your sales team log in to your product regularly?
        • When was the last time your leadership team or advisors were active in your product?
        • Another reason sharing customer use cases with your team is valuable
      • So who are the top 5-10% of your customers?
        • You can measure this "top group" by revenue or engagement
        • Users who are using your product the most (engagement)
        • Users who are paying the most amount of money (revenue)
        • You probably want to blend these together for a well-rounded representation
          • Unless you are collecting feedback on a specific metric that makes sense to isolate them
      • Building for the customer is usually a winning strategy - especially when you know they are the most active or most invested customers
        • You are solving their problems, so getting input from the most valuable cohorts is a smart place to start
        • Listen to your customers, document their feedback, and bucket common concerns or suggestions together to drive roadmap decisions
    • Keep in mind that customers don't always know what they want
      • This is another reason it is important to get to know your ideal customer profile deeply
      • You can then blend your expertise and product vision with the problem they are trying to solve and really delight them
        • Even if they can't verbalize exactly what they want
    • Customer feedback isn't intended to convert customers
    • You are building a company and a product, not a consulting service
  • When it comes to bucketing feedback there are a few things to consider
    • Did the user offer you the suggestion or did you request it?
      • Gain an understanding of the types of suggestions that come in without your request to determine how to proceed
        • These users are oftentimes not in the group of value customers
        • This feedback should not be ignored, but place more emphasis on the feedback coming from a valuable user
      • Don't get bogged down by focusing on the casual users
        • Especially a free user without skin in the game
        • These users are often quick to offer suggestions or complain without being invested
  • How you are talking to customers is just as vital as who you are talking to
    • Customer surveys are a great place to start because they are simple to create and implement
    • A few things to keep in mind when creating a survey
      • Keep the survey short
      • Only ask one question at a time
      • Before sending the survey review and remove unnecessary questions
        • Have a clearly defined goal of what you want to accomplish with the survey
        • Remove all questions whose answers will not get you to your goal
      • It doesn't have to be only multiple-choice; use open-ended questions as needed
  • Collecting qualitative data not easily available in surveys is another great way to gather feedback
    • Schedule recurring calls with key customers
      • Keep it very casual to build rapport
      • Begin the calls with open-ended questions like:
        • "What can we help you better understand about our product?"
        • "What is working for you?"
        • "How can we help you be more efficient with our solution?"
        • "What do you wish it did?"
        • "Is there anyone else you'd like me to help you onboard at our next meeting?"
      • Help them get more team members using your product — you'll save them work and ensure the early adopters are up to speed on your product
      • This call can also be a great time to show off any new features or collect initial feedback on roadmap items
        • This should be done after you have provided value to the customer
          • Can be phrased: "We have some items on our roadmap that I think you might find really interesting - would you like to check them out quick?"
        • Similar to launching your MVP — you can simply show mockups and ask what they think about this new feature in the works
        • Often the customer will feel special that you're having an intimate conversation and requesting their insights
      • Record the conversation for internal review
    • Build these relationships with early customers
      • Frequent conversations will build trust and credibility
      • Encourage users to tell you what you are doing wrong
      • Honesty from your most active and invested users is invaluable
      • This feedback will help drive product development and lessen the change of churn
  • Balance feedback with product vision
    • Fitbod does a great job of balancing feedback with their core development
    • They keep the product vision at the forefront but are willing to experiment with features suggested by customers
    • The three streams of product development they focus on at their high-growth startup are:
      1. Core Feature Development
      2. Experiment-based Feature Development
      3. Importance of product-led growth
    • Core feature development
      • These are features that your product cannot exist without
        • For example, video processing for YouTube
      • The core features of your product can not be neglected for one-off suggestions from users
      • Feedback on your core features should be considered over requests for new features
      • You must do one thing extremely well and that should be your main focus of development
        • What is your main value proposition? What is your one simple sentence?
        • You must build around this first and foremost
        "The number one thing I end up talking to people about is, how can we do fewer things, but do them faster and better." - Charles Hudson
    • Experiment-based feature development
      • Be careful as this is where startups can "overbuild"
        • You never want to go on a feature death march where you build without a clear purpose until you run out of money
        • Avoid what Jason calls "founder ADHD" and wanting to build the newest shiny thing simply because you are excited about it in the moment
      • The experiments you try need to be with a purpose
      • Keep in mind that a vision or product without users is not a company
        • As much as you need to be focused on your vision, if people aren't using the product the company won't succeed
        • Listen to the users, find product-market fit for your core features, then consider which additional features to experiment with
      • You can leverage experiments and customer feedback to define parts of your roadmap
        • This is outside of core feature development
        • Bucket user feedback based on their suggestions
          • Identify alignment with your vision and across your valued customers to prioritize which to try first
      • Keep the MVP mindset for this feature development
        • Implement the experiment quickly, test the new feature with users, and gather feedback
        • Make quick decisions (based on the metrics you define beforehand) to move forward with additional development or cut the experiment
        • Do not be afraid to delete unused features or unsuccessful experiments
      • The Lean Startup has an entire chaptered entitled "Experiment" that touches on this exact concept
        • Ries dives deep on questions like "which customer opinions should we listen to, if any?"
        • And he reiterates the scientific method of entrepreneurship: "If you cannot fail, you cannot learn"
        • He suggests that you should treat each experiment like an MVP
          • Start small but continue to think big picture
          • Implement changes, iterate, and always be learning
        • I would highly suggest rereading chapter 4 from The Lean Startup for more insight on experiments
  • Product-led growth
  • Fitbod does a great job of blending their product vision with testing user feedback
  • They are focused on those three streams of development while staying true to their roadmap principles
  • The simplicity of Fitbod's 4 roadmap principles are a huge part of their growth and one I would consider implementing in some form at your company
    • Their four principles are:
      1. Focus on Craftsmanship
        • Build something that you're proud to share with your friends and family
        • Build something that you can happily use yourself
          • Allen and Jesse were Fitbod's first two power users
          • They built a product they needed and would use
        • This focus on craftsmanship is across their product
      2. Always make progress
        • Small steps in any direction are better than no steps
        • Each iteration of the product is an improvement and offers an opportunity to learn and improve
        • Keep in mind, that done is better than perfect
          • Done is never really done — so get the product, new feature, etc. in front of users and gather feedback
          • Reid Hoffman famously said:
            • "If you're not embarrassed by the first version of your product, you've launched too late!"
            • This applies beyond the first version, to new features and experiments
              • But should not proceed the first principle — focus on craftsmanship
        • This is one of Jason's key focuses also — he is always encouraging founders to get just 1% better every day
      3. Leverage core differentiators
        • Prioritize features that leveraged and highlighted core differentiators
        • It is important to be aware of your competition but to not obsess over them
        • Make your core features faster, better, easier to use, etc.
      4. Delight early adopters
        • Listen to customers and deliver
        • Constant improvements and iterations build trust with early users
        • Talk to them regularly
  • It is important to find a blend of product vision and customer feedback that works for you
  • But prioritizing your core features should always come first
    • And adding new features should not be used as a sales tactic
    • Users who say they "need just this one little thing" and they will buy, will always need just one more thing...
    • Focus on what you do best and delight the users that have the problem you solve for
  • Prioritize on improving your core features!
"You have this bundle of energy as a founder; we just need to make it into a laser, not a grenade." - Jason
  • In other words, do fewer things, but do them faster and better
  • Jason continuously reiterates with our portfolio companies...
    • When you find something that works double down!
    • When you strike oil, keep drilling!
      • Improve the digging tools to be more efficient
      • Create faster augers and bigger wells for the oil
    • Don't start looking for diamonds!
  • Focus on improving the core features every single day while listening to your users and experimenting with their suggestions
  • Find a balance that works for you and keep building

Additional Resources


Week 8

Identify your top 5% of users
  • Document who they are in a CRM or spreadsheet
Schedule a recurring meeting with a top user
  • Can be more than one
  • The meeting doesn't need to be long, 10-20 minutes is fine
  • Find a cadence that works for them — once a week, every other week, once a month

Week 9

Bucket initial user feedback
  • This should be based around your core features at this point
Identify core differentiators
  • Which of your core features will help you stand out from the competition?

Week 10

Does a product-led growth strategy make sense for you?
Continue to collect and bucket feedback from users regularly