👋 Welcome to this week’s edition of 5-Bit Fridays. Your weekly roundup of 5 snackable—and actionable—insights from the best-in-tech, bringing you concrete advice on how to build and grow a product.

Happy Friday, friends 🍻

Here’s some useless information I just came across to get us started. The world record just broke for solving a Rubik’s Cube. Can you actually imagine how your mind must work to solve one in 3.13 seconds? Nope, me neither. 🫡

Anyway, if you’re out having a drink with someone this weekend, you can impress/bore them with that trivia. In more relevant news…

I’m not sure how much of the Reddit debacle around their API changes (i.e. making people pay for it in the ramp-up to Reddit’s IPO) you’ve been following. But, originally expected to only go dark for 48 hours, more than 300 subreddits so far have already decided to go dark indefinitely. Read about it here.

As the world’s leading community platform…that’s pretty substantial.

And lastly, in case you missed Wednesday’s deep dive, a link is below. I had a lot of fun writing this. You might be able to tell. ⛹️‍♂️

Onto today’s post…

Here’s what we’ve got this week:

• Running experiments that don't suck

• How to run a successful sales call

• How to convince someone in <50 words

• The why and how of daily user touchpoints

• Developing your technical chops as a PM

(#1) How to run experiments that don't suck

Earlier this week, shared his conversation with Eric Metelka. Eric is the Head of Product at Eppo, an experimentation platform empowering product teams to run reliable and impactful experiments.

In Peter’s interview, they spoke about:

1. When to run experiments

2. How to run experiments that don’t suck

3. How to introduce experimentation to your company

Given how important running things like A/B tests are (as we saw with Duolingo), I thought I’d share some of Eric’s first-hand insights into experimentation.

Here’s a summary:

When to run experiments

To add to that, here’s an awesome graphic put together:

How to run experiments that don’t suck

Erik also calls out 2 non-obvious effects that may dilute the results of your experiments:

1. The Novelty Effect. Like giving a new shiny toy to a kid, there’s a bias for them to play with it more in the beginning. A similar thing can happen when testing a new feature. To avoid this, aim to run the experiment for at least 2-3 weeks before drawing any conclusions. Another significance in a test is often measured by traffic volume, but it’s not just about the number of people seeing the test, it’s about how that variable performed over time. So, even if you get a high volume of users into your test very quickly…let it run. This also ensures you get a better spread in your data, as, for instance, Wednesdays may perform very differently from Fridays.

2. The Interaction Effect. Sometimes pulling a nob that affects one type of customer can impact (unexpectedly) another. This is especially true in two-sided marketplaces, where a change on one side could impact the other side. In the context of experiments, one experiment may start impacting the results of another. To avoid this, test different hypotheses and try to space your experiments out. That’s why Erik says that guardrail metrics (AKA high-level metrics or adverse effects to monitor, like overall revenue and the number of support tickets) are so important.

How to introduce experimentation to your company

At a later stage company, this shouldn’t really be a problem as your company should have some testing culture already. However, let’s say you’re the PM at an earlier-stage startup that hasn’t gotten there yet and you want to convince leadership on experimentation. Here are three approaches Erik suggests:

1. Share examples of companies that have grown (and are growing) through experimentation (like Duolingo). The more relatable the product is to yours, the more convincing.

2. Emphasized how experiments can help you find the truth faster, and make more data-informed decisions

3. Find 3rd party tools (like Eppo or Amplitude) that make it easy and affordable to start running experiments in a scrappy way.

And a last nugget of wisdom from Erik:

Further reading 🧠

• How Duolingo Grows

• When to A/B Test

• AB Testing Mistakes (podcast)

(#2) How to run a successful sales call

Or more expertly said by Emery Rosanksy (VP at First Round Capital):

I was very recently on the receiving end of a sales call with Amplitude. My team and I use Amplitude extensively at the moment for Analytics and Experimentation, but we’ve been interested in their Audiences product, so we’ve been going through the upsell sales process.

While on the call, I felt I was observing a really well-oiled, nicely polished, sales-led framework. Wanting to clarify my own understanding of what I was witnessing and learn more about what makes a good sales call, I did some searching and quickly came across an article by First Round Capital, where Emery Rosanksy shares some of her hard-won wisdom on sales.

Here’s a summary of what I learned, specifically, about what separates a great sales call—one that closes a customer—from a kicking-the-can one. AKA: “We’ll get back to you.” Founders who are selling (or need to start selling)… this one is for you.

1. Start the call by defining: “Purpose, Benefit, Check”

This is why Emery pushes the PBC method, namely, “Purpose, Benefit, Check”. Simply, it’s an excellent (often underutilized) tool that creates verbal acknowledgment and agreement with your prospect about why you’re having this call, and what the outcome should be. Here’s the example she gave:

Why do this?

• It confirms alignment with your prospect on how they want to spend the time on the call. If there’s any confusion, find out earlier rather than frustrating someone and wasting time.

• It creates a nice opportunity for you at the end of the call to ask for a decision on the next step.

• It earns you credibility and trust for the professional and polished approach. There’s a saying that goes something like “First tell them what you’re going to tell them. Then tell them. Then tell them what you just told them.”

2. Dig deep to find the impact of the problem, not just the problem.

Emery has found that most founder sales calls end up being situational discovery questions. For example, “What challenges do you have in your billing process today?” or “What tools are you using to manage your vendor billing?”

But, as with most things, digging deeper and pulling on threads is where the good stuff is buried. ⛏️

3. Use peoples’ own words back at them.

This touches on a powerful communication principle: mirroring.

4. People remember stories. So, use them.

Numbers don’t create emotional connections. Stories do.

That’s why it’s far more powerful to tell a story about a specific customer and how they went from struggling to scaling with your solution, compared to bombarding someone with numbers (even if they are really good).

A great sales call finds opportunities to integrate storytelling. As Emery describes:

5. Don’t rush the close.

Here are four steps to ending a sales call the right way.

1. Ask for feedback. A simple, ‘I’ve shown you a lot today, what are your thoughts right now?’ can tell you a lot about whether to continue investing in the relationship. This is the moment to close that PBC loop.

2. Outline the next steps. Always aim to schedule the next conversation while you’re still on the call. Once you hang up, it can be near impossible to get that person’s attention again and you’ll lose momentum.

3. Follow up the same day. Always get a follow-up email out before you go home. This should include a summary of the call, what you learned, plus any action items you owe them.

4. Share out. If you want candid feedback about how you did on the call, then send a recording (which you should be doing) to someone you trust to share a few notes.

Further reading 🧠

• Founder-led sales (podcast)

• The Greatest Sales Pitch I’ve Seen All Year

• How I’ve approached founder-led sale

Enjoying this post so far? Feel free to share it ☕

(#3) How to convince someone in <50 words

Here’s a lovely short, and very true, quote by from The Looking Glass.

Further reading 🧠

• How to get better at influence

• Tobi Lütke: The Trust Battery

• How To Influence Without Authority As a Product Manager

(#4) The why and how of daily user touchpoints

A core part of the PM’s and designer's job is being close to the customer. But, things are always popping up to fill up our plates, and often “speaking to customers” gets pushed off the to-do list so we can make time to knock out that last minutes deck a manager asked for.

In the end, we often end up talking with users occasionally, and as Bart Krawczyk says, “That’s not enough.”

Daily touchpoints with customers change everything.

Why?

Okay, but having a user interaction every day sounds like an unattainable goal. And while Bart argues for it (likely as a hyperbole), frankly, it’s an impossible goal. However, what is feasible is probably increasing the number of touchpoints you currently have as a PM. I know I can.

So, how do you do that?

• User interviews (with automated scheduling). The main headache with having more interviews is the logistics of getting them on the calendar. The key is automation, and to automate, you can:

  • Recruit directly from your product/site. Try implementing a banner ad, or using a tool like Hotjar (they actually just launched a neat user interview tool), that invites users to schedule 30m calls using an automatic scheduler like Calendly.

  • Setup triggered emails. This is a personal favorite that drives most of my general calls. Say you’re using HubSpot with some decent contact properties in there. You can create lists of users based on some combo of audience and engagement requirements (i.e. hasn’t logged in in X days, just performed X event, etc), and send automatic emails personalized to that situation, asking for an interview.

  • Recruit from sales. If applicable to your company, ask your sales team to schedule some interviews as part of their sales process. For instance, people that just bought the product or refused to buy it are great sources of insights.

  • Recruit from customer support. Similarly to the above, ask your CS team to funnel certain people your way.

• Doing occasional customer service activities. Not all touch points have to be speaking to customers. There are other ways to keep your finger on the pulse, such as:

  • Actively answering customer requests (usually only doable at smaller companies)

  • Going through older (hopefully tagged) support tickets

• Sitting in on sales calls. Again, you need to apply judgment here, but if you’re able to get added to some sales calls, it gives you a chance to see directly (in the context of selling the product) what kind of objections or questions potential customers have. An extra benefit is you’ll see if there’s alignment between how the sales and product teams perceive and communicate the product.

• Reading reviews. For instance, on:

  • Reddit, app stores, FB groups, other sites, your own forum or community pages if you have them, or anywhere else your customers are talking about you.

• Surveys. Besides exploratory surveys you might send out to a segment of people when you’re trying to learn something specific, you can have different automatic surveys going, helping you get a stream of insights coming in. For example:

  • Regular NPS surveys

  • Setting up an exit survey for people who drop out of a conversion funnel

  • Using a churn survey to understand why people stop using your product (either direct cancellations or lack of product usage for extended periods)

  • Sending a welcome survey for brand new customers (or baking it into product onboarding)

• Go spend time where your users hang out. This could be:

  • Meetups

  • Conferences

  • Online forums

  • Communities (Discords, Reddits, Slack Groups)

Further reading 🧠

• Create an Automated Customer Research Machine in 2 Hours

(#5) Developing your technical chops as a PM

Product managers don’t need to be engineers. It’s not our job to step on toes and waste time getting too deep into the codebase. That’s what the engineers are for. But, the more we understand the technical side of things (like the development process), the better we can move things along, prioritize, and earn more trust from our engineering partners.

So, how do you level up your technical knowledge and find that sweet spot?

Justin Gage—author of the Technically newsletter (fittingly described as “a place for not software engineers to learn about software engineering”)—shared his actionable tips on becoming a more technical PM a while back on Lenny’s Newsletter. According to him, there are the top 5 things PMs should learn about engineering.

1. Learn your company’s tech stack

   1. What to know: The technologies your product runs on (i.e what front/backend languages, database, server infrastructure)

   2. Why it matters: The languages, frameworks, and infrastructure being used all come with important trade-offs. And tradeoffs can impact things like how long it might take to build a certain feature. Also, as this will apply to all the points below, the more you can speak the language of your engineering team, the more trust you’ll build.

   3. How to learn about it: The easiest thing to do is just ask a developer.

2. Learn sensitive points in the codebase

   1. What to know: You want to know which pieces of the codebase are tangled, sensitive, or just near-impossible to change.

   2. Why it matters: Knowing the hardest parts of the app to work on can help you plan more effectively, like resource allocation and road mapping.

   3. How to learn about it: If there are seasoned PMs on your team, ask them about projects that went surprisingly quickly or surprisingly slowly. Also, speak to developers.

3. Learn your build and deploy process

   1. What to know: How your app gets built and deployed to your users, what’s manual and what’s automated, and how long it usually takes.

   2. Why it matters: Larger teams and more complex apps can have multi-day deployment processes that may delay projects and add to lead time. Knowing these pressure points can also help you more effectively batch features and understand what’s worth doing and when.

   3. How to learn about it: Ask a developer.

4. Learn how to contribute some code

   1. What to know: How to make code changes and open a PR, specifically for small, self-contained changes like updating copy or changing colors.

   2. Why it matters: Being able to make very small changes yourself, like fixing a typo, will result in more polished products shipped faster. But, as Justin notes: “Don’t bite off more than you can chew, though; keeping the code you contribute to smaller copy- or design-related tweaks will win you points.”

   3. How to learn about it: Sit down with your favorite engineer and ask them to walk you through how to contribute code.

5. Learn technical basics

   1. What to know: Outside of your own codebase, you want to have some understanding of how apps, data, and infrastructure work together.

   2. Why it matters: This gives you a conceptual understanding of what pieces come together to build applications, and without it, prepare to listen to lots of gibberish in any engineering discussion. Whether you know how to code or not, you’ll want to know what your developer means when they say they can keep this feature to a “frontend change only.’”

   3. How to learn about it: Outside of getting hands-on experience, the easiest is to read engineering newsletters geared towards non-tech-heavy folks, such as Justin’s Technically or ByteByteGo , watch YouTube videos, or even take a basic (free) Codecademy course.

Further reading 🧠

• 5 Technical Skills Product Managers Should Have

• How do I get more technical? (more details by Justin)

🌱 And now, byte on this if you have time 🧠

By now, you know we don’t really have a problem getting to space “cheaply” anymore. SpaceX has laid down some essential infrastructure, with the ability for companies to use their rockets like an Uber to space.

So, the next question… what do we do up there now?

Well, earlier this week, Varda launched a commercial space factory into orbit.

Interested? 🌝

Read | Varda: The Space Drug Factory, by Packy McCormick

And that’s it for this week!

As always, thank you for reading and giving me some of your time today.

If you learned something new, or just enjoyed the read, the best way to support this newsletter is to give this post a like, share, or a restack — it helps other folks on Substack discover my writing.

Or, if you’re a writer on Substack, enjoy my work, and think your own audience would find value in my various series, I’d love it if you would consider adding HTG to your recommendations.

Until next time.— Jaryd✌️

Start writing today. Use the button below to create your Substack and connect your publication with How They Grow