Nir’s Note: In this guest post, Abhay Vardhan, discusses how to measure the strength of user habits with cohort analysis and retention rate. Abhay is a founder of Blippy.com and blogs at abhayv.com. Follow Abhay on Twitter @abhayvardhan.
You would think PinterestForDogs is doing quite well, right? Well, it depends.
A common mistake entrepreneurs make is to focus too much on user growth. Instead, it is often more important to ask: “Is the product creating a habit so users keep coming back?” and “How do we measure the strength of such a habit?” These questions are crucial because without establishing user habits, it is impossible to sustain a healthy user base. Eventually, all user acquisition channels saturate.
I learned this the hard way with our first product, a walkie-talkie app called blip.me that allowed users to record and share audio clips. Initially, we saw good viral growth as users invited others. User retention, however, was not so great; people were excited initially but dropped off within a few weeks. So how did it end? The app was downloaded by millions, but used by few.
Given the importance of establishing a user habit, how do we track it? In my experience, the best way to do this is by measuring retention rate through cohort analysis. Let us define these terms.
- A cohort is a group of people who share a particular characteristic over a period of time. An example is a group of users who joined on a particular day. A cohort analysis follows these users’ actions over a period of time.
- The retention rate of a cohort for a period of time is the ratio of number of active users at the end of the period to the number of active users at the start of the period. The definition of an active user is usually very specific to the product. For our purposes, a user is active on a given day if he or she has visited the product on that day.
Continuing with our example, PinterestForDogs, we will measure the retention rate through cohort analysis for two cohorts of 100 users, one cohort consisting of users who joined on January 1 and another who joined a week later on January 8.
As you can see from the example, the retention rate decays with time and eventually plateaus. Although the January 1 cohort seems to have higher retention, both cohorts experience a sharp drop off and settle to a retention rate of 0.03 (3%). This final level consists of users that have become hooked: they have established a habit pattern to keep coming back to the product. These are your most interesting users.
How is cohort analysis helpful?
- Critical data does not get thrown off by external events. If you just look at user growth, a mention by a blog may cause a huge spike in user sign-ups but pre-existing cohorts would not be affected much.
- Cohorts give you actionable data. Why is the user retention weak? Could it be the onboarding? Do you need reminders to establish a consumption habit? By watching the cohort closely, you can understand how users are acting.
- Cohort analysis naturally leads you to an intense focus on the consumer. If you are looking at a cohort of 100 users, these are real people. The more you analyze the cohort, the better you will understand them.
- Cohort analysis is an objective view of progress. It’s easy to drink your own kool-aid when you are experiencing heady user growth. Keeping a close eye on cohorts forces you to continue asking the right questions.
What retention rate numbers should you expect?
Naturally, retention rates vary by industry. An e-commerce application that is able to monetize users well can probably live with much lower retention rate than a free social networking app that needs millions of active users to be profitable. The following numbers are tailored to mobile or web apps in the social networking/sharing space.
In our space, 4 weeks after the cohort join date should be when user engagement plateaus, which is the ideal time to measure retention rate through cohort analysis. Generally a retention rate of more than 20% is excellent. To compare yourself to two successful companies in this space, Twitter and Pinterest, you can look at the analysis by RJMetrics (reproduced with permission below).
The graph above is a cohort analysis that analyzes cohorts in time intervals of weeks instead of days and an active user is someone who has published at least once a week. We can see that by this measure, the retention rate for Twitter plateaus around 22%.
On the other hand, the retention rate for Pinterest settles at an astonishing 45%!
If your retention rates are less than 5% after 4 weeks, it’s likely that your product has not establish a user habit. With some cohort analysis, you may be able to determine why. Is it something that users really want? Are your users getting confused? If we look at the retention rate of PinterestForDogs, we see that it is only 3%. This implies that it is unlikely to succeed, even though the first graph of user growth looked great.
Cohort analysis asks tough questions. But, the sooner you ask them the better your startup will be.
To succeed, a product has to create a lasting habit so that users keep coming back. Without habits, even if there is good user acquisition, it is hard to establish a growing base of active users. Simple metrics, such as retention rate, can be applied to new users to determine the strength of the habit. Cohort analysis can lead to valuable insights crucial to the success of any habit-forming product.
Top Consumer Psychology Articles
- Can We Regulate Social Networks To Curb Addiction—Without Making Them Suck?
- So, You Want To Become a Great Product Manager? [Q&A with Jackie Bavaro]
- Will Clubhouse be a Habit or Has-Been?
- The One Fitness App That Hooked Me For Good
- Here’s How Fortnite ‘Hooked’ Millions
- How Apps Can Shape Your Future Self
- How Netflix’s Customer Obsession Created a Customer Obsession
- Want to Design User Behavior? Pass the ‘Regret Test’ First
- How to Trigger Product Usage that Sticks
- How to Get People to Help Each Other, Online and Off
- Here’s How Amazon’s Alexa Hooks You
- How to Use Psychology to Make Persuasive Video
- How to Use Personality Science to Drive Online Conversions
- The Unbelievable Future of Habit-Forming Technology
- The Secret Marketing Power of Evolutionary Psychology
- Don’t Ask People What They Want, Watch What They Do
- How Cognitive Biases Can Help (and Hurt) Your Business
- What Most People Don’t Know About Behavioral Design
- How to Start a Career in Behavioral Design
- Your World is Full of Placebo Buttons (and That’s a Good Thing)
- How to Build Technology that Feels Like a Friend
- 3 Pillars of the Most Successful Tech Products
- Here’s How to Ethically Manipulate Other People
- How Two Companies Hooked Customers On Products They Rarely Use
- How to Hook Users in 3 Steps: An Intro to Habit Testing
- Die Dashboards, Die! Why Conversations Will Reinvent Software
- The Secret to Sending Emails and Notifications That Work
- How to Win Your Competition’s Customers
- Hooked for Good: How Habit-Forming Products Improve Lives
- Good Products Start With Good Questions
- Human + A.I. = Your Digital Future
- Why ‘Assistant-As-App’ Might Be the Next Big Tech Trend
- People Don’t Want Something Truly New, They Want the Familiar Done Differently.
- 4 Ways to Win Your Competitor’s Customer Habits (Slides)
- Here’s Why You’ll Hate the Apple Watch (and the Important Business Lesson You Need to Know)
- The Secret Psychology of Snapchat
- The Psychology of Notifications: How to Send Triggers that Work
- How Technology Tricks You Into Tipping More
- Building Community Starts with Understanding People
- When Persuasion Becomes Deception
- Mastering Pricing Principles
- A Handy Behavioral Design Toolkit
- Onboarding Matters – Getting Users Engaged in your Product
- The Limits of Loyalty: When Habits Change, You’re Toast
- Dual Process Theory: Is Your Product the Elephant or the Rider?
- 4 Ways to Use Psychology to Win Your Competition’s Customers
- Web Psychology – The Science of Online Persuasion
- Developing User Empathy with Design Sprints
- The Real Reason “Stupid” Startups Raise So Much Money
- Understanding the Psychology Behind Game Design
- The Psychology Behind Why We Can’t Stop Messaging
- How to Do Effective User Research
- Context Driven Design (The “Context Effect”)
- The Psychology of a Billion-Dollar Enterprise App: Why is Slack so Habit-Forming?
- Writing Copy for Your Reader’s Brain
- Designing Habit-Forming Products
- Framing Reward is as Important as Reward Itself
- Games, Play, and Motivation
- How Scarcity & Impatience Drive Irrational User Behavior
- Should You Listen To Your Users or Your Data?
- Emotional Engagement – Designing with the Heart in Mind
- A Free Course on User Behavior
- It’s Not All Fun And Games: The Pros and Cons of Gamification at Work
- Product Psychology: The 3 Things Everyone Should Know About
- Getting Traction: How to Hook New Users
- Designing for Behavior Change Book Review
- The Sneaky Trick Behind the Explosive Growth of the Kardashian Game
- How Successful Companies Design for Users’ Multi-Device Lives
- The Link Between Habits and User Satisfaction
- What Triggers The Best Word of Mouth Marketing?
- What Tech Companies Can Learn from Rehab
- The Secrets of Addictive Online Auctions
- Teach or Hook? What’s the Real Goal of Online Education?
- Using Mind Control to Raise Startup Cash
- How To Build Habits In A Multi-Device World
- How To Cope with Your Insane Jealousy Of The WhatsApp Deal
- Why Do Fads Fade? The Inevitable Death Of Flappy Bird
- You’d Be Surprised By What Really Motivates Users
- Nostalgia: A Product Designer’s Secret Weapon
- How You Can Help Users Change Habits
- Is “Lean Startup” Right for Your Idea?
- Hunting for Habits: Keying in on smart design to make a product irresistible
- Are Companies Too Obsessed With Growth? How to Measure Habits
- Refresh: The App a Secret Agent Would Love
- Angel or Devil: Who’s Really Investing In Your Start-Up?
- In 10 Years, We Won’t Use Personal Technology
- 4 Simple Things I Did to Control My Bad Tech Habits
- “Yes, And”: The Two Words that Created a #1 App
- From Laid to Paid: How Tinder Set Fire to Online Dating
- What if In-App Purchases Came to Real Life?
- Hooking Users One Snapchat at a Time
- How To Save Your Startup From The “Spotlight Effect”
- Bible App: Getting 100 Million Downloads is More Psychology Than Miracles
- How to Boost Desire Using the Psychology of Scarcity
- Marketplaces & The Curse of the Network Effect
- Today’s Behaviors, Tomorrow’s Startups
- Venture Capital and The Superstitious Investor
- The Future is Driven by Interface Changes
- Why Business is Addicted to Habits
- Viral Loops Or Viral ‘Oops’?
- Making a Marketplace
- What Killed Turntable.fm?
- What You Don’t Know About Human Intuition Can Hurt You
- Designing to Reward our Tribal Sides
- New Video – “Hooked: Building Habit-Forming Products”
- How Technology is Like Bug Sex
- Ways To Get People To Do Things They Don’t Want To Do
- The Network Effect Isn’t Good Enough
- Mass Persuasion, One User At A Time
- How Investment Drives Engagement (Slides)
- Getting Your Product Into the Habit Zone
- Where Have The Users Gone?
- Infinite Scroll: The Web’s Slot Machine
- Designing User Habits Video
- Psychology of Sports: How Sports Infect Your Brain
- This is Your Brain On Boarding: How to Turn Visitors Into Users
- User Investment: Make Your Users Do the Work
- Behavior by Design Video
- When Designing for Good Is Bad
- Stop Building Apps, Start Building User Behaviors
- The Next Secrets of the Internet
- User Growth and Engagement: A Hacker Metric
- Spotting the Next Facebook: Why Emotions are Big Business
- The Billion Dollar Mind Trick: An Intro to Triggers
- Why Everyone Hates I.T. People
- Hooking Users In 3 Steps: An Intro to Habit Testing
- Abolish The Reference Check
- Variable Rewards: Want To Hook Users? Drive Them Crazy
- How to Design Behavior (The Behavior Change Matrix)
- How To Design For “Normals”
- The Hook Model: How to Manufacture Desire in 4 Steps
- User Habits: Why Startups Must Be Behavior Experts
- What Is, and Is Not, Your Product’s Job
- Pinterest’s Obvious Secret
- Personalized eCommerce Is Already Here, You Just Don’t Recognize It
- Where is the Web Going?
- The Developer Divide: When Great Companies Can’t Hire
- Being a Quitter Makes You a Good Entrepreneur
- Behavior by Design
- Why You Should Run Your Business Barefoot
- Are you a Startup Star, Wacko, or Wannabe?