What are the ethical responsibilities of companies that are able to manipulate human behavior on a massive scale? It’s a question one hopes technologists and designers ask themselves when building world-changing products — but one that hasn’t been asked often enough.
Operant conditioning, intermittent reinforcement, the search for self-actualization — the techniques used by product managers at the world’s largest companies are equal parts psychology and technology. As Sean Parker, founding president of Facebook, recently acknowledged, the company has long been engaged in the business of “exploiting a vulnerability in human psychology.”
Our gadgets and apps are more persuasive than ever. Yet for the makers of these technologies, few guidelines exist on how to change user behavior ethically. Without a standard, businesses tend to unthinkingly push the envelope in the never-ending quest for more engagement, more growth, and, ultimately, more profits. As one startup founder told me, “At the end of the day, I have an obligation to my investors and employees, and I’ll do anything I can, short of breaking the law, to get people using my product.”
The tech industry needs to do better than the threat of jail time to decide to do the right thing.
Thankfully, most technologists and designers I know are working to make people’s lives better. Around the world, entrepreneurs aspire to build products customers love. Whether working at a large Silicon Valley tech company or out of a garage, they dream of moving people to action by offering them the next indispensable improvement to their lives, and most try to go about this in an aboveboard way.
How It’s Used
Of course, many of them also wouldn’t mind getting rich. But this mix — the drive to make both a difference and a profit — is how humankind has solved many of our most vexing problems. There’s nothing wrong with building products people want to use, but the power to design user behavior ought to come with a standard of ethical limitations.
The trouble is the same techniques that cross the line in certain cases lead to desirable results in others. For example, Snapchat’s use of streaks — which tally the number of consecutive days friends have shared photos — has been criticized for conditioning teens to compulsively keep coming back to the app. But the same persuasion technique is used by the language app Duolingo to help people learning a new language stick with the program.
The same variable rewards used to extract cash from gamblers playing electronic slot machines are also used in video games that help kids with cancer distract themselves as they receive painful treatments.
Clearly, it’s not the persuasion technique itself that’s the problem — it’s how the technique is used.
But without a test to tell the difference between good and evil uses, it’s easy to see how designers can go astray.
The Regret Test
The tech industry needs a new ethical bar. Google’s motto, “Don’t be evil,” is too vague. The Golden Rule, “Do unto others as you would have them do unto you,” leaves too much room for rationalization.
I’d argue that what we ought to be saying is, “Don’t do unto others what they would not want done to them.” But how can we know what users do and don’t want?
I humbly propose the “regret test.”
If we’re unsure of using an ethically questionable tactic, “If people knew everything the product designer knows, would they still execute the intended behavior? Are they likely to regret doing this?”
If users would regret taking the action, the technique fails the regret test and shouldn’t be built into the product, because it manipulated people into doing something they didn’t want to do. Getting people to do something they didn’t want to do is no longer persuasion — it’s coercion.
So how do we tell if people regret using a product? Simple! We ask them.
Just as companies test potential features they’re considering rolling out, they could test whether a questionable tactic is something people would respond to favorably if they knew what was going to happen next.
This testing concept isn’t new to the industry — product designers test new features all the time. But the regret test would insert one more ethical check by asking a representative sample of people if they would take an action knowing everything the designer knows is going to happen.
The test wouldn’t necessarily require much added effort or cost. In a recent article, Jakob Nielsen of the Nielsen Norman Group wrote that he believes usability test results can come from testing with as few as five people.
The history of technological innovation involves many unintended consequences. As the cultural theorist Paul Virilio once said, “The invention of the ship was also the invention of the shipwreck.” The beautiful thing about the regret test is that it could help weed out some of those unintended consequences, putting the brakes on unethical design practices before they go live to millions of users.
The regret test could also be used for regular check-ins. Like many people, I’ve uninstalled distracting apps like Facebook from my phone because I regret having wasted time scrolling through my feed instead of being fully present with the people I care about. Wouldn’t it be in Facebook’s interest to know about people like me?
If any company, be it Facebook or another business, doesn’t listen to users who increasingly resent it for one reason or another, it risks more people ditching its service altogether. And that’s exactly why understanding regret is so important. Ignoring people who regret using your product is not only bad ethics, it’s also bad for business.
Nir’s Note: Thank you to Jason Amunwa, Rafael Arizaga Vaca, Ahmed Bouzid, Jamie Kimmel, Julie Li, Jennifer McDonald, Bo Ren, Irina Raicu, Julian Shapiro, Shannon Vallor, AnneMarie Ward, Susan Weinschenk, Guthrie Weinschenk, and Casey Winters for reading versions of this essay.
Illustrations by John Devolle
Top Consumer Psychology Articles
- The One Fitness App That Hooked Me For Good
- Here's How Fortnite 'Hooked' Millions
- How Apps Can Shape Your Future Self
- How Netflix's Customer Obsession Created a Customer Obsession
- Want to Design User Behavior? Pass the 'Regret Test' First
- How to Trigger Product Usage that Sticks
- How to Get People to Help Each Other, Online and Off
- Here's How Amazon's Alexa Hooks You
- How to Use Psychology to Make Persuasive Video
- How to Use Personality Science to Drive Online Conversions
- The Unbelievable Future of Habit-Forming Technology
- The Secret Marketing Power of Evolutionary Psychology
- Don’t Ask People What They Want, Watch What They Do
- How Cognitive Biases Can Help (and Hurt) Your Business
- What Most People Don't Know About Behavioral Design
- How to Start a Career in Behavioral Design
- Your World is Full of Placebo Buttons (and That's a Good Thing)
- How to Build Technology that Feels Like a Friend
- 3 Pillars of the Most Successful Tech Products
- Here's How to Ethically Manipulate Other People
- How Two Companies Hooked Customers On Products They Rarely Use
- How to Hook Users in 3 Steps: An Intro to Habit Testing
- Die Dashboards, Die! Why Conversations Will Reinvent Software
- The Secret to Sending Emails and Notifications That Work
- How to Win Your Competition’s Customers
- Hooked for Good: How Habit-Forming Products Improve Lives
- Good Products Start With Good Questions
- Human + A.I. = Your Digital Future
- Why 'Assistant-As-App' Might Be the Next Big Tech Trend
- People Don’t Want Something Truly New, They Want the Familiar Done Differently.
- 4 Ways to Win Your Competitor's Customer Habits (Slides)
- Here’s Why You'll Hate the Apple Watch (and the Important Business Lesson You Need to Know)
- The Secret Psychology of Snapchat
- The Psychology of Notifications: How to Send Triggers that Work
- How Technology Tricks You Into Tipping More
- The Limits of Loyalty: When Habits Change, You're Toast
- 4 Ways to Use Psychology to Win Your Competition's Customers
- The Real Reason “Stupid” Startups Raise So Much Money
- The Psychology Behind Why We Can't Stop Messaging
- The Psychology of a Billion-Dollar Enterprise App: Why is Slack so Habit-Forming?
- Framing Reward is as Important as Reward Itself
- A Free Course on User Behavior
- It’s Not All Fun And Games: The Pros and Cons of Gamification at Work
- Getting Traction: How to Hook New Users
- Designing for Behavior Change Book Review
- The Sneaky Trick Behind the Explosive Growth of the Kardashian Game
- How Successful Companies Design for Users' Multi-Device Lives
- The Link Between Habits and User Satisfaction
- What Triggers The Best Word of Mouth Marketing?
- What Tech Companies Can Learn from Rehab
- The Secrets of Addictive Online Auctions
- Teach or Hook? What's the Real Goal of Online Education?
- Using Mind Control to Raise Startup Cash
- How To Build Habits In A Multi-Device World
- How To Cope with Your Insane Jealousy Of The WhatsApp Deal
- Why Do Fads Fade? The Inevitable Death Of Flappy Bird
- You'd Be Surprised By What Really Motivates Users
- Nostalgia: A Product Designer's Secret Weapon
- How You Can Help Users Change Habits
- Is “Lean Startup” Right for Your Idea?
- Hunting for Habits: Keying in on smart design to make a product irresistible
- Are Companies Too Obsessed With Growth? How to Measure Habits
- Refresh: The App a Secret Agent Would Love
- Angel or Devil: Who’s Really Investing In Your Start-Up?
- In 10 Years, We Won't Use Personal Technology
- 4 Simple Things I Did to Control My Bad Tech Habits
- "Yes, And": The Two Words that Created a #1 App
- From Laid to Paid: How Tinder Set Fire to Online Dating
- What if In-App Purchases Came to Real Life?
- Hooking Users One Snapchat at a Time
- How To Save Your Startup From The “Spotlight Effect”
- Bible App: Getting 100 Million Downloads is More Psychology Than Miracles
- How to Boost Desire Using the Psychology of Scarcity
- Marketplaces & The Curse of the Network Effect
- Today's Behaviors, Tomorrow's Startups
- Venture Capital and The Superstitious Investor
- The Future is Driven by Interface Changes
- Why Business is Addicted to Habits
- Viral Loops Or Viral 'Oops'?
- Making a Marketplace
- What Killed Turntable.fm?
- What You Don't Know About Human Intuition Can Hurt You
- Designing to Reward our Tribal Sides
- New Video - "Hooked: Building Habit-Forming Products"
- How Technology is Like Bug Sex
- Ways To Get People To Do Things They Don’t Want To Do
- The Network Effect Isn’t Good Enough
- Mass Persuasion, One User At A Time
- How Investment Drives Engagement (Slides)
- Getting Your Product Into the Habit Zone
- Where Have The Users Gone?
- Infinite Scroll: The Web’s Slot Machine
- Designing User Habits Video
- Psychology of Sports: How Sports Infect Your Brain
- This is Your Brain On Boarding: How to Turn Visitors Into Users
- User Investment: Make Your Users Do the Work
- Behavior by Design Video
- When Designing for Good Is Bad
- Stop Building Apps, Start Building User Behaviors
- The Morality of Manipulation
- The Next Secrets of the Internet
- User Growth and Engagement: A Hacker Metric
- Spotting the Next Facebook: Why Emotions are Big Business
- The Billion Dollar Mind Trick: An Intro to Triggers
- Why Everyone Hates I.T. People
- Hooking Users In 3 Steps: An Intro to Habit Testing
- Abolish The Reference Check
- Variable Rewards: Want To Hook Users? Drive Them Crazy
- How to Design Behavior (The Behavior Change Matrix)
- How To Design For “Normals”
- Hooks: An Intro on How to Manufacture Desire
- User Habits: Why Startups Must Be Behavior Experts
- What Is, and Is Not, Your Product's Job
- Pinterest’s Obvious Secret
- Personalized eCommerce Is Already Here, You Just Don’t Recognize It
- Where is the Web Going?
- The Developer Divide: When Great Companies Can't Hire
- Being a Quitter Makes You a Good Entrepreneur
- Behavior by Design
- Why You Should Run Your Business Barefoot
- Are you a Startup Star, Wacko, or Wannabe?