Hide Notes 

Slide 1: Lean Product Management for  Web 2.0 Products Dan Olsen, CEO, YourVersion O’Reilly Web 2.0 Expo SF May 6, 2010 Copyright © 2010 YourVersion
Slide 2: Lean Product Management Lean: iterating and improving quickly in a  resource‐efficient manner Product Management: role responsible for  achieving product‐market fit Lean Product Management: achieving  product‐market fit quickly in a resource‐ efficient manner Will post slides to slideshare.net/dan_o Copyright © 2010 YourVersion
Slide 3: What’s So Great about “Lean”? What’s wrong with  being not‐so‐lean? Startups are at risk until  they’re profitable Funding cocoon only lasts  so long Limited resources Tech markets move fast Time is the real enemy “Time is the scarcest  resource and unless it is  managed nothing else can  be managed.” ‐ Peter Drucker Copyright © 2010 YourVersion
Slide 4: My Background Education BS, Electrical Engineering, Northwestern MS, Industrial Engineering, Virginia Tech MBA, Stanford UI design and web development 18 years of Product Management Experience Managed submarine design for 5 years 5 years at Intuit, led Quicken Product Management Led Product Management at Friendster PM consultant to startups: Box.net, YouSendIt, Epocrates CEO & Cofounder of YourVersion, startup building  “Pandora for your real‐time web content” Copyright © 2010 YourVersion
Slide 5: What’s the Formula for Product‐Market Fit? A product that: Meets customers’ needs Is better than other alternatives Is easy to use Has a good value/price Simple, right? It’s easy to understand conceptually  what we want to achieve HOW we achieve it is the hard part Copyright © 2010 YourVersion
Slide 6: Understanding Customer Needs Copyright © 2010 YourVersion
Slide 7: Problem Space vs. Solution Space Problem Space A customer problem,  need, or benefit that the  product should address A product requirement Solution Space A specific  implementation to  address the need or  product requirement Example: Ability to write in space  (zero gravity) NASA: space pen ($1 M R&D cost) Russians: pencil Copyright © 2010 YourVersion
Slide 8: Problem Space vs. Solution Space Product Level Problem Space (user benefit) Prepare my taxes Solution Space (product) Pen and paper TurboTax File my taxes TaxCut Copyright © 2010 YourVersion
Slide 9: Problem Space vs. Solution Space Feature Level Problem Space (user benefit) Make it easy to share a link with my friends Design #1 Solution Space (feature) Gmail importer Design #2 Design #3 Allow me to reuse my email contacts Design #1 #2 #3 Preview with  User can edit  checkboxes before import No Yes Yes No No Yes Copyright © 2010 YourVersion
Slide 10: How Do You Prioritize User Benefits  and Product Features? Need a framework for prioritization Which user benefits should you address? Which product features to build (or improve)? Importance vs. Satisfaction Importance of user need (problem space) Satisfaction with how well a product meets the  user’s need (solution space) Opportunity = High Importance need with low Satisfaction Copyright © 2010 YourVersion
Slide 11: High Importance + Low Satisfaction = Opportunity Importance of User Need High Opportunity Competitive Market Not Worth Going After Low Low High Copyright © 2010 YourVersion User Satisfaction with Current Alternatives
Slide 12: Importance vs. Satisfaction Ask Users to Rate for Each Feature 100 95 90 85 Importance 80 75 70 65 60 55 50 40 50 60 70 Satisfaction 80 90 100 41 Great 84 87 98 Bad 55 70 72 75 79 80 80 86 84 Recommended reading: “What  Customers Want” by Anthony Ulwick Copyright © 2010 YourVersion
Slide 13: Kano Model: User Needs & Satisfaction User Satisfaction Delighter (wow) Performance  (more is better) Need not met Must Have Needs & features  migrate over time Need fully met User Dissatisfaction Copyright © 2010 YourVersion
Slide 14: Olsen’s Hierarchy of Web User Needs (adapted from Maslow) Customer’s Perspective How easy to use is it? Does the functionality meet my needs? Does the functionality work? Decreasing Dissatisfaction Increasing Satisfaction What does it mean to us? Usability & Design Feature Set Absence of Bugs Is the site fast enough? Page Load Time Is the site up when I want to use it? Uptime Copyright © 2010 YourVersion
Slide 15: What is Your Value Proposition? Which user benefits are you providing? How are you better than competitors? Competitor A Competitor B Must Have Benefit 1 Performance Benefit 1 Performance Benefit 2 Performance Benefit 3 Delighter Benefit 1 Delighter Benefit 2 Y High Low Med Y ‐ Y Low High Med ‐ ‐ You Y Med Low High ‐ Y Copyright © 2010 YourVersion
Slide 16: Prioritization and Scope Customer value is only half the equation How much engineering effort will it take? Need to consider value and effort (ROI) Ruthlessly prioritize: rank order Be deliberate about scope & keep it small It’s easy to try to do too much Strategy = deciding what you’re NOT doing Break features down into smaller chunks Smaller scope → faster iterations → better Copyright © 2010 YourVersion
Slide 17: Prioritizing Product Ideas by ROI Return (Value Created) 4 3 Idea A Idea B ? Idea D 2 Idea C 1 Idea F 1 2 3 4 Investment (developer‐weeks) Copyright © 2010 YourVersion
Slide 18: UI Design and Ease of Use Copyright © 2010 YourVersion
Slide 19: User Benefits vs. Ease of Use Q: If two products equally deliver the exact  same user benefits, which product is better? A: The product that’s easier to use “Ease of use” provides benefits Saves time Reduces cognitive load Reduces frustration UI Design can be differentiator Olsen’s Law: “The less user effort required, the  higher the percentage of users who will do it” Copyright © 2010 YourVersion
Slide 20: The UI Design Iceberg What most people see and react to Visual Design Interaction Design Information Architecture Conceptual Design Recommended reading: Jesse James Garrett’s “Elements of User Experience” chart, free at www.jjg.net What good product people think about Copyright © 2010 YourVersion
Slide 21: Elements of User Interface Design Consists of Three Distinct Elements: Information Architecture Structure and layout at both site and page level How site is structured (sitemap) How site information is organized (site layout) How each page is organized (page layout) Interaction Design How user and product interact with one another User flows (e.g., navigation across multiple pages) User input (e.g., controls and form design) Visual Design “How it looks” vs. “What it is”, often called “chrome” Fonts, colors, graphical elements Copyright © 2010 YourVersion
Slide 22: Learning from Customer Feedback Copyright © 2010 YourVersion
Slide 23: Iterating Your Product Vector Based on  User Feedback in Solution Space Problem Space (your mental model) Help user book travel Help user plan travel Solution Space (what users can react to) Mockups / Code Customer Feedback Copyright © 2010 YourVersion
Slide 24: Persevere or Pivot? Increasing Product-Market Fit Pivot Product-Market Fit = Getting enough data to validate that you’re climbing up the right mountain Pivot Copyright © 2010 YourVersion
Slide 25: What Are You Getting Feedback &  Learnings About? Problem Space (your mental model) Solution Space (what users can react to) Feature Set Customer Understanding  (needs &  preferences) UI Design Messaging  Copyright © 2010 YourVersion
Slide 26: What Can Solution Space Objects Can You Test with Customers? Range of solution space options: Wireframe: low‐fidelity graphics Mockup: high‐fidelity graphics Prototype: interactive graphics or code Alpha product: production code Graphics usually quicker/cheaper to  change than code Goal is to gain learnings quickly Copyright © 2010 YourVersion
Slide 27: The Value of User Feedback Sessions Critical to talk with customers 1‐on‐1 Gain better understanding of Customer needs and problems Issues with your feature set, UI, messaging What alternatives customers are using, pros & cons of each, customer preferences QA: use cases & bugs you haven’t seen Really a “user learning” session Make test as real for user as possible Copyright © 2010 YourVersion
Slide 28: “Ramen” User Feedback for Startups Anyone can do it! Ingredients: Solution‐space product/mockup to test 1 customer (with laptop if testing code) 1 desk 1 person to conduct the session Pen and paper Optional note‐taker and observers Copyright © 2010 YourVersion
Slide 29: Typical Format for Customer Session 5 ‐ 10 min: Ask questions to understand user  needs and solutions they currently use 30 ‐ 50 min: User feedback Show user product/mockup Non‐directed as much as possible When necessary, direct user to attempt to  perform a specific task 5 ‐ 10 min: Wrap‐up Answer any user questions that came up Point out/explain features you want to highlight Ask them if they would use the product Copyright © 2010 YourVersion
Slide 30: Dos & Don’ts of Conducting Usability Do Explain to the user: Their usability test will help improve the product Not to worry about hurting your feelings “Think Aloud Protocol” Ask user to attempt the task, then be a fly on the wall Ask non‐leading, open‐ended questions Take notes and review them afterwards for take‐aways Don’t Ask leading questions “Help” the user or explain the UI (e.g., “click over here”) Respond to user frustration or questions (until test is over) Get defensive Blame the user Copyright © 2010 YourVersion
Slide 31: Case Study on Product‐Market Fit: MarketingReport.com Copyright © 2010 YourVersion
Slide 32: Product‐Market Fit Case Study: MarketingReport.com My consulting client, CEO of TrustedID,  had an idea for a new product Team: me, CEO, head of marketing, UI design consultant Goal: Validate product‐market fit quickly, cheaply  without writing a single line of code Determine if their was a business  opportunity here Copyright © 2010 YourVersion
Slide 33: Product‐Market Fit Case Study: Developing Product Concept Product Concept was “marketing report” that let  consumers control the direct mail that they receive Concept was fuzzy with various components, so we  broke it into 2 different “flavors”: #1 “Marketing Shield”: Service to reduce/stop junk mail #2 “Marketing Saver”: Opt in & receive money‐saving offers Each product concept consisted of several modules that  each mapped to a specific user benefit Worked with UI designer to create paper mockups of  pages for each flavor concept (5 pages each) Copyright © 2010 YourVersion
Slide 34: Copyright © 2010 YourVersion
Slide 35: Copyright © 2010 YourVersion
Slide 36: Clustering Potential User Benefits to  Create Product Concepts “Shield” Concept “Saver” Concept Reduce Junk Mail Find out what “they” know about you Money Saving Offers Compare Yourself to Others Social Networking Save Trees Marketing Report Marketing Score Marketing Profile Copyright © 2010 YourVersion
Slide 37: Product‐Market Fit Case Study:  Recruiting People Telephone recruit of prospective customers Wrote phone‐screen questionnaire to create rough  target customer segmentation Wanted users who work full‐time & use internet Fit for opt‐in concept: use coupons, Costco membership Fit for anti‐junk mail concept: use paper shredder, block  caller ID Scheduled 3 groups of 2 or 3 people to discuss  each product concept for 90 minutes Moderated each group through the paper  mockups to hear their feedback Copyright © 2010 YourVersion
Slide 38: Product‐Market Fit Case Study: Findings on Concepts &User Benefits “Shield” Concept “Saver” Concept Reduce Junk Mail Find out what “they” know about you Money Saving Offers Compare Yourself to Others Social Networking Save Trees Marketing Report Legend Marketing Score Marketing Profile Strong appeal Somewhat positive Low appeal Copyright © 2010 YourVersion
Slide 39: Product‐Market Fit Case Study: Learnings from Research Learned that “Shield” (anti‐junk mail) concept was  stronger than “Saver” People didn’t like many of the “Saver” concept  components Learned users’ concerns / questions about “Shield” concept Refined “Shield” concept: Removed irrelevant components Improved messaging to address user concerns / questions Validated revised “Shield” concept with quick 2nd round of tests No customer concerns Clear willingness to pay Copyright © 2010 YourVersion
Slide 40: Copyright © 2010 YourVersion
Slide 41: Product‐Market Fit Case Study: Summary 4 weeks from 1st meeting to validated  product concept Cost $1,500 to talk with 20 users ($75  each) 1 round of iteration on product concept Identified compelling concept that users  are willing to pay $10/month for Trimmed away non‐valuable pieces You can achieve similar results Copyright © 2010 YourVersion
Slide 42: Getting Quantitative: Optimization Using Metrics Copyright © 2010 YourVersion
Slide 43: Approaching Your Business as an  Optimization Exercise Given reality as it exists today, optimize our business results subject to our resource constraints. Copyright © 2010 YourVersion
Slide 44: Define the Equation of your Business “Peeling the Onion” Advertising Business Model: Profit = Revenue ‐ Cost Unique Visitors x  Ad Revenue per Visitor Impressions/Visitor x  Effective CPM / 1000 Visits/Visitor  x  Pageviews/Visit  x  Impressions/PV New Visitors + Returning Visitors Invited Visitors + Uninvited Visitors # of Users Sending Invites  x  Invites Sent/User  x  Invite Conversion Rate Copyright © 2010 YourVersion
Slide 45: Equation of your Business Subscription Business Model Profit = Revenue ‐ Cost Paying Users x  Revenue per Paying User New Paying Users +  Repeat Paying Users Trial Users x  Conv Rate Previous Paying Users  x  ( 1 – Cancellation Rate ) ( SEO Visitors + SEM Visitors + Viral Visitors )  x  Trial Conversion Rate Copyright © 2010 YourVersion
Slide 46: How to Track Your Metrics Track each metric as daily time series Date 4/24/08 4/25/08 … Unique  Visitors 10,100 10,500 Page  views 29,600 27,100 Ad  Revenue 25 24 New User  Sign‐ups 490 480 … Create ratios from primary metrics:  X / Y Example: How good is your registration page? Okay: # of registered users per day Better: registration conversion rate = # registered users / # uniques to reg page Copyright © 2010 YourVersion
Slide 47: Sample Signup Page Yield Data Daily Signup Page Yield vs. Time New Registered Users divided by Unique Visitors to Signup Page 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 1/31 2/14 2/28 3/14 3/28 4/11 4/25 5/9 5/23 6/6 6/20 7/4 7/18 8/1 8/15 8/29 9/12 9/26 10/1 0 Changed messaging Added questions to signup page Started requiring registration Daily Signup Page Yield Copyright © 2010 YourVersion
Slide 48: Identifying the  “Critical Few” Metrics What are the metrics for your business? Where is current value for each metric?  How many resources to “move” each metric? Developer‐hours, time, money Which metrics have highest ROI opportunities? Metric A Good ROI Return Return Metric B Bad ROI Return Metric C Great ROI Investment Investment Investment Copyright © 2010 YourVersion
Slide 49: Metrics to Validate Product‐Market Fit Survey results Importance & Satisfaction Net Promoter Score Survey.io “How would you feel if you could no longer use Product X?” Very disappointed, Somewhat disappointed, Not disappointed User behavior Prospects sign up (high conversion rate) They keep using it (high retention rate) They use it often (high frequency of use) Copyright © 2010 YourVersion
Slide 50: Continuous Improvement Copyright © 2010 YourVersion
Slide 51: Adding Metrics and Optimization to  your Product Process Plan Business Objectives Product Objectives Prioritized  Feature List Site Level Scoping Feature  Level Design Develop Optimize Requirements  & Design Code Test Launch Metrics & User  Feedback Copyright © 2010 YourVersion
Slide 52: Optimization through Iteration: Continuous Improvement Measure the metric Analyze the metric Identify top  opportunities to improve Design & develop   the enhancement Learning Gaining knowledge: • Market • Customer • Domain • Usability Launch the enhancement Copyright © 2010 YourVersion
Slide 53: Lean Product Management Cheat Sheet Clarify problem space by iterating in the  solution space & getting user feedback Revise feature set, UI design, and  messaging to improve product‐market fit Ruthlessly prioritize based on ROI Define equation of your business Identify and track key metrics Launch, learn, and iterate Copyright © 2010 YourVersion
Slide 54: www.yourversion.com ‐“Pandora for your real‐time web  content” ‐Aggregates latest news, blogs,  twitter, etc. tailored to your  specific interests ‐Free iPhone App, browser tools ‐Intimate, all‐day workshop  for startups on Product  Management, Design, &  Marketing toolboxevent.com ‐Sat June 12th in Palo Alto Copyright © 2010 YourVersion
Slide 55: Questions? @danolsen dan@yourversion.com www.yourversion.com Copyright © 2010 YourVersion