Hide Notes 

Slide 1: How to Measure Social Media Relations: The More Things Change, the More They Remain the Same by Katie Delahaye Paine CEO, KDPaine & Partners, LLC Member, Commission on Public Relations Measurement & Evaluation Published by the Institute for Public Relations April 2007 How to Measure Social Media Relations by Katie Delahaye Paine Copyright © 2007, Institute for Public Relations www.instituteforpr.org
Slide 2: 2 How to Measure Social Media Relations: The More Things Change, the More They Remain the Same Katie Delahaye Paine Katie Delahaye Paine is the founder of KDPaine & Partners LLC and publisher of The Measurement Standard and The One-Minute Benchmarking Bulletin, the first newsletters for marketing and communications professionals dedicated entirely to measurement and accountability. Prior to launching KDPaine & Partners in 2002, she was the founder and president of The Delahaye Group, which she sold to Medialink in 1999. She was an initial founder of the Institute for Public Relations’ Commission on PR Measurement & Evaluation. She was previously director of corporate communications for Lotus Development Corporation and manager of merchandising for Hewlett-Packard Personal Computer Group. A Cum Laude graduate of Connecticut College, she majored in history and Asian studies. She received an Honorary Doctorate of Laws from New Hampshire College. She is an Athena award winner and a Board member of New Hampshire Public Radio and the New Hampshire Political Library. Her life is featured in Mark Albion’s book “Making a Life, Making a Living.” She can be reached at kdpaine@kdpaine.com . INTRODUCTION To paraphrase Ken Kesey, in today’s media environment, the inmates are now in charge of the asylum. Thanks to advances in technology that have made it incredibly easy and virtually free to create content, both consumers and the media and everyone in between are creating content at unprecedented rates that is turning both the communications and PR measurement world upside down. The Internet has brought about a revolution in marketing far beyond the scope that even the most forward-thinking of us might have imagined. Today, despite the best efforts of PR and marketing types, consumers continue to seize power from the marketers. Mitch Kapor has described the Internet as the “ultimate democratic society - a truly chaotic universe.” As “The Cluetrain Manifesto” (Locke, Levine, Searls, Weinberger) and “Naked Conversations” (Scoble, Israel), so clearly point out, the consumers are the media, the editors and the reviewers. They are in control and they’re going to let you know what they think by changing their behavior. More and more people, be they journalists, pundits, experts or ordinary gadflies, are taking to the Internet to put forward their views to anyone who will listen and many are now wielding considerable influence over what consumers buy, think and do. WHY YOU SHOULD CARE A few facts: A new blog is created about once every two seconds. New videos are posted to YouTube even more frequently. Virtually everyone with a computer (98%) goes on line to search for information before making a purchase. Video search is taking over text search as the most popular form of searching. Politicians, marketers, and individuals are embracing new forms of social networking such as Second Life, Twittr, Flickr and other forms of content sharing at unprecedented levels. How to Measure Social Media Relations by Katie Delahaye Paine Copyright © 2007, Institute for Public Relations www.instituteforpr.org
Slide 3: 3 Most PR people envision the blogosphere as yet another new medium to address, a new way to scream more loudly at their stakeholders. In fact, PR people need to completely rethink their entire approach from pitching to engaging in “Naked Conversations.” The new reality is, as David Weinberger told the 2007 New Communications Forum gathering in Las Vegas, “There is no market for your message.” People now have access to so much content, and have so many ways to gather news and information, that the likelihood of your corporate message penetrating through the clutter is virtually nil. Instead, if you engage the audience in a conversation and learn what the social community is looking for, you might be able to persuade them to hear your message. Within this environment, PR researchers need to rethink their approaches as well. The normal maxim for measurement is, "If you can't measure it, you can't manage it." The problem with measuring blogs is not how to do it, but rather that the nature of blogs renders management impossible. You simply can't manage what 100 million independent-minded, opinionated people are going to say. And woe to those who try, since the blogosphere resembles nothing more than a cornered porcupine that will begin to throw darts the moment it sees someone trying to control it. That’s not to say that they can’t be influenced, just that it takes a new approach. The old command and control, top down message delivery is no longer an option. Dr. James Grunig’s Excellence Model of two way synchronous communications is the rule of the day. Consumers can now choose to accept or reject your messages, depending on whether they find them useful, interesting or relevant. And they’ll be more than happy to tell you what they like and don’t like. FIRST QUESTION, WHAT ARE YOU MEASURING? The first question is whether you need to measure you own blog, or are you trying to assess what others are saying about you in the blogosphere? The tools and techniques for measuring your own blog are typically financial in nature – assessing ROI, impact on sales or lead generation. On the other hand, measuring what consumers are saying about you in their blogs is not all that dissimilar to traditional media analysis. You are looking at the accumulated content of many blog postings and determining trends and tendencies based on that content. We will address each challenge individually. A. Measuring Your Own Blog Like any other form of communications, before you start trying to measure blogs you need to know what your objectives are. As with any other measurement program, there are essentially three things you can measure: outputs, outtakes and outcomes. 1. Outcomes: Measuring How Blogs Affect People's Behavior and Relationships Outcomes are defined by the Institute for Public Relations as (1) quantifiable changes in awareness, knowledge, attitude, opinion, and behavior levels that occur as a result of a public relations program or campaign; (2) an effect, consequence, or impact of a set or program of communication activities or products, and may be either short-term (immediate) or long term. How to Measure Social Media Relations by Katie Delahaye Paine Copyright © 2007, Institute for Public Relations www.instituteforpr.org
Slide 4: 4 In the blogosphere, outcomes can be financial or relational or transactional. Financial Outcomes: If the objective of your blog is financial -- i.e. to raise money (e.g. www.blogforamerica.com) or sell something (www.englishcut.com) -- the metrics and the math are very simple. What was the cost per click through, cost per sale, cost per lead or the cost per dollar raised? If the objective is not as directly commercial, e.g., you want to move people along the purchase cycle, you can measure the number of people who click through from a blog to your site. The percentage of all visitors who take action or click through is a fundamental measure of success. More importantly, if you factor in your budget, you can determine your cost per click through, which can be easily compared to other Web marketing tools. By assigning specific and unique URLs to links, it becomes easy to track the click through rate from individual blogs. To determine the efficiency, divide the cost of the program with the number of click throughs to get cost per click through. So if it costs you $120 a year to set up a blog and you spend an hour a day on it, and your time is worth $150 an hour, your cost for the year is $54,870. If the blog generates 50 click throughs a day or 18,250 a year, your cost per click is $3. Relationship Outcomes: A far more typical outcome of a blog, however, is to build relationships with your customers, your employees or your marketplace. In this case, the metrics are quite different. The strength and power of the blogosphere is in the networks it creates and the relationships you can form. In the traditional marketing space, you would survey your constituencies to determine the strength of their feelings towards your brands on issues like trust, satisfaction, commitment and control mutuality. However, the nature of the blogosphere is to eschew traditional marketing techniques in favor of far more direct interactions. That's not to say that some attempt to measure relationships shouldn't be applied. You could create a mechanism (a contest, free white paper, etc.) to capture the emails of people who follow your blog and then conduct an email survey using Dr. Linda Hon and Dr. James Grunig’s relationship measurement instrument (available from the Institute for Public Relations). Other outcomes: At the most basic level, if you are hosting your own blog, the server log files can tell you how many visitors there have been, how long they stayed and where they came from. Of course the term "visitors" must be taken with a grain of salt, since the technology behind determining a visitor is far from perfect, based on very generalized assumptions about human behavior. More sophisticated tools like ClickTracks and WebTrends, WebSideStory and Omniture provide far more data and unlike your basic log files, can display it in an understandable way. 2. Outtakes: Social Capital and Social Networking Measures According to the Institute for Public Relation's Dictionary of Public Relations Measurement and Research, prepared and edited by Dr. Don Stacks, outtakes are: (1) How to Measure Social Media Relations by Katie Delahaye Paine Copyright © 2007, Institute for Public Relations www.instituteforpr.org
Slide 5: 5 measurement of what audiences have understood and/or heeded or responded to a communication product's call to seek further information from PR messages prior to measuring an outcome; (2) audience reaction to the receipt of a communication product, including favorability of the product, recall and retention of the message embedded in the product, and whether the audience heeded or responded to a call for information or action within the message. There has been extensive research done by Robert Putnam and others on the value of social capital and social networks. In essence, for an individual, the more relationships you have, the better your life is, the longer you live and the healthier you are. For a company, good social capital means that information flows more easily, innovation and efficiency increase, and legal costs go down. You can extend this concept to the networks created by blogs. If a blog is generating favorable comments, engaging employees or customers in the business of the organization, and disseminating information quickly and accurately, it is contributing to the social capital of your organization. 3. Outputs: How Many People are Paying Attention to Your Blog? The Institute for Public Relations dictionary defines outputs as: (1) what is generated as a result of a PR program or campaign that impacts on a target audience or public; (2) the final stage of a communication product production process resulting in the production and dissemination of a communication product (brochure, media release, web site, speech, etc.); (3) the number of communication products or services resulting from a communication production process, the number distributed and/or the number reaching a targeted audience. Rankings: Rankings are the currency du jour in the blogosphere. The higher you are ranked, the more speaking engagements you get, the more influence you have. Because most rankings are based on the number of links to your site, rankings are a reflection of how important or interesting people find your site. There are a number of sites such as Technorati (www.technorati.com), BlogLines, and Kineda (http://www.kineda.com/are-you-an-a-list-bloglebrity) rank blogs in terms of their popularity. You may want to keep track of your rank over time as a measure of your blog’s popularity. The Conversation Index: Stowe Boyd (www.stoweboyd.com) developed this index as a way to measure the degree to which a blog was generating a conversation. You count the number of comments for each posting. A 1:1 ratio is acceptable. More comments per blog is better. No comments per blog is a pretty good indication that no one cares much for what you have to say. Simply counting the volume of conversations, comments and track backs is another indication of the size and scope of the network surrounding your blog. Whether or not those comments are in agreement or disagreement requires content analysis, but presumably positive or neutral comments would be indicative of a healthy relationship between the blogger and his/her audience. How to Measure Social Media Relations by Katie Delahaye Paine Copyright © 2007, Institute for Public Relations www.instituteforpr.org
Slide 6: 6 B. Measuring Your Presence on Other Peoples’ Blogs Assessing your reputation in the blogosphere – what other bloggers are saying about you – starts with the same basic parameters of outputs, outtakes and outcomes. 1. Outcomes There’s no doubt that a mention in an A-list blog can have both financial and reputational outcomes on your organization. Just ask Dell, whose customer service problems were brought to light by a blogger and the resulting outcome was a significant drop in the company’s stock price. More typically, financial outcomes take the form of increased traffic to your web site or increased leads. Reputational outcomes require a longer term integrated approach to measurement. First you need to make sure you understand what drives consumer behavior and purchase. Then, when you are analyzing how you are discussed in blogs, you should match those drivers against the reputational characteristics seen in those comments and postings. Finally, you need to keep close track of activity on your web site and correlate that against the various postings in the blogosphere that mention your brand. 2. Outtakes Think of the blogosphere as one enormous focus group with customers, prospects, employees and potential employees constantly sharing their thoughts with the world. The blogosphere gives you the opportunity to listen in on their conversations. As a result you should have a much better understanding of how your audiences are responding to your initiatives. The words shared in the blogosphere are an important source of outtake information. Content analysis of blogs should look for messages and theme to determine how your customers and constituencies perceive your organization or brand. How does the blogosphere position your brand on issues like employer of choice, value, or customer service? A good analysis will pull out recurring themes, complaints and messages, and quantify them to determine if they require action or can be ignored. However, just reading blogs is not a substitute for a well-crafted survey. A survey is better used to determine what they have internalized and taken away from all this chatter. 3. Outputs While it might give you a feel good feeling to know that your brand is being mentioned with increasing frequency in the blogosphere, it would be highly dangerous to simply assume this to be good news. Edelman did a lot better when it was one of the least mentioned PR firms in the blogosphere than when it soared to the top of everyone’s most talked about list. To determine the quality as well as the quantity of the discussion about your brand requires a thorough content analysis. How to Measure Social Media Relations by Katie Delahaye Paine Copyright © 2007, Institute for Public Relations www.instituteforpr.org
Slide 7: 7 Does this mean slogging through 1000 blogs a day? Probably not. You can generally cut down on the number of relevant blogs by making sure you search only those blogs with high authority. The easiest form of monitoring is to go to Technorati, BlogLines, Sphere or Google Blogs and see what people are saying about you. If that seems like too much work, there are several firms that scrape the blogosphere on a daily basis and will send you a daily update based on a selected set of search terms. One word of caution, however: only a small percentage may be relevant to you. The vast majority of conversation is teenage chatter that may or may not be relevant to your stakeholders. While a traditional media feed such as Factiva will deliver about 90% relevant content, the opposite is true with social media. Due to the limitation of automated content gathering, typically only about 10% will be relevant to the topic at hand. As with all media old and new, one needs to look beyond just quantity of postings to the quality of the dialog. Postings and comments in blogs can take many different forms. Some may be complaints about customer service, others may be speculation on stock price, and still others may be protests over personnel policies. So the next step in setting up a blog measurement system is to make a list of the various categories the postings fall into and to prioritize the categories. Are they all equally important, or are there some that are potentially more damaging or require faster action? In the media and in most news groups, the vast majority of what is said about a particular organization is neutral. But the unfettered and unfiltered nature of the blogosphere brings more opinions and frequently more negative opinions. Remember to step back as far as you can and remain objective. Think like your target audience. Just because someone leaked a piece of information or got a name wrong is not reason to respond or get involved in a discussion. Some standard criteria to look for include: • • • • Depth of Coverage: The number of times your brand or issue is mentioned within a posting. Dominance: Is the posting exclusively about your brand? Does the blogger go into the subject in depth with numerous links or is it just a passing mention? Subject: What was the primary topic of the blog posting? Tonality: Did the blog posting leave a reader more or less likely to do business with your organization? Did it make a recommendation or a specific “don’t buy this model” message? Positioning on key issues: Did the posting discuss any of the key issues facing your industry and if so, how did the blogger position your organization? Did the blog mention any specific benefits that would lead your audience to buy or not buy your product. How was your brand positioned on those benefits relative to the competition? Nature of the posting: Was the posting designed to solve a problem, compare different brands, or simply allow the author to rant? Who is being discussed or quoted? Is it your CEO or a low-level disgruntled employee? • • • • How to Measure Social Media Relations by Katie Delahaye Paine Copyright © 2007, Institute for Public Relations www.instituteforpr.org
Slide 8: 8 QUANTIFYING THE DATA There are three essential things that get measured in the blogosphere: links, hits/eyeballs, and sentiment. Let's talk about them individually. Links There are a plethora of services that measure links and rank the importance of various blogs including: Feedster (which maintains a ranking of the top 500 most interesting blogs), BlogPulse, BlogLines and Technorati. They all rank blogs by the number of links to each. Measuring Authority Ever since the first “A-list” blogger was crowned and Technorati put out its rankings, PR people have been begging for a way to measure authority. The generally accepted practice is to consider the number of links and track backs and comments, and roll them up to a ranking or authority index. The simplest thing to do is to look up the URL of the blog on Technorati and see what the rank is. A step up from that is a nifty widget on the Kineda site that when you type in a URL will tell you instantly if it is an A, B, C or D-list blogger – all based on Technorati. There are also a number of businesses who have created products or services around the need for measuring authority. Examining the credibility and authority of your own blog is another way to assess the impact of your blog. Mark Rogers of Market Sentinel has developed a "net promoter's index" that takes the number of bloggers that would recommend your brand, subtracts the number that would not recommend your brand, and comes up with an index number. Rogers claims that there is a direct correlation between the index and sales. If the number of detractors outnumbers the number of promoters, chances are your reputation is being trashed and sales will be impacted. Just ask Dell. Biz 360 just introduced another new metric, Media Signal, that looks at the positive, negative and neutral coverage in blogs, and then factors in links and connections with an index number to gauge the total impact of a blog. Another approach is to look at the specific industry and/or market and design an authority index around your particular business or market. This requires more in-depth bespoke research up front, but will yield more useful results in the long term. Measuring Visitors vs. Eyeballs As of this writing, there really is no accurate count for the number of eyeballs that view each blog. There are some statistics that show the number of visitors, but so far there's no way to exclude the visitors counted every time you or anyone else does a search, so most of those numbers are hugely overstated. It is safe to say that each link to a blog represents at least one pair of eyeballs. But there is no accurate count of how many people saw a blog posting that included the link. The owner of the blog itself and/or the blog host service has the data, but unless he/she shares it with the public, it remains an unknown. How to Measure Social Media Relations by Katie Delahaye Paine Copyright © 2007, Institute for Public Relations www.instituteforpr.org
Slide 9: 9 The dominant players in the eyeball counting industry, Comscore and Nielsen, get their data by tracking consumer behavior via software that is loaded onto millions of machines and reports back to the companies exactly where on the internet those machines travel. Because there are so many individuals involved in the study, the results are reasonably reliable for the vast majority of web sites that they track. While both of them have started to include a few major blogs as part of their panels, and you can sometimes figure out eyeball rates for the major blogs, the vast majority fall into the “too small to count” realm. If you wanted to count the eyeballs reached with a mention in a relatively specialized blog such as KDPaine’s measurement blog, you couldn’t do that unless the author gave you the analytics that Typepad provides. Because blogs change daily, and each new posting takes a reader to a new URL, combined with the fact that there are so many individual blogs and so many links and comments and track backs, a panel approach simply won’t work because no one is going back to the same page day after day. Comscore has just introduced an engagement metric, based on total visits, average minutes per visit, average visits per visitor and average visits per usage a day. While it is certainly an improvement over page views, and should help Web properties understand how users are engaging with their content, it appears that the focus of this metric is on web sites, not blogs. As far as comments go, they’re not even a glimmer in anyone’s eye. None of the automated systems track comments, and no one has a clue as to how many people are actually reading comments. WHAT TO DO WITH THE DATA ONCE YOU HAVE IT The most important part of any measurement program is teasing insight from the data and drawing actionable conclusions. The most important analysis is to look at trends over time. What happened yesterday or last week is important, of course, but what you need to do is to see if complaints are going up or down over time, or if your relationships are getting better or worse, or if the ranks of complainers is growing faster than the ranks of supporters. First, take a very deep breath. Do not -- do not -- go into crisis mode the first time you get a negative comment from the blogosphere. Do a bit of research first. Read the blogger's prior postings. See how many links he/she has, how many comments, how many track backs. If it's one or two, don't do anything but watch the numbers. If they start to grow quickly, you may have an emerging crisis. If it's already in the hundreds, and/or if this blog is on Feedster's top 500 list, then you need to come up with a response. If it's not a crisis, but there is someone who is consistently writing about you, take a wait-andsee attitude. See what kinds of comments are made, and how the blogger responds, then start a dialog. Offer information, a perspective or insight on something the blogger will find relevant. Do not spam bloggers! Generic press releases sent to bloggers will probably get you labeled as a "junk sender." Nothing you ever send will get through, ever again. Woe be it to the poor marketer who makes an obvious attempt to "manage" bloggers or somehow shield their company's reputation. The blogosphere is rife with snide comments and occasional downright hostility towards marketers' blundering attempts to interject themselves into a conversation. How to Measure Social Media Relations by Katie Delahaye Paine Copyright © 2007, Institute for Public Relations www.instituteforpr.org
Slide 10: 10 SO DO BLOGS REALLY MATTER AND WHAT DO I REALLY NEED TO DO? The quick answer is "Yes, they do matter." If you care about what your stakeholders are saying thinking or doing, you should be paying attention to social media. Even if your audience is limited in size, and bases its decisions on RFPs, specifications and the personal sales call, there is always a possibility that someone somewhere is having a problem. The thing you need to remember is that when problems occur, you want people to bring them to you, not have private conversations behind your back. Those are the situations that quickly get out of hand. One major reason to pay attention to what people are saying about you in social media is that most journalists today rely on blogs for story ideas, to check facts, track down rumors, and investigate scandals and rumors. If you sell computers, cars, consumer electronics, cell phones, printers or any number of consumer items that consumers research or talk about online, you need to pay attention. If you know that your customers are going online to do research before they decide what to buy, you need to know what those customers are seeing and reading about you. If you are still up in the air about whether or not blogs are important for you, conduct a quick poll of your audience and find out just how influential the blogosphere is. There's lots of generic research out there, but most organizations would be better off surveying their own customers to find out just how big an impact the consumer generated media blogosphere has. If there is definitely no clear tie between your organization's goals and the blogosphere, then exit out of this article and go learn about measuring more relevant media. (As in any communications activity, if it doesn't support a specific corporate goal, why are you doing it?) On the other hand, if developing a network of influencers around your product, your idea or your service is important, then a blog will be a very useful tool. NOW THAT I’M TRACKING ONLINE RELATIONSHIPS, DO I NEED TO START A BLOG? First of all, corporations don’t blog, people blog. Developing a corporate blog seems to be all the rage these days. For some organizations like GM and Sun with CEOs who like to write, have something to say and are dedicated to blogging, it makes perfect sense. GM and Sun want to get closer to their customers, so their CEOs started blogging as a way to encourage conversation with the customers. The point is that a blog is a dialog. A blog is not just a corporate Web site in a different dress. It is not just a marketing tool. It is a way to establish social networks that may of may not help sell products. But direct selling is not the point of a blog. And, in order to be effective you have to have something to say. There’s a lot of debate as to whether hiring a blogger to ghost write your CEOs blog is ethical. Whatever your opinion, it’s just not very effective. The reason blogs become popular is that they reflect the real personalities and values of the people writing them. No one reads a blog to get more corporate-speak. They read blogs to get the information BEHIND the corporate speak. How to Measure Social Media Relations by Katie Delahaye Paine Copyright © 2007, Institute for Public Relations www.instituteforpr.org
Slide 11: 11 CONCLUSION Regardless of what your organization does or who its constituencies are, the revolution that social media is creating will sooner or later have an impact. What that impact is, and how you respond will depend on data. Only by measuring this new social media will you be able to manage that impact. How to Measure Social Media Relations by Katie Delahaye Paine Copyright © 2007, Institute for Public Relations www.instituteforpr.org