Hide Notes 

Slide 1: Committed to making the world’s scientific and medical literature a public resource Article-Level Metrics at PLoS Peter Binfield, Publisher PLoS, pbinfield@plos.org www.plos.org
Slide 2: Outline • The Public Library of Science and Open Access • PLoS ONE • Article Level Metrics • Slides will be put at: http://everyone.plos.org (http://tiny.cc/ALM0) www.plos.org
Slide 3: What is Open Access (to the Journal Literature)? • Three Central Aspects: – Free, immediate access – Deposition in a digital public archive – Unrestricted reuse Bethesda definition, 2003 www.plos.org
Slide 4: Who are the Public Library of Science? • Six years old and the largest not-forprofit Open Access (OA) publisher • The publisher of 7 Open Access journals including the 3rd largest journal in the world (PLoS ONE) • Staffed by professional publishers from the likes of Nature, BMJ, Springer etc • Based in San Francisco, and Cambridge UK www.plos.org
Slide 5: OA Comes in Many Varieties. What is the PLoS ‘flavor’? • Not for Profit organisation – Mission driven • Online only – On an Open Source platform • ‘Free to submit’ but ‘pay to publish’ – However ability to pay does not affect ability to publish • Creative Commons Attribution license – Author retains full copyright – Full re-use is permitted, when credit is given • Free to read, download, re-use www.plos.org
Slide 6: And what is the alternative to OA? • Subscription based journals – Restricted Access to those who can pay, and therefore, by definition, restricted circulation – Restricted ownership (© held by publisher), and therefore restricted ability to re-use the content • Approx 25,000 journals – Publishing an average of about 50 papers/yr – Typically circulating to no more than 1,000 major libraries worldwide, and often far less • Approx $9 billion / yr business – 4 major publishers control about 40% of the literature www.plos.org
Slide 7: The Public Library of Science – our publishing strategy • Establish high quality journals – put PLoS and open access on the map • Build a more extensive OA publishing operation – an open access home for every paper • Make the literature more useful – to scientists and the public – accelerate science www.plos.org
Slide 8: PLoS Biology October, 2003 PLoS Medicine October, 2004 PLoS Community Journals June-September, 2005 October, 2007 PLoS ONE December, 2006 www.plos.org
Slide 9: PLoS ONE’s Key Innovation – The editorial process • Editorial criteria – – – – Scientifically rigorous Ethical Properly reported Conclusions supported by the data • Editors and reviewers do not ask – How important is the work? – Which is the relevant audience? • Use online tools to sort and filter scholarly content after publication, not before www.plos.org
Slide 10: What else is different? • Inclusive scope – all of science and medicine • Encouraging discussion and debate – on PLoS ONE: commenting, rating and annotation – elsewhere: Editorial Board discussion forum; EveryONE blog; Twitter; FriendFeed; Facebook • Streamlined production – publication on every weekday – online-only www.plos.org
Slide 11: PLoS ONE – statistics Year 2006* 2007 2008 2009** Annual Annual Submissions Publications % of Annual PubMed 473 2497 4401 6619** 138 1231 2723 4310** 0.02% 0.16% 0.34% 0.52%** * Started publishing Dec 20th, 2006 **Projections for 2009 • Third largest journal in the world in 2009 • Winner of the ALPSP Award for Publishing Innovation 2009 • 930 Academic Editors, 45,000 authors www.plos.org
Slide 12: Question… If you wanted to buy a good baseball team, would you buy one that consistently wins the World Series? If you wanted to buy a good journal, would you buy one that has a high Impact Factor? www.plos.org
Slide 13: Question… If a baseball team wins the World Series, does that mean that all their players are star performers? If a paper appears in Nature, does that mean that it was a high impact paper? www.plos.org
Slide 14: Answer… “89% of Nature’s 2004 impact factor was generated by just 25% of their papers” Phil Campbell, Editor in Chief of Nature www.plos.org
Slide 15: Question… www.plos.org
Slide 16: Is this Good Chocolate? Is this Bad Chocolate? www.plos.org
Slide 17: • Journals are just ‘packaging’ but many people confuse the packaging for the content • The content, the stuff you should really care about, is the article • Therefore the packaging is just one way of telling if the articles are likely to be any good • Is this a good method? Do you trust the packaging? Or would you rather taste the chocolate? www.plos.org
Slide 18: www.plos.org
Slide 19: PLoS ONE seeks to avoid this packaging effect • Specifically: – we publish anything that is scientifically sound (and therefore publishable); – we do not filter or ration the content (other than to reject ‘bad’ science or ‘bad’ content); – we get good content in front of the right audience as swiftly as possible; – and we allow the readership to determine what is useful to them as individuals (we don’t decide for them) www.plos.org
Slide 20: Evaluating the article after publication - putting research in context www.plos.org
Slide 21: Researchers Institutions Funders Who cares about measuring research impact? Librarians The public Authors Publishers www.plos.org
Slide 22: Given all those stakeholders, how do we measure impact of a paper and hence an individual? In the ‘real world’ theTH E worth of a paper tends to be M D judged on the OO basis of the impact factor of the G A journalTin which it was published. D! O NO Recommended reading: Adler, R., Ewing, J. Taylor, P. Citation statistics. A report from the International Mathematical Union. http://www.mathunion.org/publications/report/citationstatistics/ www.plos.org
Slide 23: So, how could we measure impact? • • • • • • • • Citations Web usage Expert Ratings Social bookmarking Community rating Media/blog coverage Commenting activity and more… Current technology now makes it possible to measure many of these with… www.plos.org
Slide 24: ‘Article-Level Metrics’ from PLoS • • • • • • Citations – Scopus, PubMedCentral, CrossRef Web usage – HTML, PDF, XML (COUNTER) Expert Ratings – none yet Social bookmarking – CiteULike, Connotea Community rating – user generated in 3 categories Media/blog coverage – Postgenomic, Nature Blogs, Bloglines • Commenting activity – Notes and Comments • and more… www.plos.org
Slide 25: Article-Level Metrics (at PLoS) • Article-Level Metrics at PLoS are not just about citations and usage. The concept refers to a whole range of additional measures which might provide insight into ‘impact’ • We are providing metrics at the article-level, for every article, in every one of our titles. • We are the first publisher to provide this range of data, but we hope that others will follow www.plos.org
Slide 26: Article-Level Metrics (at PLoS) • We have started out by acquiring and presenting the relevant data. • We have a preference for data that is not ‘owned’ by third parties, and we are seeking ‘open’ APIs onto the data (so that anyone else can verify or replicate these measures themselves). • ‘Phase 2’ will provide data analysis, filtering and navigation tools using this data. www.plos.org
Slide 27: (http://tiny.cc/ALM1)
Slide 30: www.plos.org
Slide 31: CrossRef Landing Page www.plos.org
Slide 32: www.plos.org
Slide 33: Ratings Landing Page www.plos.org
Slide 34: www.plos.org
Slide 35: Comments Landing Page www.plos.org
Slide 36: www.plos.org
Slide 37: citeulike Landing Page www.plos.org
Slide 38: www.plos.org
Slide 39: Postgenomic Landing Page www.plos.org
Slide 40: (http://tiny.cc/ALM2)
Slide 44: www.plos.org
Slide 45: www.plos.org
Slide 46: Downloading the data www.plos.org
Slide 47: Downloading the data http://www.plosone.org/static/plos-alm.zip (http://tiny.cc/ALM3)
Slide 49: Evaluating the data http://article-level-metrics.plos.org (http://tiny.cc/ALM4)
Slide 50: Evaluating the (usage) data
Slide 51: Evaluating the (usage) data
Slide 52: Evaluating the (usage) data
Slide 53: Evaluating the data From GitHub - http://plos-alm.opensci.info/articles_search
Slide 54: Evaluating the data http://manyeyes.alphaworks.ibm.com/manyeyes/visualizations?tag=plos&sort=rating (http://tiny.cc/ALM7)
Slide 55: Evaluating the data http://manyeyes.alphaworks.ibm.com/manyeyes/visualizations/plos-article-citations-per-day-col
Slide 56: Evaluating the data http://manyeyes.alphaworks.ibm.com/manyeyes/visualizations/plos-article-downloads-per-day
Slide 57: Evaluating the (commenting) data (http://tiny.cc/ALM5)
Slide 58: Evaluating the (commenting) data http://blogs.nature.com/wp/nascent/2009/02/commenting_on_scientific_artic.html
Slide 59: How Have Article Level Metrics been received? www.plos.org
Slide 60: How Have Article Level Metrics been received? www.plos.org
Slide 61: How Have Article Level Metrics been received? www.plos.org
Slide 62: How Have Article Level Metrics been received? “The PLoS article-level metrics are a substantial value-add for authors, including a range of download statistics, citations and social bookmarking data, and more. As an author, I would love to see this kind of service!” Heather Morrison, Librarian, British Colombia “Your innovation of the article level metrics is an extremely promising development in the evaluation of scientific publications. We are hopeful that this will transform the way that impact is assessed for published articles.” Dr Michael Markham, Neurobiology, University of Texas at Austin (PLoS ONE Author) “As paying customers of commercial publishers, should scientists and their funders be demanding more of this kind of information in the future? I reckon they should.” Dr. Duncan Hull, European Bioinformatics Inst, UK (Blogger) www.plos.org
Slide 63: How Have Article Level Metrics been received?
Slide 64: How Have Article Level Metrics been received? “As with so many other things, PLoS is to be entirely commended with this …kudos to PLoS” Dr Alan Cann, Univ. of Leicester (http://tiny.cc/ALM6)
Slide 65: Extending Article Level Metrics The ‘Frontiers’ Series of Journals (http://tiny.cc/ALM9)
Slide 66: Extending Article Level Metrics The Faculty 1000 service
Slide 67: Extending Article Level Metrics “Article level metrics are important, yes, but much more important for an Institutional Repository, is the ability to show Author level metrics, and Institution level metrics,” David Palmer, Systems Librarian, Univ of Hong Kong www.plos.org
Slide 68: Extending Article Level Metrics The Assoc for Computing Machinery (http://tiny.cc/ALM8)
Slide 69: Extending Article Level Metrics University of Tasmania, Institutional Repository
Slide 70: Next steps for article-level metrics at PLoS • More sources for each data type – e.g. blog coverage (researchblogging.org) • New data types – e.g. F1000, Mendeley • Provide data analysis tools • Not a PLoS-only initiative • Develop and adhere to standards – e.g. COUNTER, NISO www.plos.org
Slide 71: Article-Level Metrics We could be at the start of an important new development in academic publishing. To misquote from Jerry Maquire: “Show Me The Metrics!” Peter Binfield Publisher, PLoS ONE and the Community Journals pbinfield@plos.org www.plos.org