PLoS's picture
From PLoS rss RSS  subscribe Subscribe

ALM presentation for Science Commons 



Presentation to Science Commons salon, 20th Feb 2010. See: http://opensciencefoundation.com/scs/

 

 
 
Tags:  PLoS  Open Access  article-level metrics  PLoS ONE 
Views:  3704
Downloads:  7
Published:  February 20, 2010
 
0
download

Share plick with friends Share
save to favorite
Report Abuse Report Abuse
 
Related Plicks
Article-Level Metrics (at PLoS and beyond)

Article-Level Metrics (at PLoS and beyond)

From: PLoS
Views: 7755 Comments: 9
This was a webinar presented to NISO covering the way in which 'Article-Level Metrics' are being implemented by PLoS. Also see: http://everyone.plos.org/2009/05/27/article-level-metrics-at-plos/

Art (more)

 
Article Level Metrics from PLoS

Article Level Metrics from PLoS

From: PLoS
Views: 3242 Comments: 1
This was a presentation on the PLoS Article Level Metrics program (http://article-level-metrics.plos.org) to Berkeley and UCSF on 9th November, 2009
 
Article-level Metrics for UKSG 2010

Article-level Metrics for UKSG 2010

From: PLoS
Views: 4073 Comments: 0
The slides of a presentation to the UK Serials Group (UKSG) 2010 regarding the PLoS Article-Level Metrics
 
Let's Have an Awesome Time Publishing Science

Let's Have an Awesome Time Publishing Science

From: PLoS
Views: 4183 Comments: 3
This is a presentation, with audio, on PLoS ONE (http://www.plosone.org) and to some extent PLoS, given to a conference at UCSF called "Let's Have an Awesome Time Doing Science" (http://awesomescience.wordpress.com/) – an interesting blend of 'regul (more)

 
See all 
 
More from this user
Article Level Metrics from PLoS

Article Level Metrics from PLoS

From: PLoS
Views: 3242
Comments: 1

Let's Have an Awesome Time Publishing Science

Let's Have an Awesome Time Publishing Science

From: PLoS
Views: 4183
Comments: 3

The Future of Online (Academic) Publishing - Presentation to the ISMTE August 2009

The Future of Online (Academic) Publishing - Presentation to the ISMTE August 2009

From: PLoS
Views: 4528
Comments: 1

Article-Level Metrics (at PLoS and beyond)

Article-Level Metrics (at PLoS and beyond)

From: PLoS
Views: 7764
Comments: 9

Article-level Metrics for UKSG 2010

Article-level Metrics for UKSG 2010

From: PLoS
Views: 4073
Comments: 0

 
See all 
 
 
Creative Commons License  URL:          AddThis Social Bookmark Button
Embed Thin Player: (fits in most blogs)
Embed Full Player :
 
 

Name

Email (will NOT be shown to other users)

 

 
 
Comments: (watch)
 
 
Notes:
 
Slide 1: Committed to making the world’s scientific and medical literature a public resource PLoS ONE Peter Binfield, Publisher PLoS, pbinfield@plos.org www.plos.org
Slide 2: Outline • The Public Library of Science • Re-thinking the academic journal • Article-Level Metrics www.plos.org
Slide 3: Why are we all here today? • To accelerate and improve science • For the benefit of all And why am I here? • To discuss this in the context of scholarly publishing www.plos.org
Slide 4: Who are the Public Library of Science? • Six years old • A ‘web native’ academic journal publisher, producing ‘born digital’ content, using an ‘internet friendly’ business model • The largest not-for-profit Open Access publisher • The publisher of 7 Open Access journals • Staffed by professional publishers from the likes of Nature, BMJ, Springer etc • Based in San Francisco, and Cambridge UK www.plos.org
Slide 5: Re-thinking the Academic Journal? www.plos.org
Slide 6: The 4 traditional functions of a journal • Registration – Registering the primacy of your work • Certification – Some ‘seal of authority’ • Dissemination – People need to know it exists and be able to read it • Archiving – People need to find it in the future But journals also now… • Filter for ‘quality’ • Filter for topic (aka ‘scope’) www.plos.org
Slide 7: The 4 traditional functions of a journal • Registration – Registering the primacy of your work • Certification – Some ‘seal of authority’ • Dissemination People need to know it exists and be able to read it • Archiving – People need to find it in the future But journals also now… • Filter for ‘quality’ • Filter for topic (aka ‘scope’) www.plos.org
Slide 8: The 4 traditional functions of a journal • Registration – Registering the primacy of your work • Certification – Some ‘seal of authority’ • Dissemination People need to know it exists and be able to read it • Archiving – People need to find it in the future But journals also now… • Filter for ‘quality’ • Filter for topic (aka ‘scope’) www.plos.org
Slide 9: The 4 traditional functions of a journal • Registration – Registering the primacy of your work • Certification – Some ‘seal of authority’ • Dissemination People need to know it exists and be able to read it • Archiving – People need to find it in the future But journals also now… • Filter for ‘quality’ • Filter for topic (aka ‘scope’) www.plos.org
Slide 10: The 4 traditional functions of a journal • Registration – Registering the primacy of your work • Certification – Some ‘seal of authority’ • Dissemination People need to know it exists and be able to read it • Archiving – People need to find it in the future But journals also now… • Filter for ‘quality’ • Filter for topic (aka ‘scope’) www.plos.org
Slide 11: The 4 traditional functions of a journal • Registration – Registering the primacy of your work • Certification – Some ‘seal of authority’ • Dissemination People need to know it exists and be able to read it • Archiving – People need to find it in the future But journals also now… • Filter for ‘quality’ • Filter for topic (aka ‘scope’) www.plos.org
Slide 12: The 4 traditional functions of a journal • Registration – Registering the primacy of your work • Certification – Some ‘seal of authority’ • Dissemination People need to know it exists and be able to read it • Archiving – People need to find it in the future But journals also now… • Filter for ‘quality’ • Filter for topic (aka ‘scope’) www.plos.org
Slide 13: Certification • Peer review – – – – A pre-publication evaluation The opinion of a handful of people (typically 2) A confidential and therefore secretive process Often subjective and typically based on ill defined criteria – Potentially at risk of bias based on decisions which can have nothing to do with the science – Supposed to be about ‘the science’ but often distorted for other reasons, such as making quality judgments www.plos.org
Slide 14: The 4 traditional functions of a journal • Registration – Registering the primacy of your work • Certification – Some ‘seal of authority’ • Dissemination People need to know it exists and be able to read it • Archiving – People need to find it in the future But journals also now… • Filter for ‘quality’ • Filter for topic (aka ‘scope’) www.plos.org
Slide 15: Filtering for ‘quality’ Via peer review, the following process happens to a surprising number of papers: • • • • Submit, review, reject Revise, submit, reject Revise, submit, reject Repeat until successful… ‘Top Tier’ journal at the next ‘tier’ down at the next ‘tier’ down accepted at journal X! Congratulations (?), the paper found a home but… • How long did it take? How many people had to work on this paper? How much time and opportunity cost was wasted? • We filtered. But was this even a good way to do the filtering? Or are we suffering from ‘filter failure’? www.plos.org
Slide 16: Case History: Paper 1 vivo magnetic resonance microscopy of differentiation in Xenopus laevis embryos from the first cleavage onwards “ “In • Submitted to Nature in 2003 – Rejected as out of scope • Submitted to 5 more journals – Repeatedly rejected – Finally the authors split the paper into two, before someone would publish them – The most recent journal which rejected as ‘out of scope’ actually published a competing paper soon after • The two halves were finally published in 2006, in 2 different journals • One of the papers made the cover of it’s journal • Almost 4 years to get published… (http://tiny.cc/ALM10) www.plos.org
Slide 17: Case History: Paper 2 “An Analysis of the Feasibility of Short Read Sequencing” • Rejected by Nature, Science, Nature Biotech, Nature Chemical Biology • As a result, it went through multiple rewrites and got cut in half • One half published 18 months later in Nucleic Acids Research – It now has almost 80 citations – The author claims that earlier publication could have significantly advanced the field • The other half was rejected by 17 (!) journals – Four years later, it was eventually published in “Complex Systems”, a journal which isn’t even online www.plos.org
Slide 18: In what way did this process… • accelerate and improve science? • for the benefit of all? And who actually benefited? • Certainly not the authors, • not society, • and, I would say, not science. www.plos.org
Slide 19: So, what is the answer? • PLoS ONE (of course!) • Open Access – widest possible dissemination • Online only – no size limitations, no topic or scope limitations • ‘Publication Fee’ Business Model – scalable business model www.plos.org
Slide 20: PLoS ONE • Our peer review asks the ‘right’ questions – Is it publishable? Is the science sound? • Aside from basic criteria, there is no filtering for ‘quality’ – Everything that passes peer review, and is deemed ‘publishable’, is published – We get good science, in front of the right audience, as fast as possible and with the minimum of ‘hassle’ www.plos.org
Slide 21: And does it work? • Our growth is unparalleled in the history of the industry – 4 years old & the largest journal in the world in 2010 Year 2006 2007 2008 2009 Annual Submissions Annual Publications % of Annual PubMed 473 2497 4401 6819 138 1231 2723 4404 0.02% 0.16% 0.34% 0.52% • Incredible community acceptance – 50,000 authors, 1,000 academic editors • We are promoting a paradigm shift – from the journal to the article • We believe we are accelerating science www.plos.org and the Scientific Process
Slide 22: Evaluating the article after publication - putting research in context www.plos.org
Slide 23: www.plos.org
Slide 24: So, how could we measure ‘impact’? At the article level, we could track: • • • • • • • • Citations Web usage Expert Ratings Social bookmarking Community rating Media/blog coverage Commenting activity and more… Current technology now makes it possible with… www.plos.org
Slide 25: ‘Article-Level Metrics’ from PLoS • • • • • • Citations – Scopus, PubMedCentral, CrossRef Web usage – HTML, PDF, XML (to COUNTER standards) Expert Ratings – none yet Social bookmarking – CiteULike, Connotea Community rating – user generated in 3 categories Media/blog coverage – Postgenomic, Nature Blogs, Blogline, Researchblogging.org • Commenting activity – Notes, Comments, replies A basket of individual metrics, all informative at some level and collectively hard to ‘game’ www.plos.org
Slide 26: Article-Level Metrics (at PLoS) • Article-Level Metrics at PLoS are not just about citations and usage. The concept refers to a whole range of additional measures which might provide insight into ‘impact’ • We are providing metrics at the article-level, for every article, in every one of our titles. • ALMs are not just about quality evaluation, they are also a way to filter and discover content • We are the first publisher to provide this range of data, but we hope that others will follow www.plos.org
Slide 27: (http://tiny.cc/ALM1)
Slide 30: www.plos.org
Slide 31: CrossRef Landing Page www.plos.org
Slide 32: www.plos.org
Slide 33: Ratings Landing Page www.plos.org
Slide 34: www.plos.org
Slide 35: Comments Landing Page www.plos.org
Slide 36: www.plos.org
Slide 37: citeulike Landing Page www.plos.org
Slide 38: www.plos.org
Slide 39: Postgenomic Landing Page www.plos.org
Slide 40: How Have Article Level Metrics been received? “The PLoS article-level metrics are a substantial valueadd for authors … As an author, I would love to see this kind of service!” Heather Morrison, Librarian, British Colombia “Your innovation of the article level metrics is an extremely promising development in the evaluation of scientific publications. We are hopeful that this will transform the way that impact is assessed for published articles.” Dr Michael Markham, Neurobiology, University of Texas at Austin (PLoS ONE Author) www.plos.org
Slide 41: How Have Article Level Metrics been received? “It is interesting to note that of our two previous PLoS papers where similar technology has been utilized, the level of interest was above average for both of them (see Appendix 1 – data from respective PLoS journals). Thank you for your consideration of this manuscript.” An author, submitting to PLoS “As paying customers of commercial publishers, should scientists and their funders be demanding more of this kind of information in the future? I reckon they should.” Dr. Duncan Hull, European Bioinformatics Inst, UK (Blogger) www.plos.org
Slide 42: Evaluating the data GitHub - http://plos-alm.opensci.info/articles_search http://tiny.cc/ALM11
Slide 43: Evaluating the data http://manyeyes.alphaworks.ibm.com/manyeyes/visualizations?tag=plos&sort=rating (http://tiny.cc/ALM7)
Slide 44: Evaluating the data http://manyeyes.alphaworks.ibm.com/manyeyes/visualizations/plos-article-citations-per-day-col
Slide 45: Evaluating the data http://manyeyes.alphaworks.ibm.com/manyeyes/visualizations/plos-article-downloads-per-day
Slide 46: Evaluating the (commenting) data (http://tiny.cc/ALM5)
Slide 47: Evaluating the (commenting) data http://blogs.nature.com/wp/nascent/2009/02/commenting_on_scientific_artic.html
Slide 48: Extending Article Level Metrics The ‘Frontiers’ Series of Journals (http://tiny.cc/ALM9)
Slide 49: Extending Article Level Metrics “Article level metrics are important, yes, but much more important for an Institutional Repository, is the ability to show Author level metrics, and Institution level metrics,” David Palmer, Systems Librarian, Univ of Hong Kong www.plos.org
Slide 50: Extending Article Level Metrics The Assoc for Computing Machinery (http://tiny.cc/ALM8)
Slide 51: What Metrics are ‘Missing’? • • • • • • Predictive (day one) metrics Expert Ratings Media coverage More sophisticated types of usage metrics Tracking ‘conversations’ outside of the publisher Reputation metrics for commenters www.plos.org
Slide 52: What Work Still Needs to be Done? • • • • • • Filtering, Navigation tools, and an API (at PLoS) More data sources providing more complete data Tracking new metrics? De-duplication? Expert analysis of the data Standards – NISO? • Community understanding / adoption of the metrics – One Number (to Bind Them All)? – Being valued by tenure committees – Being quoted in resumes • Broad adoption by publishers – Or a third party? CrossRef? www.plos.org
Slide 53: Article-Level Metrics We could be at the start of an important new development in academic publishing. To misquote from Jerry Maquire: “Show Me The Metrics!” Peter Binfield Publisher, PLoS ONE and the Community Journals pbinfield@plos.org www.plos.org

   
Time on Slide Time on Plick
Slides per Visit Slide Views Views by Location