Scientific communication is evolving rapidly with new technologies, challenging long-established dogma within the publication system. The recent rise in the use of preprints in the biomedical sciences has opened up a discussion on the value and/or necessity of prepublication peer review as a quality control mechanism. Nevertheless, there is currently no empirical data on how preprints measure up to peer-reviewed articles in terms of quality to solve this debate. Although scientific quality has many dimensions, a relatively simple one to access is quality of reporting of methods and results – something that can be gauged by objective metrics and should be one of the most accessible features for improvement by peer review. With this in mind, we developed a crowdsourced initiative to compare quality of reporting between preprints and peer-reviewed articles in the life sciences. We developed and registered a protocol composed of objective questions to measure quality of reporting, and subsequently recruited evaluators within the biomedical community as volunteers to evaluate random samples of articles in bioRxiv and PubMed. We are currently working with 14 evaluators across the globe, and should have final results for our predefined sample size of 76 articles per group available in the middle of the year. This would put us in an ideal position to present the results on the FORCE2018 Meeting, with a project that is both relevant to the central interests of the FORCE11 community and to the specific theme of engaging researchers through a crowdsourced approach to solve a common problem. More information on the project can be assessed at http://asapbio.org/amaral-quality and at https://osf.io/rxqn4/. Conflict of interest statement: I am currently an ambassador for ASAPbio, a scientist-driven non-profit organization for promoting transparency and innovation in life sciences communication.
