The STAR Methods way towards reproducibility and open science

Abstract

The original concept of iScience included the word integrity as one of the 5 “i's” that defined the journal. Truthful to this mission, we have published transparent methods without any length restrictions, to give authors plenty of space to describe what they have done in a reproducible manner. Much has been written about reproducibility (and replicability) in science, especially in recent years, for example by the National Academy of Sciences of the United States of America. Yet comprehensive standards for reporting across disciplines are still uncommon. Without sufficient disclosure of the details of a research project, the crucial “validation” phase of any open science pipeline is all but bound to result in findings which are not reproducible, and in the frustration and loss of precious time and resources for many researchers at all levels of seniority. As a multidisciplinary, and interdisciplinary, scientific journal team, we feel that it is now time to take our commitment to reproducibility and open science one step further, and to join the life sciences journals within Cell Press in adopting STAR Methods as our format of choice for the Methods section of our articles. We will begin implementing this change from the first of March 2021. The reason we are making this change is that STAR Methods, simply, works. You can see some of the results of STAR Methods with respect to the reproducibility of Cell Press papers in Menke et al. (2020). In particular, Figure 2 of that article shows that after adoption of STAR Methods there was a 4-fold jump in the percentage of antibodies which can be uniquely identified. That was not the case for journals which did not adopt STAR Methods or equivalent measures. Promoting usage of resource IDs (RRIDs) is but one of the key aspects of STAR Methods. A detailed description of the STAR format can be found in our STAR authors guide and it is beyond the scope of this editorial. However, we want to discuss its salient aspects, and what it means for iScience. One of the most obvious changes to iScience articles will be moving all the Method out of the Supporting Information and into the article itself, where it will be typeset and more prominently featured. In addition, STAR Methods has three guiding principles, and a structure, which we know improve outcomes. The principles are no length limitations for the methods write-up, the listing in one place (the Key Resources Table) of the important items necessary to reproduce the research results contained in the paper, and the listing of resource IDs for each of them. The structure is exemplified, aside from the Key Resources Table, by the standard headings in which STAR Methods text is always divided:1)Resource availability, with its three subheadings: Lead Contact, Materials Availability, and Data and Code Availability. This part was already required in the previous iScience Methods format. Experimental model and subject details (applicable only for life sciences experimental papers).2)Method details: precise details of all the procedures in the paper, without recurring to “as described earlier” citations, wherever feasible.3)Quantification and statistical analysis.4)Additional resources: links to websites that provide further information relevant to the study. This is not just a cosmetic change, it is a sea change especially for research outside of the life sciences, where commitment to structured, and reproducible, methods sections are uncommon in the publishing landscape. In particular, STAR Methods introduces the requirement, for researchers in every area, to prominently feature statistical analysis of the work at hand. Statistics are of fundamental importance to have confidence in research in any field. Yet although this has been a requirement broadly accepted and adopted across the life science, this needs to be matched in other disciplines as well. In this regard, as an example, for papers discussing new devices, it will be mandatory to feature results for all devices which were produced during the work (at least three, and certainly more than a “best device” example) and to have at least means and standard deviation of their performance. For specific areas within the physical sciences (solar cells, batteries and catalysis), iScience will further adopt checklists spearheaded by some of our sister journals such as Joule and Chem Catalysis, which will further improve on reporting for these more specific fields. As in all of our initiatives, we are very interested in the feedback of our authors, referees, and readers, to continuously improve our journal, as well as reproducibility and open science across the board.

Ähnliche Arbeiten