The gold standard for supporting a research finding is replication: conducting a new experiment and coming to the same conclusion. However, replication is not always feasible. For example, researchers have only one chance at measuring real world events in real time like climate data. Additionally, clinical research can only feasibly recruit a certain number of participants, and the specimens collected are limited.
Even when replication isn’t possible, research can be reproducible. Reproducibility is the minimum standard to assure the validity of your results. To make your work reproducible, you must provide enough information about what you did to allow someone else to get the same results. The type of research you do determines which type of reproducibility is most relevant:
Empirical reproducibility (methods + data) is important to experimental researchers. They can make their work reproducible by providing detailed descriptions of the methods and reagents used in their work along with the data collected from the experiments. Other researchers should be able to follow these instructions and get similar data.
Tools for empirical reproducibility:
Computational reproducibility (code + data) is important to computational researchers. They can make their work reproducible by making their data and code available with publications along with any specific instructions needed to install or run the code. Users should be able to run their code on their data and get the same result.
Tools for computational reproducibility:
- GitLab
- Computational Notebooks (Jupyter/RMarkdown)
- Zenodo (UW-Madison’s Dryad membership integrates with Zenodo, please contact us to learn more!)
Statistical reproducibility (preregistration + statistical details) is important in fields that use statistical techniques to analyze large datasets that can be analyzed in many different ways. Researchers can provide detailed information about the choice of statistical tests, model parameters, and threshold values to prevent p-value hacking and other manipulations. Users should be able to run the same tests and models and get similar significance values.
Tools for statistical reproducibility:
- OSF, or other tools that enable preregistration
- R programming language