Column

Shifting Science and Research Culture Towards Openness and Reproducibility Through Education and Training

Luke W. Johnston1, Helene Baek Juel 2, Bettina Lengger 3, Daniel R. Witte 1,4, Hannah Chatwin 5, Malene Revsbech Christiansen 2, Anders Aasted Isaksen 4

1. Steno Diabetes Center Aarhus; 2. Novo Nordisk Foundation Center for Basic Metabolic Research; 3. Novo Nordisk Foundation Center for Biosustainability; 4. Department of Public Health, Aarhus University; 5 University of Southern Denmark

Other instructors and contributors include Stine Scheuer, Mario Garcia Urena, Anders Askeland, Omar Silverman, Andreas Eiset, and Signe Storgaard.

Contact:
Luke W. Johnston, MSc, PhD
Email:
ORCID iD icon0000-0003-4169-2616
posters.lwjohnst.com/2021/dda-ws
Creative Commons License 		DDA Logo

SDCA Logo

Reproducibility, a core principle of science, is rarely done

  • Reproducibility is when the same result is independently produced by others using the same data and same code/analysis plan.
  • Estimating the reproducibility of scientific studies is currently very difficult because of:
    • Nearly non-existent publishing of code/data
    • General lack of awareness of and training in it
  • Non-replication is a known major problem, but extent of non-reproducible results is unknown. Barriers to addressing the problem include:
    • Lack of incentives to be reproducible
    • Emphasis on novelty and original work

Column

We don’t share as much as we should

There are few studies on the extent of code and data availability, and whether study results can be reproduced. Figure shows results of some of them: 1) [10.1177/2515245920918872](https://doi.org/10.1177/2515245920918872), 2) [10.1007/s11306-017-1299-3](https://link.springer.com/article/10.1007/s11306-017-1299-3), 3) [10.1371/journal.pone.0251194](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0251194).

There are few studies on the extent of code and data availability, and whether study results can be reproduced. Figure shows results of some of them: 1) 10.1177/2515245920918872, 2) 10.1007/s11306-017-1299-3, 3) 10.1371/journal.pone.0251194.

Fundamental changes are needed, including education and training

Our aim was then to create an open, re-usable, and beginner-friendly learning module on how biomedical researchers can do Reproducible Research using the R statistical program (course is abbreviated to “r-cubed” or R3). Using key principles: Evidence-based learning and teaching practices; Mixture of activities (practicing, listening, reading, typing, discussing); Openly licensed (CC-BY); Publicly accessible; Create safe and supportive environment for learning; Use modern and beginner-friendly software and workflows (e.g. R); Documentation to also re-use for other instructors.

Introductory course details and website

Website that has material for introductory course: [r-cubed.rostools.org](https://r-cubed.rostools.org/). See our paper (DOI: [10.21105/jose.00122](https://jose.theoj.org/papers/10.21105/jose.00122)) describing the course and how to use it.

Website that has material for introductory course: r-cubed.rostools.org. See our paper (DOI: 10.21105/jose.00122) describing the course and how to use it.

Column

Intermediate course details and website

Website with the material for the intermediate course: [r-cubed-intermediate.rostools.org](https://r-cubed-intermediate.rostools.org/).

Website with the material for the intermediate course: r-cubed-intermediate.rostools.org.

Skills from the course are in demand and needed

We’ve done the introductory course 4 times and intermediate 3 times with the DDA. The responses to the pre-course survey question about what they want and expect to learn emphasize the need of these skills and knowledge:

  • “… would like to have a more structured workflow … and to have some tools for reproducibility”
  • “[To] manage data in a more organized way…” and “Version control. Sharing of code.”
  • “Git and good practice in the context of reproducible and open science”
  • “Better structure in my data analyses so that in the future I can clearly see, what I have done and why.”
  • “…create pipelines for my analysis that is more reproducible, readable, and that can be easily re-used for my future self and for my collaborators and colleagues”
  • “Structured, reproducible approach to using R and [G]it for future use in analyses”
  • “…information about open science and reproducibility…”
  • “… components about the reproducibility crisis and Open Science problems, so that we could have a context for why we should be incorporating R in our standard practice.”

Next steps and plans

Possible plans include developing an advanced course, short online tutorials, a video series of the material, and to run frequent (online) coding review sessions.

Are you interested in being involved in any of these or current projects? Contact us!

And of course, a HUGE thanks to DDA for hosting these courses and allowing them to grow ❤️ ❤️