Collaborations Workshop 2019 (CW19) #CollabW19          2019-04-01 to 2019-04-03

How do you motivate researchers to adopt better software practices? - DSR10-CW2CC

Reporter

Patrick McCann - pgm5@st-andrews.ac.uk

Participants

Rachael Ainsworth, Jason M. Gates (jmgate@sandia.gov), Jakob S. Jørgensen, Diego Alonso-Álvarez, Cerys Lewis

Notes from the discussion

Please use the area below to capture notes from the discussion session.

  • How to get people out of bad habits, create good habits.

  • “Reproducibility is like brushing your teeth”

  • Awareness? Many researchers simply don’t know there are tools which will solve these problems for them.

  • Make it easy for them

  • Onboarding checklists - can be an issue in GitHub/GitLab

  • Easier with younger developers. Hard to get older developers into open development mindset.

  • What’s in it for them? Software is not seen as the final goal, but a means to an end - why put in the extra effort.

  • Outline the benefits for them. Correlation between citations, opportunities and making outputs available.

  • Needs cultural change at senior level - software as a research output

  • GitHub as portfolio - use in recruitment

  • Some researchers don’t see themselves as programmers and their stuff not as software

  • Software publication requirements from funders, following on from OA and Open Research Data

  • Need to get people used to it.

  • Benefits to researchers of version control - ability to roll back

  • How to researchers find out about these tools? Mandatory training? Induction?

  • 2-week training programme as part of the hiring process - how does that work for people already in positions?

  • These skills need to be included in education at all levels

  • Can programming lessons incorporate good practice e.g. use GitHub

  • Encourage people to start small

  • Carpentry courses can be overwhelming

  • Where are things going wrong? What good practice would address that? Agile

    development applied to team practice.

  • Link to studies that show benefits in blog

  • Software papers. Awards? Journal badges

  • Wellcome Trust policy on open research outputs

  • Altmetrics

Speed Blog

Please use the area below to draft the speed blog. Consult https://www.software.ac.uk/speed-blogging-and-tips-writing-one for information, tips and examples.

What are the challenges?

For many researchers, the development of software is a means to an end—a chore that is necessary to allow them to get on with the real work of conducting research and publishing papers. They may not see themselves as programmers or recognise the code that they write as being software. Their supervisors or senior colleagues may not see the value of devoting perceived extra effort to following good practice.

Moreover, good software practices can be seen as an extra burden. They involve the learning of new skills and technologies, and the application of these can seem to introduce complexity and extra work. Researchers may even be unaware of the existence of tools and resources that can help them follow good practices, let alone the benefits they may bring.

Introductory training events, like The Carpentries, can guide researchers into good software practices, but they can also be very intense and overwhelming, resulting in the attendees mixing concepts and without time to practice what they have learnt. If, on the contrary, researchers are left on their own, and assuming they have the interest, they might not find the correct tools or may gain an incomplete, or plainly wrong, knowledge of what developing software for research means and what adopting good software practices actually entitles.

However, good software practices do bring real benefits to researchers and it is in communicating these to researchers, and in getting them to a point where they can realise those benefits, that the biggest challenge lies.

What measures can be taken?

Encourage people to start small. Every little bit helps. A single test is much better than no tests at all. A few lines of documentation are better than none. Spending a few minutes goes a long way. It’s not unlike going “green”: Replacing a couple single-use plastic coffee cups a week by a reusable one helps significantly. So does reducing the number of days one eats meat, which may be easier to manage for most individuals compared to going vegetarian. Hopefully the benefits become clear once started, and progress will continue from there.

Many researchers who use good software practices have had to discover and teach themselves how to use the tools such as version control. Some universities/faculties/departments already offer training courses on such tools, but only a small part of researchers/students actually take these. Would it be possible to make mandatory Software Carpentry type course units to be taken by all students in certain subjects? This would expose students to the tools and mindset, and start them early on using good practices, or at least they would be aware of the existence of such tools later on when the need may arise. In this way the next generation of researchers would be reached. Ideally, these skills need to be included in education at higher levels, as well and across subject areas, such as part of a mandatory introduction programme as PhD student or postdoc joining the institution. Some form of credit should be given for completion; for students this would naturally be course credits toward the degree.

Other measures that could be taken to motivate researchers to adopt better software practices are:

  • Breaking down Carpentries courses into smaller blocks in order to reduce the feeling of being overwhelmed for people learning new skills.
  • In addition, or as an alternative, to dedicated Carpentries-style training courses as part of the education, the tools could be taught indirectly by being incorporated into other courses. For example hosting course material on GitHub or using GitHub Classroom to manage a course.
  • Can programming lessons incorporate good practice, e.g., using GitHub for distance learning (check out homework, etc.?)
  • Using pain points within a team to find methodologies that are seen as a positive change when introduced and allowing time for these to become accepted part of day to day working.
  • Building in training or resources as part of the onboarding process for a company or institution and making it easy to access.

The process is already well underway. Funders increasingly demand openness: Requirement of open-access publication, deposit of data sets already required by Wellcome (to be confirmed), and a natural next stage will be fully open-source code. In addition to such requirements, some more forms of positive encouragement should be considered. This could be awards for “championing” openness by making data and open-access. This could be dedicated funding opportunities to help increase open sharing of code and data. If such already exists, increase visibility.

What benefits will result?

Open research practices, including incorporating better software practices into the research workflow, lead to better and more efficient science. The challenges associated with encouraging researchers to adopt these practices have been described above, along with some potential solutions. But what are the benefits for researchers for working in this way?

McKiernan et al. (2016) demonstrates that open research is associated with increases in citations, media attention, potential collaborators, job opportunities and funding opportunities relative to more traditional closed practices. In short, the more you share your research outputs, the more opportunities you create for citations and gaining exposure.

Publications and citations act like currency in academia, often used to assess the quality of a researcher and the metric associated with career progression. While there are flaws associated with this, one motivation for researchers to adopt better software practices is that they can lead to more publications and citable research outputs. For example, if the software is open source and well documented, it can be easily submitted to the Journal for Open Source Software, resulting in a peer-reviewed and citable publication for your software.

Other benefits are:

  • Altmetrics
    • Tracks impact of research output
    • For example, tracks how many times a paper/presentation/poster/etc. has been viewed, downloaded and shared on social media (do they also track citations? - check)
  • “Your primary collaborator is yourself 6 months from now, and your past self doesn’t answer emails,” is a famous quote from The Carpentries.
  • Using, for example GitHub, as a project management tool in addition to version control, allows you to easily find, access and share work.
    • GitHub as portfolio - use in recruitment
    • Benefits to researchers of version control - ability to roll back
  • In an interview with Nature, Irakli Loladze, a mathematical biologist at Bryan College of Health Sciences in Lincoln, Nebraska says, “Reproducibility is like brushing your teeth. It is good for you, but it takes time and effort. Once you learn it, it becomes a habit.”