The second Evidence Synthesis Hackathon was held at The University of New South Wales in Canberra, Australia from 8-10 April 2019. This event was a rare opportunity to meet and work with leading thinkers in the fields of evidence synthesis and meta-research, including speakers from all three evidence synthesis collaborations: Cochrane, Campbell, and the Collaboration for Environmental Evidence.
Our intention was to inspire and support the development of new ideas in this important field. By bringing together technologists and developers with subject-matter experts, participants formed new collaborations and developed new ways to approach critical problems. Attendees worked together to build software tools and research papers that will profoundly influence the field of meta-research.
SoftwareThis event included development of the following software projects:
Systematic Maps are, according to the Environmental Evidence Journal, “overviews of the quantity and quality of evidence in relation to a broad (open) question of policy or management relevance.” In simple terms, this means that documents are categorized according to the type, location, and publication information available for each work within a particular topic. Systematic maps are often used for environmental research, where it is particularly important to track the location of study sites. The spatial nature of a systematic map, particularly for environmental research, means that academics often use some kind of geographic map to analyze and present their information. Understanding the academic community’s familiarity with the R programming language, we built a webapp using R Shiny that could automate certain parts of creating a systematic map for environmental research.
Forestr is an online platform for interactive visualisations of forest plots.
In evidence synthesis it is relatively easy to find published literature on a given topic, and to record what the search string was used in which database, how many records were found, how many were used or not used and why. It is much more difficult to be systematic in searching and recording grey literature.
metadat is an R package that collates and acts as a repository for meta-analytic data across diverse research fields (e.g., psychology, education, ecology, evolution). These data can be used freely for meta-analytic training to demonstrate concepts and complex analyses, re-analysis, and updating past meta-analyses. Meta-analysts from any field can contribute data and/or examples of data use.
Evidence synthesis (ES) is the process of identifying, collating and synthesising primary scientific research (such as articles and reports) for the purposes of providing reliable, transparent summaries. The goal of this project is to collect, integrate and expand the universe of available functions for ES projects in R, via our proposed metaverse package. Like tidyverse, metaverse is envisioned as a collector package that makes it straightforward to install a set of functions - currently located in separate packages - for a common purpose.
Automation tools are speeding up the conduct of evidence synthesis. However, the uptake of these tools amongst reviewers are slow. Potential barriers to use are; 1) the tools often operate in isolation, 2) reviewers need to manipulate their citation data into a specific format to use the tool, and 3) different tools require different levels of programming or computing expertise.
robvis is an R package that allows users to quickly visualise risk-of-bias assessments performed as part of a systematic review. It allows users to created weighted bar-plots of the distribution of risk-of-bias judgements within each bias domain, in addition to “traffic light” plots of the specific domain-level judgements for each study. The resulting figures are formatted according the risk-of-bias assessment tool use to perform the assessments (currently supported tools are ROB-2, ROBINS-I and QUADAS-2). An associated Shiny app provides a user-friendly interface for the tool.
A new ecosystem for evidence synthesis | Published academic paper
The number of publications has been increasing exponentially, and as a result, so has the research field of evidence synthesis. Consequently, there is now a need to maintain the quality, currency and credibility of evidence synthesis approaches. Within this commentary, we provide a vision for evidence synthesis as a fundamental tool for generating and guiding decision-making. This paper is aimed at all stakeholders, including researchers, institutions, and the broader general community.