The second Evidence Synthesis Hackathon was held at The University of New South Wales in Canberra, Australia from 8-10 April 2019. This event was a rare opportunity to meet and work with leading thinkers in the fields of evidence synthesis and meta-research, including speakers from all three evidence synthesis collaborations: Cochrane, Campbell, and the Collaboration for Environmental Evidence.
Our intention was to inspire and support the development of new ideas in this important field. By bringing together technologists and developers with subject-matter experts, participants formed new collaborations and developed new ways to approach critical problems. Attendees worked together to build software tools and research papers that will profoundly influence the field of meta-research.
Organisers
Participants
Software
This event included development of the following software projects:
EviAtlas
| An R tool for systematic maps
Systematic Maps are, according to the Environmental Evidence Journal, “overviews of the quantity and quality of evidence in relation to a broad (open) question of policy or management relevance.” In simple terms, this means that documents are categorized according to the type, location, and publication information available for each work within a particular topic. Systematic maps are often used for environmental research, where it is particularly important to track the location of study sites. The spatial nature of a systematic map, particularly for environmental research, means that academics often use some kind of geographic map to analyze and present their information. Understanding the academic community’s familiarity with the R programming language, we built a webapp using R Shiny that could automate certain parts of creating a systematic map for environmental research.
forestr
| Interactive forest plots
Forestr is an online platform for interactive visualisations of forest plots.
Grey Literature Reporter
| Chrome plugin for grey literature searching
In evidence synthesis it is relatively easy to find published literature on a given topic, and to record what the search string was used in which database, how many records were found, how many were used or not used and why. It is much more difficult to be systematic in searching and recording grey literature.
metadat
| Meta-analytic datasets for R
The metadat package contains a large collection of meta-analysis datasets. These datasets are useful for teaching purposes, illustrating/testing meta-analytic methods, and validating published analyses.
metaverse
| Evidence synthesis workflows in R
Evidence synthesis (ES) is the process of identifying, collating and synthesising primary scientific research (such as articles and reports) for the purposes of providing reliable, transparent summaries. The goal of this project is to collect, integrate and expand the universe of available functions for ES projects in R, via our proposed metaverse package. Like tidyverse, metaverse is envisioned as a collector package that makes it straightforward to install a set of functions - currently located in separate packages - for a common purpose.
Integration & Research Weaving
| Developing the ICASR Integration Engine
Automation tools are speeding up the conduct of evidence synthesis. However, the uptake of these tools amongst reviewers are slow. Potential barriers to use are; 1) the tools often operate in isolation, 2) reviewers need to manipulate their citation data into a specific format to use the tool, and 3) different tools require different levels of programming or computing expertise.
robvis
| Risk of bias assessments in R
robvis is an R package that allows users to quickly visualise risk-of-bias assessments performed as part of a systematic review. It allows users to created weighted bar-plots of the distribution of risk-of-bias judgements within each bias domain, in addition to “traffic light” plots of the specific domain-level judgements for each study. The resulting figures are formatted according the risk-of-bias assessment tool use to perform the assessments (currently supported tools are ROB-2, ROBINS-I and QUADAS-2). An associated Shiny app provides a user-friendly interface for the tool.
Discussions
A new ecosystem for evidence synthesis | Published academic paper
The number of publications has been increasing exponentially, and as a result, so has the research field of evidence synthesis. Consequently, there is now a need to maintain the quality, currency and credibility of evidence synthesis approaches. Within this commentary, we provide a vision for evidence synthesis as a fundamental tool for generating and guiding decision-making. This paper is aimed at all stakeholders, including researchers, institutions, and the broader general community.