The second Evidence Synthesis Hackathon was held at The University of New South Wales in Canberra, Australia from 8-10 April 2019. This event was a rare opportunity to meet and work with leading thinkers in the fields of evidence synthesis and meta-research, including speakers from all three evidence synthesis collaborations: Cochrane, Campbell, and the Collaboration for Environmental Evidence.
Our intention was to inspire and support the development of new ideas in this important field. By bringing together technologists and developers with subject-matter experts, participants formed new collaborations and developed new ways to approach critical problems. Attendees worked together to build software tools and research papers that will profoundly influence the field of meta-research.
SoftwareThis event included development of the following software projects:
Systematic Maps are, according to the Environmental Evidence Journal, “overviews of the quantity and quality of evidence in relation to a broad (open) question of policy or management relevance.” In simple terms, this means that documents are categorized according to the type, location, and publication information available for each work within a particular topic. Systematic maps are often used for environmental research, where it is particularly important to track the location of study sites. The spatial nature of a systematic map, particularly for environmental research, means that academics often use some kind of geographic map to analyze and present their information. Understanding the academic community’s familiarity with the R programming language, we built a webapp using R Shiny that could automate certain parts of creating a systematic map for environmental research.
Forestr is an online platform for interactive visualisations of forest plots.
In evidence synthesis it is relatively easy to find published literature on a given topic, and to record what the search string was used in which database, how many records were found, how many were used or not used and why. It is much more difficult to be systematic in searching and recording grey literature.
metadat is an R package that collates and acts as a repository for meta-analytic data across diverse research fields (e.g., psychology, education, ecology, evolution). These data can be used freely for meta-analytic training to demonstrate concepts and complex analyses, re-analysis, and updating past meta-analyses. Meta-analysts from any field can contribute data and/or examples of data use.
Evidence synthesis (ES) is the process of identifying, collating and synthesising primary scientific research (such as articles and reports) for the purposes of providing reliable, transparent summaries. The goal of this project is to collect, integrate and expand the universe of available functions for ES projects in R, via our proposed metaverse package. Like tidyverse, metaverse is envisioned as a collector package that makes it straightforward to install a set of functions – currently located in separate packages – for a common purpose.
Automation tools are speeding up the conduct of evidence synthesis. However, the uptake of these tools amongst reviewers are slow. Potential barriers to use are; 1) the tools often operate in isolation, 2) reviewers need to manipulate their citation data into a specific format to use the tool, and 3) different tools require different levels of programming or computing expertise.
robvis is an R package that allows users to quickly visualise risk-of-bias assessments performed as part of a systematic review. It allows users to created weighted bar-plots of the distribution of risk-of-bias judgements within each bias domain, in addition to “traffic light” plots of the specific domain-level judgements for each study. The resulting figures are formatted according the risk-of-bias assessment tool use to perform the assessments (currently supported tools are ROB-2, ROBINS-I and QUADAS-2). An associated Shiny app provides a user-friendly interface for the tool.
Future of research synthesis - Utopia or dystopia? | Proposed Academic Paper
The number of publications has been increasing exponentially, and as a result, so has the research field of evidence synthesis. Consequently, there is now a need to maintain the quality, currency and credibility of evidence synthesis approaches. Within this commentary, we aim to provide a vision for evidence synthesis as a fundamental tool for generating and guiding decision-making. This paper will be aimed at all stakeholders, including researchers, institutions, and the broader general community.
Making primary research synthesis ready | Proposed Academic Paper
Evidence synthesis relies on primary research that is reliable, transparent, and where key information is readily accessible and useful for broader synthesis. We propose a succinct list of ideal attributes that primary research articles should report as standard so that they are more likely to be found and included in evidence syntheses. We discuss how implementing these changes to primary research reporting might be incentivised for authors, peer reviewers, editors, journals, and institutions such changes and this broad across medicine, environment, ecology, and social science disciplines.