Technology

This is a version of our article ‘The Evidence Synthesis Hackathon: A global community building technology for more rigorous, efficient, and just evidence synthesis’, published by The Conversation, 1 July 2019.

Although the term the ‘Fourth Industrial Revolution’ has become a bit of a buzzword in South Africa, two recent commissions/task teams – one appointed by the President of the country and another by the Minister of Higher Education – hint at the potential of artificial intelligence and biotechnologies to disrupt the way we do things. Of these two technologies, it is especially machine learning that has the potential to reduce the time and effort of laborious repetitive activities in any evidence synthesis process. Technology is moving forwards rapidly and in ways that may make a perfect marriage with key aspects of evidence synthesis methods, for example, with machine learning and transparent web scraping.

As we continue to see more and more research published across the globe (currently around 3 million articles per year), we will need new technology that can help us synthesise and make sense of this evidence. Otherwise, we risk missing opportunities to make effective decisions and design efficient public policies.

What is evidence synthesis?

Evidence synthesis is the process of converting large bodies of scientific research (such as articles, reports and data) into reliable and digestible evidence that can inform management or policy. It is vital to have important decisions about public policy informed by rigorous scientific evidence to ensure public funding is used efficiently, and only on projects that we know stand a chance of working (see this article by ACE colleagues). Reliable evidence enhances transparency, public trust and accountability in decision-making processes.

But there’s a catch: evidence synthesis projects can be extremely time-consuming and costly. This is because reviewers must read through vast numbers of research papers from a range of different sources.

Some researchers believe that evidence syntheses can be made more resource-efficient, timely, transparent, rigorous and accessible by developing and using freely accessible software tools and workflows. This new wave of technological advancements could change the face of evidence synthesis – letting machines do much of the laborious work so that humans can focus on planning, thinking and doing.

Using technology to make evidence synthesis human-friendly

To support this endeavour, Neal Haddaway and Martin Westgate created the Evidence Synthesis Hackathon (ESH) series in 2017 to bring together world-leading researchers, practitioners and software developers to produce new Open Source tools and frameworks to support evidence synthesis.

We have run three ESH events so far: two in 2018 in Stockholm and one in 2019 in Canberra. These highly interactive events have involved 64 participants from 13 countries on six continents. Some 19 projects have been initiated, including tools that help researchers visualise databases of studies and those for extracting information from documents. We have also produced discussion papers that introduce new ways to think about evidence synthesis.

Many of these outputs are already being used by researchers, such as EviAtlas, a tool for producing maps of evidence unearthed during systematic literature reviews. This tool converts a database into a set of attractive, interactive figures and tables that show patterns in the ‘evidence base’ and where knowledge gaps and clusters might exist. EviAtlas also allows users to produce free, interactive websites displaying the nature of the evidence on a geographical map – something that would previously have been expensive to do and highly complex to achieve.

Funding the future

So far, the ESH has received generous funding of around 100 000 Euro over three years from six different institutions. This funding has enabled effective meetings to be organised, supported participants from around the world to attend those meetings, and covered publication costs for research papers coming out of the event. Our funders include universities, research councils, and software organisations.

In 2019 we were very fortunate to receive R250 000 from the Faculty of Humanities at the University of Johannesburg, facilitated through the Africa Centre for Evidence. This funding enabled three African participants to attend the Hackathon in Canberra, Australia in April 2019. Sadly, two further funded participants were denied visas by the Australian government at the last moment, preventing their involvement in the Hackathon (this seems to be an increasingly common problem for Africans). The three participants that were able to attend were integral to the discussions and coding work at the Hackathon, and were instrumental in its success.

Participating in the Evidence Synthesis Hackathon

Two of the participants were experienced software programmers: Christopher Penkin and Mandlenkosi Ngwenya. They produced a tool to keep track of and save web-based searches for research: something that has until now been almost impossible to do in evidence syntheses. The tool is a Chrome browser extension that logs user search information and downloads it into a central database. This tool represents a huge step forward in ensuring the transparency, efficiency, and repeatability – key central tenets for rigorous evidence syntheses.

The third attendee from Africa – Witness Mapanga – is an evidence synthesis specialist. Witness worked with other world-leading researchers on projects to build a brighter future for evidence-use in policy. We plan to publish this work by the end of the year.


Participants at the ESH2019 event in Canberra (Photo courtesy of Neal Haddaway)

Future events

Plans are underway for future ESH events in South Africa, Norway, Canada, and the Netherlands in 2020 and beyond. Through holding the ESH in Africa in 2020 we hope to foreground the particular needs of and solutions for evidence synthesis in resource-constrained environments, and thereby ensure that such technologies enhance equity in evidence synthesis production. Funding for this and future events will be vital to ensure equitable representation across disciplines and backgrounds. The continued interest in these events demonstrates that they are critical to building communities of practice: networks of researchers, data scientists, practitioners and software developers focussed on driving progress towards a sustainable future as part of the Fourth Industrial Revolution. The ESH represents a novel but increasingly important part of this pathway.

Contributors

  • Neal R Haddaway - Stockholm Environment Institute & African Centre for Evidence, University of Johannesburg
  • Martin J Westgate - Fenner School of Environment & Society, Australian National University
  • Carina van Rooyen - African Centre for Evidence, University of Johannesburg
  • Witness Mapanga - Centre for Health Policy, University of the Witwatersrand
  • Mandlenkosi Ngwenya - Department of Planning, Monitoring and Evaluation, Pretoria, South Africa
  • Christopher Penkin - Digital Solution Foundry