Status: In preparation
The Evidence Synthesis Technology world can be complicated - tools can be hard to find, it can be difficult to know what skill level is needed, and they can involve a considerable learning curve. Here at ESH, we are trying to lower the costs associated with finding, learning and using new ESTech tools and frameworks. This blog series aims to introduce key tools that can help increase transparency, accessibility, efficiency and rigour in evidence syntheses. Each month, a member of the ESH family will introduce their ESTech, explaining its purpose, the background skills needed, and how it can be used to support evidence synthesis.
Limitations and biases of commercial bibliographic databases | Proposed academic paper
Reliable evidence synthesis requires access to a comprehensive, unbiased body of literature that can be searched for relevant information. Systematic reviewers typically search multiple (upwards of 10) bibliographic databases to identify sets of search results that might yield relevant results. Access to these databases is often restrictively expensive, hampering efforts to synthesise evidence by smaller organisations and groups from low- and middle- income countries, for example. When reviewers export references from these databases they must typically do so in small batches (this supposedly stops people from replicating commercial databases for profit): for Web of Science this must be done in batches of 500, which can add considerable time to a review with 20,000 search results or more! Finally, databases such as Web of Science exacerbate publication bias by selecting journals and publishers that are perceived to be of ‘high impact’, for example using citation indices. So, these resources may be expensive, hard to use, and offer a biased selection of evidence. In order to facilitate evidence synthesis and to reduce bias in how information is indexed and found, we call for the production of an Open Source, Open Access on-stop-shop database that catalogues all known academic research. Since tables of contents are freely available online, technology exists that can produce such an important and useful tool.
Evidence Synthesis training resources | Ongoing project
Training resources for evidence synthesis are disparate and often hard to find. This project aims to collate and catalogue training resources for evidence synthesis methods from across disciplines, sectors, sources and formats, making it easier to find the right training. The result will be a web-based user-support tool.
An assessment of data and code availability in reviews | Ongoing project
Transparency is vital for repeatability and verifiability of systematic reviews, as with all other research. Authors of quantitative systematic reviews (i.e. ones that include meta-analysis) can maximise transparency by publishing the data they collected and the analytic code describing and documenting their analyses. This project will conduct a map of evidence syntheses to assess the degree to which data and code used in meta-analyses are made public by authors and journals publishing systematic reviews in environmental science and healthcare.
Standardised data files for systematic reviews | Ongoing project
Systematic reviews are complex, laborious tasks that produce vast amounts of data. However, the effort required to produce these data are typically lost once a review is completed: some information is reported in the review, but often information is missing or specific details are lacking.