esh_logo Discussion Papers: In preparation

Discussion Papers: In preparation

Limitations and biases of commercial bibliographic databases | Proposed Academic Paper

Reliable evidence synthesis requires access to a comprehensive, unbiased body of literature that can be searched for relevant information. Systematic reviewers typically search multiple (upwards of 10) bibliographic databases to identify sets of search results that might yield relevant results. Access to these databases is often restrictively expensive, hampering efforts to synthesise evidence by smaller organisations and groups from low- and middle- income countries, for example. When reviewers export references from these databases they must typically do so in small batches (this supposedly stops people from replicating commercial databases for profit): for Web of Science this must be done in batches of 500, which can add considerable time to a review with 20,000 search results or more! Finally, databases such as Web of Science exacerbate publication bias by selecting journals and publishers that are perceived to be of ‘high impact’, for example using citation indices. So, these resources may be expensive, hard to use, and offer a biased selection of evidence. In order to facilitate evidence synthesis and to reduce bias in how information is indexed and found, we call for the production of an Open Source, Open Access on-stop-shop database that catalogues all known academic research. Since tables of contents are freely available online, technology exists that can produce such an important and useful tool.


OpenDataOpenCode | An assessment of data and code availability in reviews

Transparency is vital for repeatability and verifiability of systematic reviews, as with all other research. Authors of quantitative systematic reviews (i.e. ones that include meta-analysis) can maximise transparency by publishing the data they collected and the analytic code describing and documenting their analyses. This project will conduct a map of evidence syntheses to assess the degree to which data and code used in meta-analyses are made public by authors and journals publishing systematic reviews in environmental science and healthcare.


Making primary research synthesis ready | Proposed Academic Paper

Evidence synthesis relies on primary research that is reliable, transparent, and where key information is readily accessible and useful for broader synthesis. We propose a succinct list of ideal attributes that primary research articles should report as standard so that they are more likely to be found and included in evidence syntheses. We discuss how implementing these changes to primary research reporting might be incentivised for authors, peer reviewers, editors, journals, and institutions such changes and this broad across medicine, environment, ecology, and social science disciplines.