Some thoughts on systematic reviews

I remember that in my first year of university (in 2000) we were taken to the library and shown how to search the Beilstein Handbook of Organic Chemistry to identify previous instances of reactions we might be interested in. There was a wall of books and it was quite intimidating. However, I never had to look at those books because there was also an online searchable database which made the task much easier. I cannot imagine how much time must have been spent by chemists searching for reactions and reaction conditions, that was suddenly freed up by taking the data online.

This has happened with most data sources and we can now search through records of journal publications and patents with ease. This I believe has led to the increase in the publication of the systematic review and the decrease in the importance placed on narrative reviews. It has changed so much that, nowadays, before many research projects are fully initiated, a systematic review is undertaken to explore, among other things, what is already known about the topic and whether the research project is novel. Owing to their inclusion of all available, relevant data systematic reviews are considered the pinnacle of evidence in evidence-based medicine [Wikipedia]. This is, in my mind, embodied by the Cochrane Collaboration, which has a handbook on how to perform systematic reviews for inclusion in their database.

It is, therefore, important that all researchers understand what a systematic review is and how to perform one. In addition, it is important to know what to do with the output. Is it okay to summarize the data, or should it be synthesised in some way, either through meta-analysis or narrative synthesis. Furthermore, we need to be able to judge the quality of published systematic reviews so that we can judge their quality and determine whether there is any bias inherent in them, or flaws in the methods used.

The PRISMA Statement

The PRISMA Statement is a great help, as it details everything to be included in a systematic review and its planning and I have found this useful when planning, performing and writing up systematic reviews, as it provides a useful checklist to bear in mind.

I want to start my systematic review. What should I do?

The very first thing you need to do is ensure you are asking an interesting question that is relevant and that you are asking it in the correct fashion. You should discuss your idea with colleagues who work in the same field and have similar interests, and also read any relevant papers that you are aware off. This might highlight aspects that you are unaware of or have overlooked. It will also provide the groundwork for the next step, which is putting together your hypothesis and developing your search strategy which will identify the references that will potentially be included in the review. You should also work out who your co-authors will be and each person’s role. For example, you will need somebody to do the search and at least two people to go through the results separately and someone to adjudicate any conflicting views about whether or not to include a manuscript in the review.

Once this is done you are ready to think about actually starting work on your systematic review, and I shall offer my thoughts on those steps in the next few posts.