Some thoughts on systematic reviews

I remember that in my first year of university (in 2000) we were taken to the library and shown how to search the Beilstein Handbook of Organic Chemistry to identify previous instances of reactions we might be interested in. There was a wall of books and it was quite intimidating. However, I never had to look at those books because there was also an online searchable database which made the task much easier. I cannot imagine how much time must have been spent by chemists searching for reactions and reaction conditions, that was suddenly freed up by taking the data online.

This has happened with most data sources and we can now search through records of journal publications and patents with ease. This I believe has led to the increase in the publication of the systematic review and the decrease in the importance placed on narrative reviews. It has changed so much that, nowadays, before many research projects are fully initiated, a systematic review is undertaken to explore, among other things, what is already known about the topic and whether the research project is novel. Owing to their inclusion of all available, relevant data systematic reviews are considered the pinnacle of evidence in evidence-based medicine [Wikipedia]. This is, in my mind, embodied by the Cochrane Collaboration, which has a handbook on how to perform systematic reviews for inclusion in their database.

It is, therefore, important that all researchers understand what a systematic review is and how to perform one. In addition, it is important to know what to do with the output. Is it okay to summarize the data, or should it be synthesised in some way, either through meta-analysis or narrative synthesis. Furthermore, we need to be able to judge the quality of published systematic reviews so that we can judge their quality and determine whether there is any bias inherent in them, or flaws in the methods used.

The PRISMA Statement

The PRISMA Statement is a great help, as it details everything to be included in a systematic review and its planning and I have found this useful when planning, performing and writing up systematic reviews, as it provides a useful checklist to bear in mind.

I want to start my systematic review. What should I do?

The very first thing you need to do is ensure you are asking an interesting question that is relevant and that you are asking it in the correct fashion. You should discuss your idea with colleagues who work in the same field and have similar interests, and also read any relevant papers that you are aware off. This might highlight aspects that you are unaware of or have overlooked. It will also provide the groundwork for the next step, which is putting together your hypothesis and developing your search strategy which will identify the references that will potentially be included in the review. You should also work out who your co-authors will be and each person’s role. For example, you will need somebody to do the search and at least two people to go through the results separately and someone to adjudicate any conflicting views about whether or not to include a manuscript in the review.

Once this is done you are ready to think about actually starting work on your systematic review, and I shall offer my thoughts on those steps in the next few posts.

What is a key message and why have only one?

When you read a scientific article to gain knowledge of what people have done and what they are thinking, what do you take away? Often when I was working in the lab I would specifically look for reaction conditions that might work for my experiments and ignore the introduction and discussion. However, if you read the entire article you likely will not remember exact experimental conditions once finished, rather you will hopefully leave with the idea that the article answered a question and you will take away an idea. This idea is the key message for the manuscript.

Determining the key message

Often when writing articles you will discuss what the message will be when planning the article and putting together the outline. This message will infuse the article, being proposed or hinted at in the introduction, expanded upon in the discussion and possibly explicitly stated in the concluding remarks. However, if you try to include two separate messages then you will muddy the message and it will be less impactful.

For example, if my message is that the new treatment I am discussing is efficacious and well tolerated I can easily build the article around this message. Whereas if the messages are that the treatment is efficacious and well tolerated, and also that it is extremely cost-effective compared to therapies currently on the market, then there will be two narratives running through the article aimed at different groups of people and this may cause confusion.

What to do if you want to include multiple key messages?

I would suggest that if you want to have two different messages for different audiences that you consider writing two articles. This will improve the ability of the article to reach the appropriate audience, and also allow each message to have maximum impact.

Why should you read author guidelines before writing a first draft?

By the time a first draft is written I always feel that the following should have been completed:

  • Type of article/manuscript being written
  • Author list decided
  • Outline put together showing what work will be included and discussed
  • Outline agreed upon by all authors and their input included
  • Target journal selected

At this stage there is the excitement of actually writing up the work, however, it is important not to dive in without knowledge of any rules you may have to conform to. There may be a limit to the number of words, figures or tables to include, or specific guidance on section titles to use or how to draw figures. If you write the draft without knowing any of this you may have to go back and edit your work, possibly significantly. I find it much easier to write a smaller number of words originally, rather than having to delete some. This is because once something is in writing it seems important to the story and is harder to let go. Additionally, it is easier to write to a particular structure rather than contorting the work to fit once completed.

However, you may be thinking that these are only guidelines and that your work is so amazing that no matter what state it is submitted in it will be published. This may be the case but often the journal will request you edit to journal guidelines before acceptance or even before consideration. So writing in line with the guidelines should save you work later on and maybe also increase the chances of your manuscript being accepted.

Why should you choose a journal before writing a first draft?

In life it often pays to be prepared. For example, before going for a night out we check that we have enough cash to avoid embarrassment when trying to get drinks. This same level of preparation and preparedness is also essential when you are writing a manuscript for submission to a journal.

Each journal is different and may have different rules on how much you can write and how to structure your manuscript. This might limit the number of figures or references you can include, or request specific details be included in subsections. You may think that you can edit your manuscript to meet these guidelines at a late stage, but this can be hard. If you have cited 100 references and are allowed a maximum of 30 which should you cull?

Maybe you think that you do not need to abide by these rules and your work will be accepted whatever. I have found that this is often not the case. For example, if you submit a manuscript that is overly long the editor will send it back with a request to edit the word count down and resubmit. This is additional work and time that could be better spent moving the research forward or writing grant applications.

I would, therefore, advise you to select your target journal, having written an outline and before you write a first draft. This enables unconstrained creativity when developing the initial flow of the article and deciding what to include, but then allows a targeted approach after this, which can maximize your chance of being considered by your chosen journal. By selecting the journal after the outline you also have more information at your disposal about the proposed manuscript and can reject journals if they do not allow a particular aspect of your manuscript, possibly seven huge tables or a video.

So be prepared and increase your chances of success!

Why is everything a story?

When we think about scientific and technical writing we generally do not think of the outcome as a story. Indeed, saying that it is a story might be considered by some people as an insult, as they consider stories to be made up and not based completely on fact. However, to me every time we communicate we are trying to tell a story. We are trying to convey ideas in a logical and step-wise manner, leading people through our considerations and thoughts and showing how we reached our conclusions.

To demonstrate my thinking I shall break down a manuscript into its constituent parts and highlight how this links to story telling.

Introduction

This is you ‘hook’ into your research or review. Why are you asking these questions? What led you to make the decision to undertake this project? It generally ends with you hypothesis and ideally readers will be asking the same question as you at this point.

Methods and Results

These are the story of what you did to test your hypothesis and the data/results you obtained. Once again there should be a logical flow as to why you did the experiment based upon your hypothesis and how further experiments and analyses build upon this initial experiment.

Conclusion

This is the denouement of your story where you pull together everything you have learned from your research, together with what was known before and discuss what you have learned and how it affects thinking on the topic being researched. It may also include a cliffhanger where next steps are discussed.

 

Whilst I wrote this I was listening to Decade of Viper