Why is starting the most important step in any project?

This sounds like a trick question and the response should be ‘of course it is, if you don’t start there is no project’. So why mention it at all?

Why indeed?

If you are anything like me, you will have a number of projects that you would like to initiate. These include writing a short story for my daughters, putting together a zine, selling at least one piece of art and getting another peer-reviewed article published to improve my CV, among other things. I can spend ages coming up with ideas, writing lists and planning them in my head. However, most of them have not happened. Why is this?

Again, why indeed?

It is because, although I have good intentions, I cannot do everything and there are myriad distractions stopping me from starting some of these projects. It might seem more important to do the washing up and tidy out my box of magazines that I keep for collages, rather than putting a piece of art on an online shop. These tasks do not move me closer to my goal, but make me feel like something has been done, they give instant gratification and are a reward unto themselves.

So of you are aware of this procrastination, why not just do what you want?

There is only one reason, and that is fear. Fear that the story might not be perfect, fear that I will try to sell something and nobody will want to buy it, fear of trying something new and learning new skills, fear of meeting new people. All these fears and insecurities are masked by procrastination and the only way to over come them is by taking a small step towards your goal. That one small step will start your journey to your goal. It might be as simple as announcing your intent to your friends and getting them to provide support and nag you into doing something, or putting down a title or setting up an account with an online marketplace. Once you have taken that first step, you need to follow it with a second and by taking those initial small steps, momentum will grow and drive you to completing your goal. So for example, if you want to write something, if you write a small amount consistently every day you will soon have more words written than you expect!

So go, don’t procrastinate with social media or reading blogs, and take teh first baby step towards one of your goals!

What to do now that the references for inclusion in the systematic review have been selected?

You are now nearly ready to write up the systematic review. You have reviewed the full text versions of any references that looked promising for inclusion, culling out those that were inappropriate (remembering to record reasons and numbers). You have read the references that have been selected and are sitting down ready to write about the subject.

For an initial systematic review, it is acceptable to tabulate the relevant data and discuss it, without performing any further analysis. This is a purely descriptive approach and although you will provide your writing groups opinions on the results, you may not want to do any further analysis. However, to get the most out of a systematic review you will probably want to go further. This could involve one of two approaches, either a meta-analysis or a narrative synthesis.

What is a meta-analysis?

If the references detail studies that are similarly designed, with similar duration, populations and study drug/intervention use, then a meta-analysis is possible. Meta-analysis are typically used in medicine and are a statistical approach to estimating the treatment effect. By combining a number of different studies, larger populations can be explored and this can provide insight into rare complications or adverse events, or provide a better overview of clinical efficacy. A good introduction to what a meta-analysis is can be found here.

What is a narrative synthesis?

If the references all include very small study populations, or the study designs are different, for example the study lengths vary widely, then a narrative synthesis may be more appropriate. This approach involves the identification of themes and trends in the research, highlighting aspects that appear to influence outcomes or are important for the intervention. A good introduction to what a narrative synthesis is can be found here.

Do you have any tips of writing a good systematic review?

First off, write the manuscript with the PRISMA checklist next to you, to ensure that all the information that it requires is included. Also, even though it may appear that the conduct of a systematic review is rigid and does not allow any creativity it is important that the final manuscript is not overly dry, otherwise no-one will read it and it is wasted effort. Ensure that the story of why the systematic review was conducted is included and that each step or question builds logically from the previous one. You do not want to make any sudden jumps, as although you may find them intuitive, your readers quite likely will not. They have not been immersed in the review and discussions around it. To avoid this, you should get a colleague who has not been involved in any discussions about the systematic review at any point to review a near-final draft. They are likely to identify any gaps in the narrative and you should listen to them and if you disagree you discuss your reasons with that person. Also, don’t forget to thank them in the acknowledgements.

What should happen after the search is complete, should we immediately start writing the systematic review?

Once you have done the first version of the search, you may find that you have too many results or not many results at all.

Too many results?

If you have too many results it is likely that your search was too broad and that rather than manually checking each reference, using the filters provided by the search engines could help refine the results. Some possibilities for refining the search are to limit it to a specific language (English, for example), or to just searching the abstract and title, rather than the full text.

Too few results?

This might be what you expected and could be the basis for the systematic review, highlighting the need for more studies in a particular area. However, you may want to broaden your search if you feel that not all relevant manuscripts have been captured. This could be done by including additional keywords, or removing limits (e.g. looking for all studies and not just those in humans).

What if the number of results seems fine?

Once you are happy that the search has captured all the appropriate results, you will want to save a list of the results, including references, title and abstract, as well as recording the initial number of results found. You will need to keep track of this for the PRISMA flowchart. These results should then be distributed to the individuals that have been allocated as reviewers and they should review each abstract and decide whether it meets the inclusion criteria defined for the review. These criteria should have been defined before the search and it is important that both reviewers use them. The reason why each abstract is rejected should also be recorded. This can be done according to the PICOS, with abstracts rejected according to not containing appropriate participants, intervention, comparator, outcome or setting, or for another reason which should be recorded.

In a perfect world both reviewers will select the same abstracts at this stage, however, if they do not, an additional reviewer should arbitrate whether the abstract should be kept. If there is confusion, the abstract should be kept, for full text review (the next step).

After the abstracts have been filtered, should we start writing?

No. You will want to get the full version of each reference and these will also need to be reviewed, in a similar manner to the abstracts, to ensure that no inappropriate references are included. At this stage, you can note any manuscripts in the references that also look appropriate for the systematic review. This will enable you to check that they have been included and if not they can be added (and noted on the PRISMA flowchart).

This process of filtering can be long and tedious, however, it should enable all the relevant literature for the systematic review to be identified. It is at this stage that the writing can commence!

Performing the search for the systematic review

I am going to assume that you have developed your basic research question and are frothing at the mouth with anticipation that you will be able to search the literature and get to review possibly thousands of abstracts. Like many people I would suggest that you use PICOS to help determine your search.

What does PICOS mean?

From the capitalization you have probably realised that this is an acronym. It stands for:

Population
Intervention
Comparison
Outcome
Setting

By working your way through each item and defining it, you should be able to clarify your question and also define your search. I feel that the best way to show how to do this is to give an example.

Example

For this example my research question is going to be:
“Based upon differences in microvascular outcomes seen in people of South Asian origin with T2DM compared with the Western population, especially the relationship of outcomes with BMI, would SGLT2 inhibitors be more effective in this population?”

The first step is to define the population being investigated. This is “people of South Asian origin with T2DM”. However, when we perform the search we have to remember that not everyone with use the exact phrase that we have, for example South Asia includes a number of countries and we may want to include the populations of the separate countries and the names of the countries in the search as well. We will also want to add a search term for type 2 diabetes. These can then be combined as a search strategy:
(T2DM OR Type 2 diabetes) AND (South Asian OR South Asia OR India OR Pakistan OR…)

The next step is to define the intervention and for our question it would be SGLT2 inhibitors. The search strategy should also include the names of the individual drugs as well as the class. This would make the following search:
(SGLT2 OR sodium-glucose cotransporter 2 OR sodium-glucose co-transporter 2 OR canagliflozin OR dapagliflozin OR empagliflozin OR sotagliflozin…)

We then move onto the comparison. We are comparing different populations, rather than interventions, so this would be Western people with T2DM. For the first search I would not include this as a search term, and instead would keep it in mind when I was filtering the results following the search. However, the decision to include only studies comparing the two populations would help filter the results.

Next is outcomes which is undefined in our search and so would need discussion amongst the authoring group. Would we be looking at efficacy endpoints or cardiovascular outcomes, for example. Again this could be used for filtering the results and might not be included in the original search.

Finally is setting. This question could include large observational clinical data sets or clinical trial data and both would be interesting. Reviews would not be relevant to teh article but could be a useful way to ensure that all relevant references are found.

What next?

We have now defined our search and also some criteria to filter the results. So the next step would be to perform the search and the databases you use are dependent upon the subject you are researching. For example, in medicine it would include PubMed, Embase, Cochrane reviews and clinicaltrials.gov. It might also include results highlighted in the reference lists for any reviews identified.

The numbers of results should be recorded following teh search and before filtering as this is important for the PRISMA flow diagram.

A good handout to help define a search with PICOS is available from consortiumlibrary.org, which can be found here.