What to do now that the references for inclusion in the systematic review have been selected?

You are now nearly ready to write up the systematic review. You have reviewed the full text versions of any references that looked promising for inclusion, culling out those that were inappropriate (remembering to record reasons and numbers). You have read the references that have been selected and are sitting down ready to write about the subject.

For an initial systematic review, it is acceptable to tabulate the relevant data and discuss it, without performing any further analysis. This is a purely descriptive approach and although you will provide your writing groups opinions on the results, you may not want to do any further analysis. However, to get the most out of a systematic review you will probably want to go further. This could involve one of two approaches, either a meta-analysis or a narrative synthesis.

What is a meta-analysis?

If the references detail studies that are similarly designed, with similar duration, populations and study drug/intervention use, then a meta-analysis is possible. Meta-analysis are typically used in medicine and are a statistical approach to estimating the treatment effect. By combining a number of different studies, larger populations can be explored and this can provide insight into rare complications or adverse events, or provide a better overview of clinical efficacy. A good introduction to what a meta-analysis is can be found here.

What is a narrative synthesis?

If the references all include very small study populations, or the study designs are different, for example the study lengths vary widely, then a narrative synthesis may be more appropriate. This approach involves the identification of themes and trends in the research, highlighting aspects that appear to influence outcomes or are important for the intervention. A good introduction to what a narrative synthesis is can be found here.

Do you have any tips of writing a good systematic review?

First off, write the manuscript with the PRISMA checklist next to you, to ensure that all the information that it requires is included. Also, even though it may appear that the conduct of a systematic review is rigid and does not allow any creativity it is important that the final manuscript is not overly dry, otherwise no-one will read it and it is wasted effort. Ensure that the story of why the systematic review was conducted is included and that each step or question builds logically from the previous one. You do not want to make any sudden jumps, as although you may find them intuitive, your readers quite likely will not. They have not been immersed in the review and discussions around it. To avoid this, you should get a colleague who has not been involved in any discussions about the systematic review at any point to review a near-final draft. They are likely to identify any gaps in the narrative and you should listen to them and if you disagree you discuss your reasons with that person. Also, don’t forget to thank them in the acknowledgements.

What should happen after the search is complete, should we immediately start writing the systematic review?

Once you have done the first version of the search, you may find that you have too many results or not many results at all.

Too many results?

If you have too many results it is likely that your search was too broad and that rather than manually checking each reference, using the filters provided by the search engines could help refine the results. Some possibilities for refining the search are to limit it to a specific language (English, for example), or to just searching the abstract and title, rather than the full text.

Too few results?

This might be what you expected and could be the basis for the systematic review, highlighting the need for more studies in a particular area. However, you may want to broaden your search if you feel that not all relevant manuscripts have been captured. This could be done by including additional keywords, or removing limits (e.g. looking for all studies and not just those in humans).

What if the number of results seems fine?

Once you are happy that the search has captured all the appropriate results, you will want to save a list of the results, including references, title and abstract, as well as recording the initial number of results found. You will need to keep track of this for the PRISMA flowchart. These results should then be distributed to the individuals that have been allocated as reviewers and they should review each abstract and decide whether it meets the inclusion criteria defined for the review. These criteria should have been defined before the search and it is important that both reviewers use them. The reason why each abstract is rejected should also be recorded. This can be done according to the PICOS, with abstracts rejected according to not containing appropriate participants, intervention, comparator, outcome or setting, or for another reason which should be recorded.

In a perfect world both reviewers will select the same abstracts at this stage, however, if they do not, an additional reviewer should arbitrate whether the abstract should be kept. If there is confusion, the abstract should be kept, for full text review (the next step).

After the abstracts have been filtered, should we start writing?

No. You will want to get the full version of each reference and these will also need to be reviewed, in a similar manner to the abstracts, to ensure that no inappropriate references are included. At this stage, you can note any manuscripts in the references that also look appropriate for the systematic review. This will enable you to check that they have been included and if not they can be added (and noted on the PRISMA flowchart).

This process of filtering can be long and tedious, however, it should enable all the relevant literature for the systematic review to be identified. It is at this stage that the writing can commence!

How knowing your target audience could improve your writing

Sometimes when you read a scientific article you might feel that the authors are belittling you or aiming way to high, and you will put down the article and find something else to read. When this happens, the first thing I tend to consider is whether I am the target audience for that particular piece of writing. If I am, it might be that the authors had not decided on a target audience when it was written or have targeted the wrong publication venue, or it might just be that I am particularly interested or uninterested in that subject and so either no more or less than might be expected.

Why am I telling you this?

When we write, we often write to express ourselves and our thoughts and ideas on a topic, writing what interests us and not thinking about the person who will eventually read it. This is like designing a pair of gloves that fits us exactly and trying to market it as being suitable for everybody. We must remember that the point of scientific writing, if we aim to get it published, is for it to be read and thus spread our ideas, not solely for the sake of writing.

What should be done to help write appropriately for the audience?

Before you write your first outline, you should decide who will be your audience; not in a vague manner, for example saying anyone who is interested in hand care, but as precisely as possible, for example hospital hand care specialists. Once this audience has been identified you want to know what level of knowledge they will have on this topic. In the example, hospital specialists will likely already have a high level of knowledge and so would not want to have the basics of hand care told to them. This will make them assume the article is not for them and reduce the likelihood of them reading it. So in this case you could start the article at a high level, covering only the most pertinent points for the topic being discussed.

On the other hand, if your audience is hand care trainees you will want to include the basics, as you cannot assume that all trainees will have the requisite basic knowledge to make sense of the article. If the basics are missing then you could place a barrier in front of much of your audience.

Using ‘personas’

In certain industries, the target audience is identified and idealized ‘personas’ are created to emphasise key aspects that need to be catered for. A well-known version of these is the broad terms given to people born at different times or target voter populations during elections (e.g. Mondeo Man). This allows broad generalizations to be made to highlight differences between generations and allow the average member of this group to be targeted, hopefully allowing the maximum proportion of this group to be successfully reached.

I would not suggest that you should create these personas but they should serve to remind us that a target audience will often have a lot in common and this can be catered for. However, if we write from the beginning with our audience in mind then our writing is more likely to be relevant, and read, by the people we want it to reach.

Make sure you write regularly

Having told you not necessarily to publish everything in a traditional manner, I would recommend that you start trying to write regularly. This could be typing up experimental methods and results so that they are ready as soon as the decision to publish is made, putting together review articles or free-writing around a research idea.
This writing will hopefully improve your productivity, as well as helping to organize your thoughts and create new ideas, as well as potentially helping you overcome any stalling points in your research. It might be that you are trying to develop a total synthesis and you are stuck on a single step, by searching the literature and making notes on what others attempting similar transformations have done. This is likely something you are already doing, but by keeping written notes you will be have something that could potentially be the basis of a review article and you may notice connections that you wouldn’t spot if you just read and highlighted articles.
Finally, practice makes perfect and by writing more your communication skills should improve.

Don’t dilute your output

This probably sounds obvious, and it should be. Nowadays, scientists and researchers are judged, rightly or wrongly, by their publication history. This has contributed to the explosion in the amount of scientific literature available and if you browse it you will often see work that makes you wonder why it was published and what it adds. The obvious answer to this is that it adds to someone’s publication list and that is the only reason it was written up and submitted to a journal. Do you want to be one of the people who add to this pile of ‘irrelevant’ literature? I would hazard a guess that you do not want this. Therefore, this first piece of advice must be given with a proviso that you should write regularly, but only publish relevant work. This might necessitate a change in the way you think about publishing, for example if you have a series of experiments that confirm somebody else’s work that you have done as the groundwork for a research stream, you may want to publish them on your own website if there have been other people also publishing on this. This means that you will not be diluting your work and you will be known for your strong publications.

How should you decide who will be an author?

When deciding who should be an author, you might feel pressure to include people who have not directly contributed to the work being written about. For example, people who secured funding for the work or who are important in the department that you work in, might say that they should be authors.

As this might lead to questions about ghostwriting and honorary authorship, the International Committee of Medical Journal Editors produced a uniform set of criteria for authorship. These are:

Authors “should have participated sufficiently in the work to take public responsibility for relevant portions of the content” and should meet all three conditions below:

  • Substantial contributions to the conception or design of the work; or the acquisition, analysis, or interpretation of data for the work; AND
  • Drafting the work or revising it critically for important intellectual content; AND
  • Final approval of the version to be published; AND
  • Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

These criteria are also included in GPP2 produced by the International Society for Medical Publication Professionals, which was published in BMJ.

If a person has contributed to some steps but not others then they should be included in the acknowledgements. This might include people who have helped perform an experiment, but not helped in the design, or who have proof read the article prior to submission.

The guidelines also suggest that if the study is very large there could be an authoring committee that takes full responsibility for the content of the manuscript. The full list of study investigators can then be included as an appendix. The article can then be published on behalf of everyone who is involved and if a person who is not on the authoring committee wants to include the manuscript on their CV they can do so.

Hopefully, by having articles published by authority figures on this topic you can feel better able to push back on any demands from people who you feel should not be an author.

References

ICMJE authorship criteria – http://www.icmje.org/recommendations/browse/roles-and-responsibilities/defining-the-role-of-authors-and-contributors.html

GPP2 – http://www.ismpp.org/gpp2

Don’t aim to write scientifically, aim to write well

When we write something we are often trying to convey who we are, or more specifically who we want to be perceived to be, to the writer. This means that we will alter the tone of our writing according to the audience, an email to a friend will be different from a job application letter. When writing a scientific manuscript it is, therefore, often the case that people write it in a manner they perceive to be ‘scientific’.

What is scientific writing?

When I say scientific writing I mean writing with many technical words intended to convey the idea that you know what you are talking about. These technical words will often be jargon and in some cases might be better conveyed using ‘everyday’ language. This will limit the audience of your work to others who understand the jargon that you are using, as well as potentially making the manuscript more difficult to read.

How can scientific writing be avoided?

Once you have written your manuscript you should get other people to review it, at least one of whom should be reviewing the language used. This should be someone you trust as the feedback might not be what you expect from your ‘perfect’ draft. When you ask this person to review the manuscript you should specifically ask them to keep an eye out for overly ‘scientific’ language.

A second thing that you can do is to read the draft out loud once you have written it. By reading out loud, even if it seems odd, you can identify any issues with ‘pacing’ or where jargon might interfere with the ‘flow’ of the manuscript.

A word of caution

Do not remove scientific words if they are necessary, you should not dumb down just to improve readability. This is a tightrope that you will have to walk, but hopefully by asking others for their opinions this will help navigate the issue.

What should be considered when writing a first draft?

By the time you get to writing a first draft of a manuscript I would expect the following steps to have already been completed:

  • Author list confirmed
  • Data fully analysed and data for the manuscript selected
  • Target journal selected
  • Target journal author guidelines read and any appropriate guidelines (e.g. word counts, required sections) noted down, preferably on a cover page to the manuscript
  • Outline written and content agreed/commented on by all authors
  • Extended outline written and agreed/commented on by all authors

If an extended outline exists…

If an extended outline has already been written then the jump to a full first draft should be fairly easy. The bulleted text will be made into complete sentences and linking text added. At this stage further logical gaps in the “story” might be identified and highlight the need for additional introductory text or research. However, this should be fairly minor as these issues will hopefully be identified at the extended outline stage.

If an extended outline does not exist…

I would suggest that if one does not exist then it is beneficial to put one together before writing the manuscript in prose, even if you are not sending it to co-authors for review. This will make the writing of the first draft much easier and should enable the logical flow of the manuscript to be worked out as discussed in my previous blog post.

Remember to write to journal guidelines at this stage to avoid unnecessary editing later on.

What is an extended outline and why might it be beneficial?

Once an outline has been written, discussed with co-authors, revised as appropriate in line with any comments and been agreed upon, the next step is to bulk out the outline. I believe that producing an extended outline in bulleted form, rather than a first draft written in prose is the best next step. This ‘extended’ outline should be the first draft in bullet form and contain everything that is expected to go into the first full draft.

Why an additional step?

In my experience, it is much easier to make revisions to the order information is presented in, when it is in bullet form. It might be that as you write you realise that the logical flow from one point to the next is wrong and that it is might be improved by shifting bits around. Or it could be that when your co-authors read the manuscript they highlight places where more information is needed, which could again change the flow of the document. If the document is written in full sentences, it can be hard to make these changes as it is difficult to see where it might fit. It is much easier to revise a ‘work in progress’ that a ‘fait accompli’. Overall, I have found that adding in this step saves time in the long run and often results in a better manuscript.

Why write an outline?

I, like many other people I expect, have previously tried to write a novel. I attempted it as part of NaNoWriMo, in which people endeavor to write a 50,000 word ‘novel’ during the month of November. I sadly failed in this attempt and looking back I can see two reasons why I failed. Firstly, although I liked the idea I was sadly not that dedicated to spending all my spare time writing and so only managed about 25,000 words. Secondly, I ‘pantsed’ rather than planned – that is I thought that the great novel would flow fully formed without any real forethought or planning. The resultant work was a complete mess, with what amounted to three discrete short stories linked by events, none of which were that good. This experience has shown me the value of planning or outlining at least some aspects of a novel before starting and in future this is what I will be doing.

Why is this relevant to writing about science?

I feel that the same can be true when writing about science, although often to a lesser degree, as the planning and research for conducting experiments should hopefully lend an idea of structure. Nonetheless, whilst it is often easy to list of the key points in the narrative of the research, these can be lost or forgotten when writing a prose document resulting in a loss of coherence.

I would, therefore, suggest writing an outline when starting to work on a document. This should contain the key points for each section and detail any data to be included. This outline will enable you to see if the manuscript makes sense and if there are any gaps, either data gaps or gaps in reasoning, before more words are in place and the manuscript is more difficult to revise. The outline will also allow all proposed authors a chance to input into the manuscript at an early stage and make suggestions that could potentially strengthen the final piece of work.

How much detail should be included in an outline?

The first outline should not be overly detailed as this allows for the best discussion about the proposed structure and makes people more likely to offer input. I would suggest that key statements are included, without over much detail, in the introduction and discussion sections and more detail in the methods and discussion section as these are fixed and will enable people to make more informed comments on the discussion section.

Could an example be given?

If I were to be writing a manuscript on the epidemiology of type 2 diabetes in the South Asian population of the UK, the first outline for the introduction might look something like this:

  • Type 2 diabetes is a major public health issue and the incidence is increasing
  • Type 2 diabetes is more prevalent in South Asian individuals
  • It has been suggested that there are differences between South Asian and White people in the presentation and outcomes of type 2 diabetes
  • For example lower BMI cut-offs have been suggested for overweight and obesity in South Asian individuals
  • The South Asian population is not in fact a single entity, but rather made up of a number of different sub-populations
  • This database study investigates the clinical characteristics at diagnosis of type 2 diabetes for people from these sub-populations

This (fictional) example hopefully highlights the broad nature of statements made and it can be seen how these build up to the research question, showing why the issue is important. Once all authors have agreed upon the first outline, the points can be expanded upon and fully referenced.