What should happen after the search is complete, should we immediately start writing the systematic review?

Once you have done the first version of the search, you may find that you have too many results or not many results at all.

Too many results?

If you have too many results it is likely that your search was too broad and that rather than manually checking each reference, using the filters provided by the search engines could help refine the results. Some possibilities for refining the search are to limit it to a specific language (English, for example), or to just searching the abstract and title, rather than the full text.

Too few results?

This might be what you expected and could be the basis for the systematic review, highlighting the need for more studies in a particular area. However, you may want to broaden your search if you feel that not all relevant manuscripts have been captured. This could be done by including additional keywords, or removing limits (e.g. looking for all studies and not just those in humans).

What if the number of results seems fine?

Once you are happy that the search has captured all the appropriate results, you will want to save a list of the results, including references, title and abstract, as well as recording the initial number of results found. You will need to keep track of this for the PRISMA flowchart. These results should then be distributed to the individuals that have been allocated as reviewers and they should review each abstract and decide whether it meets the inclusion criteria defined for the review. These criteria should have been defined before the search and it is important that both reviewers use them. The reason why each abstract is rejected should also be recorded. This can be done according to the PICOS, with abstracts rejected according to not containing appropriate participants, intervention, comparator, outcome or setting, or for another reason which should be recorded.

In a perfect world both reviewers will select the same abstracts at this stage, however, if they do not, an additional reviewer should arbitrate whether the abstract should be kept. If there is confusion, the abstract should be kept, for full text review (the next step).

After the abstracts have been filtered, should we start writing?

No. You will want to get the full version of each reference and these will also need to be reviewed, in a similar manner to the abstracts, to ensure that no inappropriate references are included. At this stage, you can note any manuscripts in the references that also look appropriate for the systematic review. This will enable you to check that they have been included and if not they can be added (and noted on the PRISMA flowchart).

This process of filtering can be long and tedious, however, it should enable all the relevant literature for the systematic review to be identified. It is at this stage that the writing can commence!

Why you should publish in many venues

Many venues should not mean many journals. Journals are only one possible venue for your work and by limiting yourself to them you limit the potential reach of your work. Other scientific outlets are conferences/congresses, presentations to universities and societies, scientific blogs and magazines. If you are planning on publishing your work then one potential route would be to present it at a conference, either as a poster or an oral presentation, before writing it up for a journal article. Once the work has been presented at a conference it can become part of a talk to give at other universities or societies, and this talk once it has been given several times could be written up either as a review of the work of your lab for submission to a journal or magazine, or published on a blog.

By presenting the work in these different ways, a number of different audiences can be reached and introduced to your work. They can also be excellent as opportunities to network and grow your circle of associates, possibly helping to bring about collaborations and secure funding. I would therefore suggest that if you are early in your career you should be trying to get your work seen and heard of. You might not be able to be invited to present your work to another university or interested group. However, you should be submitting your work, once it has reached sufficient quantity and quality, to congresses, or offering to talk about your work to undergraduates or other research groups you know. These might also have the added benefit of providing new thoughts and ideas on your work from people with fresh perspectives, as well as identifying any holes that need to be looked into.

Another way to be able to talk about your research with more people is to volunteer for societies and attend local interest group meetings. This may bring you into contact with influential people you will not meet in any other way.

What are the next steps after the first draft is written?

Once you have produced your first draft and your co-authors have reviewed it, your aim should be to address the comments and get the manuscript submitted to your chosen journal as soon as possible. How you manage this will depend upon the comments you receive.

If they are minimal

  • Make the suggested changes, or ignore comments if you have a reason to do so and can explain it to the person who made the comment
  • Ensure the manuscript is in the style requested by the journal
  • Confirm that all authors are happy to submit
  • Submit to the journal

If there are major or conflicting comments

This will necessitate the development of a second draft and a further round of review. If the comments are conflicting then it is often best to have a teleconference to discuss them and ensure that everyone is in agreement before making any changes. If agreement cannot be reached then the lead author (or the guarantor for the manuscript) should decide what to do.

Depending on the comments on the next draft the cycle will then continue. However, it is important to remember that the manuscript is being written to be published and you should not have innumerable review rounds, demanding perfection if this is at the cost of submitting.

What should be considered when writing a first draft?

By the time you get to writing a first draft of a manuscript I would expect the following steps to have already been completed:

  • Author list confirmed
  • Data fully analysed and data for the manuscript selected
  • Target journal selected
  • Target journal author guidelines read and any appropriate guidelines (e.g. word counts, required sections) noted down, preferably on a cover page to the manuscript
  • Outline written and content agreed/commented on by all authors
  • Extended outline written and agreed/commented on by all authors

If an extended outline exists…

If an extended outline has already been written then the jump to a full first draft should be fairly easy. The bulleted text will be made into complete sentences and linking text added. At this stage further logical gaps in the “story” might be identified and highlight the need for additional introductory text or research. However, this should be fairly minor as these issues will hopefully be identified at the extended outline stage.

If an extended outline does not exist…

I would suggest that if one does not exist then it is beneficial to put one together before writing the manuscript in prose, even if you are not sending it to co-authors for review. This will make the writing of the first draft much easier and should enable the logical flow of the manuscript to be worked out as discussed in my previous blog post.

Remember to write to journal guidelines at this stage to avoid unnecessary editing later on.

Why write an outline?

I, like many other people I expect, have previously tried to write a novel. I attempted it as part of NaNoWriMo, in which people endeavor to write a 50,000 word ‘novel’ during the month of November. I sadly failed in this attempt and looking back I can see two reasons why I failed. Firstly, although I liked the idea I was sadly not that dedicated to spending all my spare time writing and so only managed about 25,000 words. Secondly, I ‘pantsed’ rather than planned – that is I thought that the great novel would flow fully formed without any real forethought or planning. The resultant work was a complete mess, with what amounted to three discrete short stories linked by events, none of which were that good. This experience has shown me the value of planning or outlining at least some aspects of a novel before starting and in future this is what I will be doing.

Why is this relevant to writing about science?

I feel that the same can be true when writing about science, although often to a lesser degree, as the planning and research for conducting experiments should hopefully lend an idea of structure. Nonetheless, whilst it is often easy to list of the key points in the narrative of the research, these can be lost or forgotten when writing a prose document resulting in a loss of coherence.

I would, therefore, suggest writing an outline when starting to work on a document. This should contain the key points for each section and detail any data to be included. This outline will enable you to see if the manuscript makes sense and if there are any gaps, either data gaps or gaps in reasoning, before more words are in place and the manuscript is more difficult to revise. The outline will also allow all proposed authors a chance to input into the manuscript at an early stage and make suggestions that could potentially strengthen the final piece of work.

How much detail should be included in an outline?

The first outline should not be overly detailed as this allows for the best discussion about the proposed structure and makes people more likely to offer input. I would suggest that key statements are included, without over much detail, in the introduction and discussion sections and more detail in the methods and discussion section as these are fixed and will enable people to make more informed comments on the discussion section.

Could an example be given?

If I were to be writing a manuscript on the epidemiology of type 2 diabetes in the South Asian population of the UK, the first outline for the introduction might look something like this:

  • Type 2 diabetes is a major public health issue and the incidence is increasing
  • Type 2 diabetes is more prevalent in South Asian individuals
  • It has been suggested that there are differences between South Asian and White people in the presentation and outcomes of type 2 diabetes
  • For example lower BMI cut-offs have been suggested for overweight and obesity in South Asian individuals
  • The South Asian population is not in fact a single entity, but rather made up of a number of different sub-populations
  • This database study investigates the clinical characteristics at diagnosis of type 2 diabetes for people from these sub-populations

This (fictional) example hopefully highlights the broad nature of statements made and it can be seen how these build up to the research question, showing why the issue is important. Once all authors have agreed upon the first outline, the points can be expanded upon and fully referenced.

What is a key message and why have only one?

When you read a scientific article to gain knowledge of what people have done and what they are thinking, what do you take away? Often when I was working in the lab I would specifically look for reaction conditions that might work for my experiments and ignore the introduction and discussion. However, if you read the entire article you likely will not remember exact experimental conditions once finished, rather you will hopefully leave with the idea that the article answered a question and you will take away an idea. This idea is the key message for the manuscript.

Determining the key message

Often when writing articles you will discuss what the message will be when planning the article and putting together the outline. This message will infuse the article, being proposed or hinted at in the introduction, expanded upon in the discussion and possibly explicitly stated in the concluding remarks. However, if you try to include two separate messages then you will muddy the message and it will be less impactful.

For example, if my message is that the new treatment I am discussing is efficacious and well tolerated I can easily build the article around this message. Whereas if the messages are that the treatment is efficacious and well tolerated, and also that it is extremely cost-effective compared to therapies currently on the market, then there will be two narratives running through the article aimed at different groups of people and this may cause confusion.

What to do if you want to include multiple key messages?

I would suggest that if you want to have two different messages for different audiences that you consider writing two articles. This will improve the ability of the article to reach the appropriate audience, and also allow each message to have maximum impact.

Why should you read author guidelines before writing a first draft?

By the time a first draft is written I always feel that the following should have been completed:

  • Type of article/manuscript being written
  • Author list decided
  • Outline put together showing what work will be included and discussed
  • Outline agreed upon by all authors and their input included
  • Target journal selected

At this stage there is the excitement of actually writing up the work, however, it is important not to dive in without knowledge of any rules you may have to conform to. There may be a limit to the number of words, figures or tables to include, or specific guidance on section titles to use or how to draw figures. If you write the draft without knowing any of this you may have to go back and edit your work, possibly significantly. I find it much easier to write a smaller number of words originally, rather than having to delete some. This is because once something is in writing it seems important to the story and is harder to let go. Additionally, it is easier to write to a particular structure rather than contorting the work to fit once completed.

However, you may be thinking that these are only guidelines and that your work is so amazing that no matter what state it is submitted in it will be published. This may be the case but often the journal will request you edit to journal guidelines before acceptance or even before consideration. So writing in line with the guidelines should save you work later on and maybe also increase the chances of your manuscript being accepted.