What should happen after the search is complete, should we immediately start writing the systematic review?

Once you have done the first version of the search, you may find that you have too many results or not many results at all.

Too many results?

If you have too many results it is likely that your search was too broad and that rather than manually checking each reference, using the filters provided by the search engines could help refine the results. Some possibilities for refining the search are to limit it to a specific language (English, for example), or to just searching the abstract and title, rather than the full text.

Too few results?

This might be what you expected and could be the basis for the systematic review, highlighting the need for more studies in a particular area. However, you may want to broaden your search if you feel that not all relevant manuscripts have been captured. This could be done by including additional keywords, or removing limits (e.g. looking for all studies and not just those in humans).

What if the number of results seems fine?

Once you are happy that the search has captured all the appropriate results, you will want to save a list of the results, including references, title and abstract, as well as recording the initial number of results found. You will need to keep track of this for the PRISMA flowchart. These results should then be distributed to the individuals that have been allocated as reviewers and they should review each abstract and decide whether it meets the inclusion criteria defined for the review. These criteria should have been defined before the search and it is important that both reviewers use them. The reason why each abstract is rejected should also be recorded. This can be done according to the PICOS, with abstracts rejected according to not containing appropriate participants, intervention, comparator, outcome or setting, or for another reason which should be recorded.

In a perfect world both reviewers will select the same abstracts at this stage, however, if they do not, an additional reviewer should arbitrate whether the abstract should be kept. If there is confusion, the abstract should be kept, for full text review (the next step).

After the abstracts have been filtered, should we start writing?

No. You will want to get the full version of each reference and these will also need to be reviewed, in a similar manner to the abstracts, to ensure that no inappropriate references are included. At this stage, you can note any manuscripts in the references that also look appropriate for the systematic review. This will enable you to check that they have been included and if not they can be added (and noted on the PRISMA flowchart).

This process of filtering can be long and tedious, however, it should enable all the relevant literature for the systematic review to be identified. It is at this stage that the writing can commence!

Why you should publish in many venues

Many venues should not mean many journals. Journals are only one possible venue for your work and by limiting yourself to them you limit the potential reach of your work. Other scientific outlets are conferences/congresses, presentations to universities and societies, scientific blogs and magazines. If you are planning on publishing your work then one potential route would be to present it at a conference, either as a poster or an oral presentation, before writing it up for a journal article. Once the work has been presented at a conference it can become part of a talk to give at other universities or societies, and this talk once it has been given several times could be written up either as a review of the work of your lab for submission to a journal or magazine, or published on a blog.

By presenting the work in these different ways, a number of different audiences can be reached and introduced to your work. They can also be excellent as opportunities to network and grow your circle of associates, possibly helping to bring about collaborations and secure funding. I would therefore suggest that if you are early in your career you should be trying to get your work seen and heard of. You might not be able to be invited to present your work to another university or interested group. However, you should be submitting your work, once it has reached sufficient quantity and quality, to congresses, or offering to talk about your work to undergraduates or other research groups you know. These might also have the added benefit of providing new thoughts and ideas on your work from people with fresh perspectives, as well as identifying any holes that need to be looked into.

Another way to be able to talk about your research with more people is to volunteer for societies and attend local interest group meetings. This may bring you into contact with influential people you will not meet in any other way.

Don’t dilute your output

This probably sounds obvious, and it should be. Nowadays, scientists and researchers are judged, rightly or wrongly, by their publication history. This has contributed to the explosion in the amount of scientific literature available and if you browse it you will often see work that makes you wonder why it was published and what it adds. The obvious answer to this is that it adds to someone’s publication list and that is the only reason it was written up and submitted to a journal. Do you want to be one of the people who add to this pile of ‘irrelevant’ literature? I would hazard a guess that you do not want this. Therefore, this first piece of advice must be given with a proviso that you should write regularly, but only publish relevant work. This might necessitate a change in the way you think about publishing, for example if you have a series of experiments that confirm somebody else’s work that you have done as the groundwork for a research stream, you may want to publish them on your own website if there have been other people also publishing on this. This means that you will not be diluting your work and you will be known for your strong publications.

How should you decide who will be an author?

When deciding who should be an author, you might feel pressure to include people who have not directly contributed to the work being written about. For example, people who secured funding for the work or who are important in the department that you work in, might say that they should be authors.

As this might lead to questions about ghostwriting and honorary authorship, the International Committee of Medical Journal Editors produced a uniform set of criteria for authorship. These are:

Authors “should have participated sufficiently in the work to take public responsibility for relevant portions of the content” and should meet all three conditions below:

  • Substantial contributions to the conception or design of the work; or the acquisition, analysis, or interpretation of data for the work; AND
  • Drafting the work or revising it critically for important intellectual content; AND
  • Final approval of the version to be published; AND
  • Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

These criteria are also included in GPP2 produced by the International Society for Medical Publication Professionals, which was published in BMJ.

If a person has contributed to some steps but not others then they should be included in the acknowledgements. This might include people who have helped perform an experiment, but not helped in the design, or who have proof read the article prior to submission.

The guidelines also suggest that if the study is very large there could be an authoring committee that takes full responsibility for the content of the manuscript. The full list of study investigators can then be included as an appendix. The article can then be published on behalf of everyone who is involved and if a person who is not on the authoring committee wants to include the manuscript on their CV they can do so.

Hopefully, by having articles published by authority figures on this topic you can feel better able to push back on any demands from people who you feel should not be an author.


ICMJE authorship criteria – http://www.icmje.org/recommendations/browse/roles-and-responsibilities/defining-the-role-of-authors-and-contributors.html

GPP2 – http://www.ismpp.org/gpp2

What should be considered when writing a first draft?

By the time you get to writing a first draft of a manuscript I would expect the following steps to have already been completed:

  • Author list confirmed
  • Data fully analysed and data for the manuscript selected
  • Target journal selected
  • Target journal author guidelines read and any appropriate guidelines (e.g. word counts, required sections) noted down, preferably on a cover page to the manuscript
  • Outline written and content agreed/commented on by all authors
  • Extended outline written and agreed/commented on by all authors

If an extended outline exists…

If an extended outline has already been written then the jump to a full first draft should be fairly easy. The bulleted text will be made into complete sentences and linking text added. At this stage further logical gaps in the “story” might be identified and highlight the need for additional introductory text or research. However, this should be fairly minor as these issues will hopefully be identified at the extended outline stage.

If an extended outline does not exist…

I would suggest that if one does not exist then it is beneficial to put one together before writing the manuscript in prose, even if you are not sending it to co-authors for review. This will make the writing of the first draft much easier and should enable the logical flow of the manuscript to be worked out as discussed in my previous blog post.

Remember to write to journal guidelines at this stage to avoid unnecessary editing later on.

What is an extended outline and why might it be beneficial?

Once an outline has been written, discussed with co-authors, revised as appropriate in line with any comments and been agreed upon, the next step is to bulk out the outline. I believe that producing an extended outline in bulleted form, rather than a first draft written in prose is the best next step. This ‘extended’ outline should be the first draft in bullet form and contain everything that is expected to go into the first full draft.

Why an additional step?

In my experience, it is much easier to make revisions to the order information is presented in, when it is in bullet form. It might be that as you write you realise that the logical flow from one point to the next is wrong and that it is might be improved by shifting bits around. Or it could be that when your co-authors read the manuscript they highlight places where more information is needed, which could again change the flow of the document. If the document is written in full sentences, it can be hard to make these changes as it is difficult to see where it might fit. It is much easier to revise a ‘work in progress’ that a ‘fait accompli’. Overall, I have found that adding in this step saves time in the long run and often results in a better manuscript.

Why write an outline?

I, like many other people I expect, have previously tried to write a novel. I attempted it as part of NaNoWriMo, in which people endeavor to write a 50,000 word ‘novel’ during the month of November. I sadly failed in this attempt and looking back I can see two reasons why I failed. Firstly, although I liked the idea I was sadly not that dedicated to spending all my spare time writing and so only managed about 25,000 words. Secondly, I ‘pantsed’ rather than planned – that is I thought that the great novel would flow fully formed without any real forethought or planning. The resultant work was a complete mess, with what amounted to three discrete short stories linked by events, none of which were that good. This experience has shown me the value of planning or outlining at least some aspects of a novel before starting and in future this is what I will be doing.

Why is this relevant to writing about science?

I feel that the same can be true when writing about science, although often to a lesser degree, as the planning and research for conducting experiments should hopefully lend an idea of structure. Nonetheless, whilst it is often easy to list of the key points in the narrative of the research, these can be lost or forgotten when writing a prose document resulting in a loss of coherence.

I would, therefore, suggest writing an outline when starting to work on a document. This should contain the key points for each section and detail any data to be included. This outline will enable you to see if the manuscript makes sense and if there are any gaps, either data gaps or gaps in reasoning, before more words are in place and the manuscript is more difficult to revise. The outline will also allow all proposed authors a chance to input into the manuscript at an early stage and make suggestions that could potentially strengthen the final piece of work.

How much detail should be included in an outline?

The first outline should not be overly detailed as this allows for the best discussion about the proposed structure and makes people more likely to offer input. I would suggest that key statements are included, without over much detail, in the introduction and discussion sections and more detail in the methods and discussion section as these are fixed and will enable people to make more informed comments on the discussion section.

Could an example be given?

If I were to be writing a manuscript on the epidemiology of type 2 diabetes in the South Asian population of the UK, the first outline for the introduction might look something like this:

  • Type 2 diabetes is a major public health issue and the incidence is increasing
  • Type 2 diabetes is more prevalent in South Asian individuals
  • It has been suggested that there are differences between South Asian and White people in the presentation and outcomes of type 2 diabetes
  • For example lower BMI cut-offs have been suggested for overweight and obesity in South Asian individuals
  • The South Asian population is not in fact a single entity, but rather made up of a number of different sub-populations
  • This database study investigates the clinical characteristics at diagnosis of type 2 diabetes for people from these sub-populations

This (fictional) example hopefully highlights the broad nature of statements made and it can be seen how these build up to the research question, showing why the issue is important. Once all authors have agreed upon the first outline, the points can be expanded upon and fully referenced.