What should happen after the search is complete, should we immediately start writing the systematic review?

Once you have done the first version of the search, you may find that you have too many results or not many results at all.

Too many results?

If you have too many results it is likely that your search was too broad and that rather than manually checking each reference, using the filters provided by the search engines could help refine the results. Some possibilities for refining the search are to limit it to a specific language (English, for example), or to just searching the abstract and title, rather than the full text.

Too few results?

This might be what you expected and could be the basis for the systematic review, highlighting the need for more studies in a particular area. However, you may want to broaden your search if you feel that not all relevant manuscripts have been captured. This could be done by including additional keywords, or removing limits (e.g. looking for all studies and not just those in humans).

What if the number of results seems fine?

Once you are happy that the search has captured all the appropriate results, you will want to save a list of the results, including references, title and abstract, as well as recording the initial number of results found. You will need to keep track of this for the PRISMA flowchart. These results should then be distributed to the individuals that have been allocated as reviewers and they should review each abstract and decide whether it meets the inclusion criteria defined for the review. These criteria should have been defined before the search and it is important that both reviewers use them. The reason why each abstract is rejected should also be recorded. This can be done according to the PICOS, with abstracts rejected according to not containing appropriate participants, intervention, comparator, outcome or setting, or for another reason which should be recorded.

In a perfect world both reviewers will select the same abstracts at this stage, however, if they do not, an additional reviewer should arbitrate whether the abstract should be kept. If there is confusion, the abstract should be kept, for full text review (the next step).

After the abstracts have been filtered, should we start writing?

No. You will want to get the full version of each reference and these will also need to be reviewed, in a similar manner to the abstracts, to ensure that no inappropriate references are included. At this stage, you can note any manuscripts in the references that also look appropriate for the systematic review. This will enable you to check that they have been included and if not they can be added (and noted on the PRISMA flowchart).

This process of filtering can be long and tedious, however, it should enable all the relevant literature for the systematic review to be identified. It is at this stage that the writing can commence!

Why you should publish in many venues

Many venues should not mean many journals. Journals are only one possible venue for your work and by limiting yourself to them you limit the potential reach of your work. Other scientific outlets are conferences/congresses, presentations to universities and societies, scientific blogs and magazines. If you are planning on publishing your work then one potential route would be to present it at a conference, either as a poster or an oral presentation, before writing it up for a journal article. Once the work has been presented at a conference it can become part of a talk to give at other universities or societies, and this talk once it has been given several times could be written up either as a review of the work of your lab for submission to a journal or magazine, or published on a blog.

By presenting the work in these different ways, a number of different audiences can be reached and introduced to your work. They can also be excellent as opportunities to network and grow your circle of associates, possibly helping to bring about collaborations and secure funding. I would therefore suggest that if you are early in your career you should be trying to get your work seen and heard of. You might not be able to be invited to present your work to another university or interested group. However, you should be submitting your work, once it has reached sufficient quantity and quality, to congresses, or offering to talk about your work to undergraduates or other research groups you know. These might also have the added benefit of providing new thoughts and ideas on your work from people with fresh perspectives, as well as identifying any holes that need to be looked into.

Another way to be able to talk about your research with more people is to volunteer for societies and attend local interest group meetings. This may bring you into contact with influential people you will not meet in any other way.

Don’t just write, make sure you read as well

One of my favourite quotes about writing comes from Austin Kleon who says:

“In every undergraduate creative writing workshop I was part of, there was that one kid who said, ‘I like to write, but I don’t really like to read,’ and it was evident right away that you could pretty much write that kid off completely.”

This is because if we write without reading first we cannot build upon the work of others and we cannot learn from people who have published before. This should be self-evident but often when we are researching and experimenting, we will do a search to help us overcome an issue and will skim through to locate what is hopefully the answer. You might not fully read the introduction or conclusions, not really caring for the context or analysis for someone else.  We do not have time to read each article in depth and have time for research, for writing, for lecturing, for mentoring and for a life outside the research institute. Yet here I am suggesting that you should be reading more in our already packed lives.

If we do not want to learn and develop then this is not something that you should commit to, however, if you do there is always time to be found, even if it is just 15 minutes a day, possibly whilst sitting on the toilet. To improve you should read great writing; this does not have to be a ‘classic’ like Crime and Punishment, but should be something relevant. I would suggest that as writing up research is akin to non-fiction writing, it can be beneficial to read scientific non-fiction books, which can also provide ideas for research or help develop new ways of thinking. This can be science magazines, books or blogs.

Some suggestions for things to read

However, we shouldn’t limit ourselves, great writing is great writing and we can always learn something from it, hopefully, how to tell the best stories possible.

Austin Kleon – http://tumblr.austinkleon.com/post/33792291289

Don’t dilute your output

This probably sounds obvious, and it should be. Nowadays, scientists and researchers are judged, rightly or wrongly, by their publication history. This has contributed to the explosion in the amount of scientific literature available and if you browse it you will often see work that makes you wonder why it was published and what it adds. The obvious answer to this is that it adds to someone’s publication list and that is the only reason it was written up and submitted to a journal. Do you want to be one of the people who add to this pile of ‘irrelevant’ literature? I would hazard a guess that you do not want this. Therefore, this first piece of advice must be given with a proviso that you should write regularly, but only publish relevant work. This might necessitate a change in the way you think about publishing, for example if you have a series of experiments that confirm somebody else’s work that you have done as the groundwork for a research stream, you may want to publish them on your own website if there have been other people also publishing on this. This means that you will not be diluting your work and you will be known for your strong publications.

How should you decide who will be an author?

When deciding who should be an author, you might feel pressure to include people who have not directly contributed to the work being written about. For example, people who secured funding for the work or who are important in the department that you work in, might say that they should be authors.

As this might lead to questions about ghostwriting and honorary authorship, the International Committee of Medical Journal Editors produced a uniform set of criteria for authorship. These are:

Authors “should have participated sufficiently in the work to take public responsibility for relevant portions of the content” and should meet all three conditions below:

  • Substantial contributions to the conception or design of the work; or the acquisition, analysis, or interpretation of data for the work; AND
  • Drafting the work or revising it critically for important intellectual content; AND
  • Final approval of the version to be published; AND
  • Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

These criteria are also included in GPP2 produced by the International Society for Medical Publication Professionals, which was published in BMJ.

If a person has contributed to some steps but not others then they should be included in the acknowledgements. This might include people who have helped perform an experiment, but not helped in the design, or who have proof read the article prior to submission.

The guidelines also suggest that if the study is very large there could be an authoring committee that takes full responsibility for the content of the manuscript. The full list of study investigators can then be included as an appendix. The article can then be published on behalf of everyone who is involved and if a person who is not on the authoring committee wants to include the manuscript on their CV they can do so.

Hopefully, by having articles published by authority figures on this topic you can feel better able to push back on any demands from people who you feel should not be an author.

References

ICMJE authorship criteria – http://www.icmje.org/recommendations/browse/roles-and-responsibilities/defining-the-role-of-authors-and-contributors.html

GPP2 – http://www.ismpp.org/gpp2