What next for your systematic review – Fully define your research hypothesis or question.

So you have decided that you are going to put together a systematic review and you have managed to persuade some of your colleagues and peers to be co-authors. What might you do next to get the review done?

Have a meeting.

The first thing I would recommend is that you get everyone who is to be involved together to define the problem. This is important because everybody will come to the table with different mind sets and different backgrounds, and also prior knowledge. Together you can make sure that everyone is agreed as to what problem will be addressed.

However, you should not go beyond that in the meeting, rather once the problem has been clarified you should end the meeting and get everybody to think of what hypothesis or question your review will be addressing. This way you will not end up with just the point of view of the most respected author or the person who shouts the loudest. Also, I find that my best ideas come when I am not really thinking directly about them and by leaving time to think you provide the opportunity for people to synthesise what was discussed and come up with any solutions or points that need clarification that were missed during the meeting.

This thinking time should not be extensive and should be defined during the meeting by booking the next meeting in for a couple of days later.

Have another meeting

Everybody has hopefully now thought about the issues and come up with ideas for the hypothesis. At this second meeting you will review everybody’s ideas. Let each person present their ideas in turn without allowing any questions to ensure that people are not intimidated and only present part of what they have come up with.

Once everybody’s ideas have been presented you can then discuss the merits of them all and finesse the hypothesis until you have it agreed by all authors. At this point you will need to discuss any limits people feel will be necessary for the systematic review, will it be limited for example by date, inclusion criteria, size or specific endpoint measures. These limits will be very important once the search has been run as you will typically need to filter huge numbers of abstracts to determine the few that will be included and as suggested by the type of review your process needs to be systematic.

What next?

The next step, once the hypothesis or question has been defined, is to define the search and that will be discussed in the next post.

References

I have taken a lot of the ideas for this post from an Accidental Creative podcast episode I listened to on brainstorming. This episode can be found here and is well worth a listen.

Some thoughts on systematic reviews

I remember that in my first year of university (in 2000) we were taken to the library and shown how to search the Beilstein Handbook of Organic Chemistry to identify previous instances of reactions we might be interested in. There was a wall of books and it was quite intimidating. However, I never had to look at those books because there was also an online searchable database which made the task much easier. I cannot imagine how much time must have been spent by chemists searching for reactions and reaction conditions, that was suddenly freed up by taking the data online.

This has happened with most data sources and we can now search through records of journal publications and patents with ease. This I believe has led to the increase in the publication of the systematic review and the decrease in the importance placed on narrative reviews. It has changed so much that, nowadays, before many research projects are fully initiated, a systematic review is undertaken to explore, among other things, what is already known about the topic and whether the research project is novel. Owing to their inclusion of all available, relevant data systematic reviews are considered the pinnacle of evidence in evidence-based medicine [Wikipedia]. This is, in my mind, embodied by the Cochrane Collaboration, which has a handbook on how to perform systematic reviews for inclusion in their database.

It is, therefore, important that all researchers understand what a systematic review is and how to perform one. In addition, it is important to know what to do with the output. Is it okay to summarize the data, or should it be synthesised in some way, either through meta-analysis or narrative synthesis. Furthermore, we need to be able to judge the quality of published systematic reviews so that we can judge their quality and determine whether there is any bias inherent in them, or flaws in the methods used.

The PRISMA Statement

The PRISMA Statement is a great help, as it details everything to be included in a systematic review and its planning and I have found this useful when planning, performing and writing up systematic reviews, as it provides a useful checklist to bear in mind.

I want to start my systematic review. What should I do?

The very first thing you need to do is ensure you are asking an interesting question that is relevant and that you are asking it in the correct fashion. You should discuss your idea with colleagues who work in the same field and have similar interests, and also read any relevant papers that you are aware off. This might highlight aspects that you are unaware of or have overlooked. It will also provide the groundwork for the next step, which is putting together your hypothesis and developing your search strategy which will identify the references that will potentially be included in the review. You should also work out who your co-authors will be and each person’s role. For example, you will need somebody to do the search and at least two people to go through the results separately and someone to adjudicate any conflicting views about whether or not to include a manuscript in the review.

Once this is done you are ready to think about actually starting work on your systematic review, and I shall offer my thoughts on those steps in the next few posts.

Why we are not uncreative

I have recently got back into listening to podcasts and enjoy those discussing productivity and creativity. This morning I was listening to the Accidental Creative podcast and Keith Sawyer was discussing his book Zig Zag: The Surprising Path to Greater Creativity. One of the points discussed was that about 80% of creativity is learned. This seems surprising at first but when I considered it, it seemed right.

I have read a number of books (including Austin Kleon’s excellent Steal Like an Artist: 10 Things Nobody Told You About Being Creative) that highlight the need for side projects as these encourage creativity. This is because if we focus on one area only we are likely to be stuck in one way of thinking and this can impact our ability to be creative.

Many people (including David Bohm and Steve Jobs) have emphasised that creativity is not just doing art or design. rather it is coming up with something new and making unexpected connections. When we consider creativity in this way, it is odd that some people are ‘creatives’ and others are not. We often do not consider scientists to be creative, when they can be some of the most creative people there are.

Why is there confusion about creativity?

I think that this confusion about creativity, with many people claiming they are not creative, starts at school. We are either classed as academically gifted or creative or persistent or musically-gifted. Often those who decide to do science are not encouraged to do anything artistic, instead focusing on science and maths to the detriment of other studies.

At school, science and maths are about remembering facts and applying equations, rather than trying to prove new things or question established hypotheses. This school-work is not creative and sets a mindset that can last throughout our lives. When we get to university we may do some research and suddenly we are faced with a completely different subject. We have the knowledge that we need to apply, developing experiments that may prove or disprove our hypothesis. If things don’t work, we need to create different experiments to confirm the results or different pathways to what we desire. This is different from at school where if something didn’t work we just needed to redo it, as it was guaranteed to work if we did it correctly.

What do I do to increase my creativity?

I have found that making sure I have a hobby increases my creativity. This means that rather than watching TV in the evening, or sitting reading a book I am trying to develop new skills. I love printed art and zines (Whatcha Mean, What’s a Zine?: The Art of Making Zines and Mini Comics), and would eventually like to put together my own. As such, I am trying out linoprinting and collage, and have for several years been taking photos on my walk to work. These on there own are unlikely to increase my creativity, however, the planning and getting ideas for the prints means that my mind is active and coming up with ideas unrelated to work. These can then eventually lead into ideas for work and for other areas of work.

I cannot say for sure that this works, but it is a lot of fun!

Why should you write less in your presentations?

For work I sometimes get to go to conferences and see many presentations in a few short days. However, when I go to these presentations I often find that everything I need to know is written on the slides and that I do not need to listen to the speaker to fully understand what is being discussed. When this happens I tend to read ahead on the slide and might potentially miss valuable information that is spoken whilst doing so, as well as getting a bit bored whilst waiting for the next slide to appear. I expect that many of you have also experienced this and when you read about presentation technique you frequently get to hear the phrase ‘death by PowerPoint’ which relates to this phenomenon.

What can you do to avoid this?

It might seem drastic, but the way to avoid this trap is to put significantly less text on your slides. Rather than detailing your objective on a slide you could include a prompt (e.g. ‘What was the objective?’) that you would then talk around, or you could include a figure or table but not your thoughts on the table. This way people will have to listen to what you have to say to be able to fully understand what is being presented. An extreme version of this is the Takahashi method where only a single word or short phrase is on each slide to act as a prompt, forcing the speaker to talk about what they are doing and engage with the audience. The Lessig method is similar.

But what if I forget what I am talking about?

It is better to have notes and to practice your presentation to make it as smooth as possible and have it memorable, rather than having a boring presentation. People will forgive you glancing at your notes to remind yourself of a pertinent point and will feel more engaged. These can be included as slide notes in PowerPoint or written separately as a text document.

Where can I find out more about making my presentations better?

I found the book Presentation Zen: Simple Ideas on Presentation Design and Delivery (Voices That Matter) by Garr Reynolds to be extremely informative, and their is a blog by the author this book that is good. It can be found here.

Also slide:ology: The Art and Science of Creating Great Presentations: The Art and Science of Presentation Design by Nancy Duarte is excellent. There is a free multimedia version of her book Resonate online which is about visual storytelling to improve presentations.