Is all scientific research equally valid?


I don’t know about you, but I often get into discussions with people and end up citing research that backs up my claims. It is not uncommon for the other person to say something like “Yea, but I bet there’s other research that says the opposite” or “I don’t trust any research – it’s all biased one way or another…”

So, if you do consider that some research is worthy of respect but get a bit frustrated trying to find research that you can really trust, where do you go to find it?

Some of my usual favourites for nutritional research are PubMed, Nutritionfacts.org, Physicians Committee for Responsible Medicine, Centre for Nutrition Studies, British Journal of Nutrition, There are also lots of nutrition journals, such as The British Journal of Nutrition, Food and Nutrition Sciences, Nutrition Journal, and the Journal of Human Nutrition and Food Science.

But, and it’s a big BUT…There are times when bias can be detected in some of the research that we come across.

For instance, we might discover that the research was funded by an organisation that wanted to see an outcome which was favourable for their purposes – whether academic or financial. Also, it is quite possible that individual researchers within the studies may have had personal or professional bias. And whilst the process of peer-review is meant to ensure a high level of transparency and honesty with the reviewed research, this is sadly not always the case.

However, there is an interesting organisation called the USPSTF (United States Preventive Services Task Force) that I would like to talk about with you. I am not stating any opinion about individual research projects that they have covered, nor am I claiming that they are the gold standard in research that everyone should trust implicitly; however, they have a really interesting method of reviewing current research findings on a given subject and it is this that I want to share with you.

It is their function to review all available published research and then publish a paper that states, in their considered opinion, whether the overall results found (for instance, on the effectiveness of mammograms or prostate cancer screening) suggest that current medical/health practices are helpful, harmful or neutral in their impact on individuals in particular and on society in general. Based on this, they then make recommendations to governments, organisations and individuals.

Their way of doing this is as close to people-power (that is, the empowerment of the average person in the street) as I have found recently in this academic field. This is how it works (cartoons are my addition!):

Recommendations Development Process: A Graphic Overview

Step 1. Topic Nomination

Anyone can nominate a new topic or an update to an existing topic at any time, via the Task Force Web site. The Task Force prioritises topics based on several criteria, including the topic’s relevance to prevention and primary care, importance for public health, potential impact of the recommendation, and whether there is new evidence that may change a current recommendation.

Step 2. Draft and Final Research Plans

Once a topic is selected, the Task Force and researchers from an Evidence-based Practice Centre (EPC), develop a draft research plan for the topic. This plan includes key questions to be answered and target populations to be considered. The draft research plan is posted on the Task Force’s Web site for four weeks, during which anyone [that includes you and me] can comment on the plan. The Task Force and the EPC review all comments and consider them while making any necessary revisions to the research plan. The Task Force then finalises the plan and posts it on its Web site.

Step 3. Draft Evidence Review and Draft Recommendation Statement

Using the final research plan as a guide, EPC researchers gather, review, and analyse evidence on the topic from studies published in peer-reviewed scientific journals. The EPC then develops one or more draft evidence reviews summarising the evidence on the topic. Members discuss the evidence reviews and use the information to determine the effectiveness of a service by weighing the potential benefits and harms. Members then develop a draft recommendation statement based on this discussion. The draft evidence review and draft recommendation statement are posted on the Task Force Web site for four weeks.

Step 4. Final Evidence Review and Final Recommendation Statement

The Task Force and EPC consider all comments on draft evidence reviews and the Task Force considers all comments on the draft recommendation statement. The EPC revises and finalises the evidence reviewed and the Task Force finalises the recommendation statement based on both the final evidence review and the public comments.

All final recommendation statements and evidence reviews are posted on the Task Force’s Web site. The final recommendation statement and a final evidence summary, a document that outlines the evidence it reviewed, are also published in a peer-reviewed scientific journal.

Interesting eh?

Final Comment

Never underestimate the tactics used by both organisations and individuals to misrepresent information in order to mislead the public for their own agendas.

If you come across conflicting and troubling opinions about nutrition (whether expressed by friends and family, in newspapers and magazine, on the TV or internet sites, in research papers or books), you can send me links to the information and I will take a look at it and help you to analyse it in a way that allows you to form your own opinion about the validity of the claims made.

There’s no better way to ensure that you have the motivation to continue with the optimal WFPB dietthan when you know intellectually that your nutrition and lifestyle decisions are backed-up by solid and reliable scientific research.

In future posts, I will outline and review various research methods, as well as introduce some intriguing alternative opinions about nutritional research as expressed by Prof. T Colin Campbell.