Let’s go back to that weight loss tea I was discussing yesterday. After questioning your friend B a little more about it, you also learnt that the company’s website claims something about 9 out of 10 women having a result after four weeks of tea consumption.
Great, they used science!
No, not so fast. How was the test done is the important question here. It is perhaps at this point in an inquiry of this sort that one can get frustrated with a skeptic. After all, we can’t know everything, and being so damn rigorous about every little thing sucks the fun out of life, doesn’t it?
Perhaps it’s more fun to just do things on a whim and try that wonderful-sounding tea, but it’s expensive, remember. You could be doing a lot more fun things with that money if the tea turns out to be bullshit.
Experimentation is at the heart of how humans explore the world. Children are natural experimenters from the tiniest of ages – for example, they will repeatedly drop things until they understand that gravity exists, even before they have words for that concept. When you try a new orange marmalade and compare it to the different one you have in the fridge, that’s an experiment, too.
Of course, in the field of scientific inquiry experiments are more formal than that.
Weight loss can be achieved through several means – calorie restriction and exercising are the two most reliable methods. If you take 10 women and give them four weeks’ worth of tea to drink, it’s not enough if 9 of them come back with a result. You need to establish a control group that doesn’t get tea, and you need to control for a whole range of variables. These variables might be the alternative reason for why the tea appears to work. In an ideal experimental set-up, everything about the participants’ diet, routine, exercise regimen and similar factors would be taken into account. Ideally, you would even have several control groups, for example, tea+more exercise, tea+less food, tea+no change, as well as the already mentioned no-tea group.
Only after gathering all this data and establishing patterns one is able to tease out the effects of the tea from any other potential factors.
Ideally, the women wouldn’t even know whether the tea they are given is the magical weight loss one, or just a benign concoction. Even better if the researcher doesn’t know that, either – this is a way to control for the placebo effect, as well as bias and unintended influencing.
More on that tomorrow.
This is skeptical week. To find out why it’s skeptical week, please read Monday’s post.