Not only do unproven treatments need to be tested but the test needs to be fair.
Without a fair test, the findings from any research may not mean much. Even worse, an unfair test could give health professionals the wrong idea and people may be given treatment that does not work, or may not be given treatment that could be helpful.
This page explains:
Size of trials
If someone who is ill takes a treatment and then gets better it could be a natural recovery that would have happened anyway.
To tell if the treatment has worked, it needs to be compared to another treatment or a placebo (see below). The two results have to be different enough to indicate a difference has not occurred by chance.
Comparing a treatment with a placebo
The treatment may be compared with a placebo (a dummy treatment), such as a sugar pill, that looks the same as the treatment.
If there are fewer symptoms after a treatment than after a placebo, then this suggests that the treatment is the cause of the difference.
Comparing a treatment with a standard treatment
Where there is already a treatment known to be effective from previous research, it is usually not considered right (ethical) to compare the new treatment with placebo. The new treatment usually needs to be compared with a standard treatment that is already known to be helpful.
Sometimes a doctor or other health professional’s reassurance and their confident way of communicating with people who are ill actually helps those people to get better. It is a largely mysterious and fascinating effect which can be quite powerful.
If you think and believe you are going to get better, you're much more likely to (although this doesn't always work, and not for all conditions).
It is similar with placebos, as dummy treatments may be given to people in clinical trials. A placebo medicine looks the same as the medicine being studied, so you don't know which one you're taking.
Placebos are particularly powerful in conditions where symptoms are important. For example, people feel pain differently and respond better to treatments they think are going to work. In extreme circumstances some people in severe pain respond to a placebo apparently as well as they would to a powerful painkiller.
Placebos do not work for all conditions. High blood pressure can be lowered by active medicines but placebos have no detectable effect. Similarly, placebo treatments do not lower blood cholesterol, but statin medicines do.
Sham treatments that work
Researchers have designed ways of creating placebos for complementary medicine treatments like acupuncture. It's possible to carry out sham acupuncture, in which needles are inserted to a different depth and in different places than those used in real Chinese acupuncture. In recent trials, both types of acupuncture appeared to be better than doing nothing.
There have also been studies that have carried out placebo surgery on people with knee pain. The placebo treatment often comes out well.
Participants in a clinical trial will be put into one of two groups:
- the test group, where they are given the treatment being studied, or
- the control group, where they are given the comparison treatment (the placebo or the standard treatment).
The aim is to compare what happens in the test group with what would have happened anyway in the control group. Participants are randomly assigned to one of these groups without knowing which group they are in (see Randomisation and blinding, below).
While treatments are different in the two groups, as much else as possible stays the same. For example, both groups should have people of a similar age, with a similar proportion of men and women, who are in similar overall health.
The best way to get similar groups is to rely on chance.
This means that individuals are allocated to one of the groups in the trial in an unpredictable, random way. It increases the chance that the two groups will be similar and so makes the trial less biased. This process is called randomisation.
In most trials a computer, not a doctor, will randomly decide which group you go into.
Randomisation means that the treatment someone receives cannot be influenced by researchers. Such bias, which may not be deliberate, would make the test unfair.
Many trials are set up so that no one knows who is taking which treatment. This is known as blinding and helps to reduce the effects of bias.
Many people feel better if they think they're getting a better new treatment, even if the treatment is ineffective and their medical condition has not really changed at all.
When both the medical staff organising the treatment and those taking part in the trial do not know who is receiving which treatment, it is a double-blind trial.
Blinding is easier when testing medicines, but more difficult when testing other types of therapy or methods of caring for people. For example, it may be impossible to blind a trial that is comparing two types of surgery.
Why blinding is important
Some clinical trials measure survival, so it is obvious if the treatment works better than the control, assuming the studies are unbiased and large enough.
Many trials look at outcomes that are less easy to measure with certainty. For example, patients and researchers may have to make some sort of judgement on how bad symptoms are.
If either researchers or participants know, or think they know, who is on which treatment (including placebo), the knowledge may influence what they report.
- Participants who think they are taking an active treatment may not want to let down the researcher, and may exaggerate benefits and minimise side effects.
- Researchers may allow their hopes about a new treatment to unconsciously influence their recording of symptoms.
The result of these biases is often to overestimate how effective the treatment is. To reduce these possible sources of bias, many trials are double-blinded.
Size of trials
For a trial to be a fair test, the number of people taking part needs to be large enough.
For example in a small trial of 20 people with 10 taking each treatment, seven people may improve on the new treatment and five on the standard treatment.
Most of us would not think of that as a fair test, because, while it may be that the new treatment is better, the finding could easily have been chance.
However, if the trial were of two groups of 100 people, and 70 people improved on the new treatment and 50 on the standard treatment, it is more likely that the new treatment was better.
If the trial was even bigger, with 700 out of 1,000 improving on the new treatment and 500 out of 1,000 on the standard treatment, that gives more confidence that the new treatment was better.
The degree of this confidence can be estimated. Tests of statistical significance help to identify where differences between treatments are unlikely to be due to chance.
Further information on fair tests can be found from the James Lind Library (external site).