It is also important to remember that an absence of evidence for a particular treatment is not the same as there being evidence that it doesn’t work (for a particular illness). There is little research trial evidence available for acupuncture in respect of many conditions, because these trials cost a lot of money and funding is hard to come by. This is not a situation unique to acupuncture or unorthodox therapies in general; it is the norm throughout large swathes of medicine. Only a third of treatments reported in RCTs have been found to be effective and 50% have unknown effectiveness (BMJ 2015). Greenhalgh et al (2014) provide a refreshing insight into the good and the bad in evidence based medicine.
For medical research there is a hierarchy of types of evidence, with the ones at the top of the pyramid (systematic reviews and meta-analyses) understood to offer more credible evidence than those below. Anecdotal evidence comes at the bottom and yet this is probably the most prevalent source for prospective patients, who draw on their social networks for information (Bishop and Lewith, 2013). Anecdotal reports are usually the only evidence available about the performance of particular individual practitioners. Likewise a n-of-1 trial may be much more useful than a randomized controlled trial (RCT) for charting the progress of an individual patient. Large case series or uncontrolled trials in naturalistic settings may be more informative about treatment effectiveness than small RCTs with little external validity (i.e. with little relevance to the nature of the acupuncture you are likely to receive in normal practice). Different research methods produce different sorts of evidence and are suited for different situations and purposes. There is no one sort that can tick all the boxes. A wide spread of types provides a better all-round evaluation of an (acupuncture) intervention (Rawlins, 2008). Refer to the section on Research Methods [section 5 in ’Your own research’, which comes under ‘Getting involved in research’] for further details of different types.
Nevertheless, most evidence summaries focus mainly or entirely on RCTs and reviews of RCTs. In this respect it is important to know that not all RCTs use a placebo or sham control group: acupuncture could be compared with no other treatment, orthodox medical care, or other therapies. For evaluating the benefits of acupuncture in the real world the most appropriate RCT approach is comparative effectiveness research [Witt et al 2012]. Sham controlled trials are tools for investigating the contribution of one or more specific components of the intervention in more artificial conditions. RCTs with so-called placebo controls, that purport to measure the effectiveness of acupuncture, are usually doing no such thing. Rather, they may be comparing two different versions of acupuncture (Paterson and Dieppe, 2005). Even specialist sham needle devices deliver some degree of stimulation; they are not inert placebos (Lund et al 2009).
The best evidence on acupuncture comes from a large meta-analysis of chronic pain trials (Vickers et al, 2012). They were able to make use of the individual patient data from the reviewed RCTs: 18,000 patients from 29 high quality trials. This has provided unequivocal evidence of acupuncture being superior even to sham. The differences were small but of a similar size to some commonly used drugs. Such small differences would require an enormous RCT to be likely to pick them up, which is why so many sham controlled trials have produced non-significant results. This is not the comparison of interest for clinicians and patients but it is useful for credibility purposes to demonstrate that acupuncture is more than a placebo. For clinical interest we take the comparison against no treatment or usual care and here the Vickers results showed a substantial effect for all three of the main conditions (low back pain, osteoarthritis and headache)
References
Bishop FL, Lewith GT. Patients' preconceptions of acupuncture: a qualitative study exploring the decisions patients make when seeking acupuncture. BMC Complement Altern Med. 2013 May 13;13:102
BMJ. What conclusions has Clinical Evidence drawn about what works, what doesn't based on randomised controlled trial evidence? http://clinicalevidence.bmj.com/x/set/static/cms/efficacy-categorisations.htmlAccessed 13 October 2015.
Greenhalgh T, Howick J and Maskrey N. Evidence based medicine: a movement in crisis? BMJ 2014;348:g3725
Freely accessible online: http://www.bmj.com/content/348/bmj.g3725
Lund I, Nasland J, Lundeberg T. Minimal acupuncture is not a valid placebo control in randomised controlled trials of acupuncture: a physiologist’s perspective. Chin Med 2009;4:1
MacPherson H, Hammerschlag R, Lewith G, Schnyer R (eds). Acupuncture Research: Strategies for Developing an Evidence Base (2007). Churchill Livingstone, Edinburgh
Paterson C, Dieppe P. Characteristic and incidental (placebo) effects in complex interventions such as acupuncture. BMJ 2005;330:1202–1205
Rawlins M. De Testimonio: on the evidence for decisions about the use of therapeutic interventions. Clin Med. 2008 Dec;8(6):579-88
Vickers AJ, Cronin AM, Maschino AC, Lewith G et al. Acupuncture for Chronic Pain: Individual Patient Data Metaanalysis.
Arch Intern Med. 2012 Sep 10:1-10.
Witt CM, Huang WJ, Lao L, Bm B. Which research is needed to support clinical decision-making on integrative medicine?- Can comparative effectiveness research close the gap? Chin J Integr Med. 2012 Oct;18(10):723-9.