Your own research

Introduction

ARRC provides advice on all aspects of research: choice of topic and research question, literature searches, methodology, outcome measures, funding sources and applications, ethical aspects, data analysis, writing up and publication. Some general information is presented on this website and more can be found elsewhere (see references below, for example). A specifically tailored advice service is offered to BAcC members: please contact the ARRC coordinator.

References and links

  • Research Council for Complementary Medicine: http://www.rccm.org.uk/
  • National Council for Osteopathic Research (NCOR): http://www.ncor.org.uk/
  • Kane M. "Research Made Easy In Complementary and Alternative Medicine". Churchill Livingstone 2003
  • Lewith GT, Jonas WB & Walach H (eds.) "Clinical Research Methodology for Complementary Therapies" Churchill Livingstone 2001.
  • MacPherson H, Hammerschlag R, Lewith G, Schnyer R (eds) Acupuncture research: Strategies for developing an evidence base. Churchill Livingstone, Edinburgh 2007
  • Scheid V, MacPherson H (eds). Integrating East Asian Medicine into Contemporary Healthcare. Churchill Livingstone 2011
  • Vincent C & Furnham A. "Complementary Medicine: A Research Perspective". Wiley 1997
  • NIHR Research Design Service www.nihr.ac.uk/research/Pages/ResearchDesignService.aspx - provides help on all aspects of health research design, primarily for NHS researchers
  • Research register for social care_research skills http://www.researchregister.org.uk/skills.asp – links to various sociological research methods sites

Topics

Most practitioners who use ARRC do so not as a resource for their own research, but to summarise or investigate other people's. Hence they can arm themselves with information for talks and meetings (especially when dealing with conventional medical people/bodies) aiming at gaining referrals, employment, funding and generally promoting themselves and their practices. This may be the motivation for doing your own research: to generate data of your own for promoting your practice.

Here are some examples of the possibilities:

  1. Analyse all your old records for demographic details, medical conditions, numbers of treatments, outcomes of treatments etc. Impress your audience with your conscientious record-keeping , rigorous analysis, variety of patients and wonderful results. That this can pay dividends is not in doubt (there have been examples at the ARRC Research Symposia), but take care: if you put your practice forward in this way you must ensure that your methods can withstand critical scrutiny.
  2. Many of us would probably own up to our old files being as leaky as sieves, especially if it comes to assessing treatment outcomes. Also, the prospect of spending hours of your time poring over months and years of old records is not usually one to quicken the pulse, except in exasperation. Why not start afresh then? Ensure that you collect all the information you will need by having a checklist or form to fill in, and do this with all your new patients over the next year/two years/five years/evermore. But how exactly would you go about assessing the outcomes of your treatments? It's not just a case of scoring 'patient got better: yes or no'. Designing your own measure is possible, but again, you should get advice from an expert if you are intending to publicise the results in any way.
  3. In respect of treatment outcomes most people use pre-existing measures like MYMOP (http://www.bris.ac.uk/primaryhealthcare/resources/mymop/). For several years a group of acupuncturists in the South West of England used MYMOP with all new patients. They had backup in terms of central administration and data collation/analysis, group meetings and technical assistance if needed. It may be possible to duplicate this scheme, and indeed individuals elsewhere have used MYMOP on their own, but we would encourage you to look closely at what support you could get in your own district: the formation of local research support groups may be a vital factor in the success of ventures like this.
  4. Many of you will not see any need to prove that acupuncture works, whether by your own or anyone else's research. What is of far more interest perhaps is to develop your own skills so as to become better practitioners, or to brighten up your business, work more efficiently, attract more patients. In this you would have the support of the experts: at the practitioner level research should aim to improve things rather than prove them. There are various possible aspects to this, and a range of techniques to draw upon, such as case study analysis, reflective practice and audit.  Whether or not these are defined as 'real research' is rather irrelevant here: they can or can not be depending how you use them. Also, use of outcomes measures such as MYMOP can be as much (or more) focussed on improving practice as on showing that it works.

Preparation

Although tempting to forge ahead and get on with the research when fired by a surge of initial enthusiasm, it is usually essential to prepare thoroughly first. This will involve thinking through what you want to do and how you're going to do it, and activities such as searching for and reading relevant literature, gauging the interest of colleagues, making new contacts, looking for possible funding and collaborators and considering any ethical implications. Your proposals should then be written up as the study protocol, a significant undertaking that may take you weeks or even months to complete. Small projects would tend to require less input but still require working methodically through the same steps. Inadequate preparation will cause problems when you come to carry out the project. Be sure to consider what resources you have and how to make good in any areas where you are weak. Do you have enough expertise and support personally and locally or do you need an external consultant?

Writing a summary of what you want to do on one side of paper is a good start; it immediately highlights some of the strengths and weaknesses. We have been contacted many times by practitioners who subsequently fail to translate their idea into even this amount of print. The BAcC sometimes offers grant money to help budding researchers (amongst its members) prepare for a study and produce a written protocol.

For examples of how to write a research protocol see the references below, or contact ARRC. You will also find many examples in the literature: the protocols for large projects tend now to be published as separate papers (searching Pubmed for ‘acupuncture and study protocol’ will locate many of them), indeed this may be a stipulation of their funding award.

References

White A, Ernst E. "The case for uncontrolled clinical trials: a starting point for the evidence base for CAM". Complementary Therapies in Medicine (2001); 9: 111-115
White A. "Conducting and reporting case series and audits - author guidelines for acupuncture in medicine". 2005; 23(4):181-187 [Free access at  http://www.acupunctureinmedicine.org.uk/]
White A, Park J. Protocols for clinical trials of acupuncture Acupunct Med 1999;17:1 54-58
[Free access at  http://www.acupunctureinmedicine.org.uk/]

Using the literature

Before embarking on your own research you should find out what has already been done in that topic area. Published work tends to be more reliable than unpublished, and peer-reviewed journals more so than non-, but these are not guarantees. The literature may tell you who is active in the field (and their email address), and contacting other researchers can be rewarding.

Many practitioners and patients (and certainly students) now search the internet for specific acupuncture related material. This need not be for research as such, it could equally well be to find out about contemporary Chinese (or other) approaches for particular situations - diagnostic patterns seen, points used, needling methods, novel techniques, adjunctive therapies, prognoses. Unless you have particular sites in mind already this is likely to be a lengthy and frustrating experience, even if bizarrely interesting in places. For most purposes we would recommend using one or more of the appropriate databases: see the section on research evidence, where other approaches to the literature are also discussed.

Choosing key words/terms for literature searches

  • You can easily accomplish most searches using the basic search facility of databases like Pubmed: simply enter ‘acupuncture’ and one or more terms to cover your topic of interest, separated with AND or OR, with parentheses where necessary.
  • More precise searches, specifying which fields (e.g. title, abstract, author) the terms should be in, can be done with the advanced search facilities of the database
  • Some topics are not well defined in that they require many different key words to encompass the material e.g. mental/emotional/psychological/spiritual/shen/depression/anxiety, plus others, may all be required if you're looking into aspects of mental health.
  • Some illnesses tend to be named differently in Chinese articles as in Western ones, for example 'facial paralysis' v. 'bells palsy'.

Watch out for different variations on the same word stem - depressed, depression, depressive. Many databases allow you to use an asterisk or other symbol as a 'wild card': thus 'acupunct*' should cover 'acupuncture' and 'acupuncturist' (but not 'acupressure' or 'electroacupuncture').

For the most part, the outcome of such a search will be list of references and abstracts (if they exist) that you will have to follow up yourself to get hold of the corresponding full articles: see above for information about access [LINK – Evidence/Accessing the Sources]

References

MacBeckner W & Berman BM. "Complementary therapies on the internet". Churchill Livingstone, 2003.
National Center for Complementary and Alternative Medicine (NACCAM). Finding and Evaluating Online Resources on Complementary Health Approaches. http://nccam.nih.gov/health/webresources
Health On the Net Foundation (HON) promotes and guides access to reliable and trustworthy medical  websites  http://www.hon.ch/

Methods

For hands-on instruction consider a university research methods module. There are now acupuncture focused and CAM focused research Masters degrees [LINK to section 10] and also short training courses co-organised by the International Society for Complementary Medicine Research: www.iscmr.org/about/rtm.
For detailed and comprehensive accounts of different methodologies used in healthcare research refer to the many text books on this subject.
For social science research methods: http://www.socialresearchmethods.net/kb/.
For an acupuncture/CAM slant see the reference list in the Introduction above.
The Research Council for Complementary Medicine document ‘Introduction to Research’ provides a simple account of the main methods: www.rccm.org.uk/node/15

Measuring outcomes

Any clinical trial requires that relevant changes in patients are measured in a reliable and valid manner. Case studies and series also will be stronger when appropriate outcomes measures are used, and they may be required for audit and service evaluation. There are very many different measures available (Bowling, 2001; 2004). Established measure have usually been validated, i.e. testing them with samples of people and against existing valid measures, to check that they do what they are meant to do. Validation is a good reason in favour of using established tools. It also dictates that you should not change anything in a questionnaire without permission, even just the odd word. If there is nothing close to your requirements then you may need to write your own, but this is usually difficult and time-consuming.

Access

Some measures are freely accessible, some require payment (usually small) and some are available only to institutions or for particular sorts of research.

What outcomes to measure?

Ethics

Will my research require ethical approval?

Here are some general guidelines, but if you are in doubt then contact your local research ethics committee (REC) or the National Research Ethics Service, who have a useful pamphlet on this:  www.nres.nhs.uk/applications/is-your-project-research.

  • Case studies will not need approval, assuming they are part of your usual practice
  • Clinical audit and service evaluation will not usually need ethics approval if they are part of your normal practice. However, if you are setting up a new service or giving treatments specifically for the purposes of the research, then you should get advice
  • Surveys are usually OK
  • Clinical trials will need ethical clearance

Where do I apply for ethics approval?

Funding

Many good projects fall by the wayside for want of funds. Conversely, just because a project has been externally funded does not guarantee its excellence. Researchers with a track record of successful applications, completed projects and publications are much more likely to attract new funding. Finding such a person who will endorse your proposal or enter into some kind of collaboration will boost your chances considerably. Most funding applications will require a detailed study protocol to accompany them and ethical approval (if appropriate) to follow on. The whole process of obtaining major funding is usually a lengthy one. You may need to apply at least a year before you plan to begin the research.

Possible sources of funding (UK focus)

Collaboration: research groups and networks

The RCCM website carries a list of CAM-related research networks: www.rccm.org.uk/node/204.

Of these, the most interesting and relevant for acupuncture research are:

Good Practice in Traditional Chinese Medicine (GP-TCM): www.gp-tcm.org.
This arose out of collaboration between China and the European Community for a project looking at TCM in the genomic era. The original EC funding is finished but the organisation lives on to provide networking and collaboration. The focus is strongly on herbal medicine and pharmacology but acupuncture is one of the four interest groups.

CAM Research Network (CAMRN): www.rccm.org.uk
This is the Research Council for Complementary Medicine’s CAM research network. Members receive regular email news and can access information, advice and support from other members. All the main CAM researchers and centres in the UK are members, so this is a good way to contact them.

International Society for Complementary Medicine Research (ISCMR): www.iscmr.org
Potentially useful for international collaboration and stages large, high quality annual conferences around the world. Currently constructing  a membership database and interest groups, so that networking opportunities will be enhanced. For researchers rather than therapists.

Alternative and Complementary Health Research Network (ACHRN): www.achrn.org.uk
For academic researchers and CAM therapists. Has run regular research meetings and conferences in the past.

Research Networks for acupuncturists

Supervision and university affiliation

Even if you are part of a group or network (see above) you  may still feel the need for more dedicated  support or access to particular expertise. It may be possible (though unlikely) to get supervision by private arrangement but generally you would need to be affiliated to a university or other relevant institution.

For most practitioners the only feasible route to sustained supervision is to enrol for a postgraduate degree, which can be expensive as well as time-consuming. Many of the acupuncturists taking Masters degrees have self-funded, and this is true also for some PhD’s, though a few grants are available. Most universities will be happy to take you on for postgraduate study if you meet their entry criteria and pay the fees; however, some have more of a focus and track record in acupuncture/CAM research:

Audit

There are many introductory texts on audit. For complementary/alternative medicine (CAM) refer to the website of the Research Council for Complementary Medicine (RCCM): www.rccm.org.uk, RCCM (1999) and Rees (1997).

Definition and aims

For a CAM practitioner context (Rees (1997) suggested this definition: ‘audit provides a framework for practitioner to look at whether they are providing the care they think their patients should be getting’. This definition from NICE gives an idea of what may be involved. ‘A systematic review of a practice, process or performance to establish how well it meets predetermined criteria. The procedure includes identifying problems, developing solutions, making changes to practice, and then reviewing the whole operation or service again. For example, an audit may be carried out on a specific service (such as 'stop smoking' services), to check whether it complies with laws, regulations or policies.’ Clinical audit is an important part of NHS development and CAM practitioners who are (or hope to be) providers of NHS services can expect that it will be part of the means used to assess the value of their work.
Irrespective of this, acupuncture practitioners may wish to audit aspects of their practice in order to investigate and improve what they do and the care they provide. Most practitioners do this already, informally, on a day to day basis. They will be asking themselves:

  • Did I treat that patient as well as I should have done?
  • Was there something else I should have done?
  • How can I make changes next time?

They may go no further than thinking about it. They may make notes on it, discuss with colleagues, or look up something on the internet. This is all part of reflective practice. It is encouraged by professional bodies and may form a substantial part of your CPD. In audit these processes are made more explicit and more systematic, for example ‘I now see audit as formalising common sense’ and ‘audit has helped me think more clearly about what I do’.

Quality assurance and quality improvement

Quality assurance (QA) and quality improvement (QI) are important concepts for institutional healthcare (e.g. Health Quality Improvement Partnership 2011) and closely related to audit. The aim in QI is to improve practice (and hence patient care) and demonstrate this by measuring it. QA is focused on meeting quality requirements, and hence compliance with standards rather than proactive improvement. For QA, if the evaluation of practice indicates that the required standards are already being met then no further action is needed. For QI, the aim is to improve beyond the basic standards, so there are more likely to be further cycles of action and evaluation. Both QA and QI audits are legitimate for acupuncture practitioners.

Audit is not research, nor is it outcomes evaluation

Research is about generating new knowledge. Audit is about examining one’s own work and taking actions to improve it. They may use similar methods to collect data but what is collected, from whom and what is done with it will differ. In an ideal world research would tell us about what works and what doesn’t and help us to formulate best practice. Audit would then address the question ‘are we following the agreed best practice?’ However, the concept of best practice in acupuncture is a nebulous one, and there is inadequate research to supply the raw materials for audit, as discussed further below.

Outcomes monitoring is often confused with audit, indeed most published CAM audit articles turn out not to be audit at all. Evaluating outcomes may form part of an audit and improving outcomes may be an audit goal but effecting change in outcomes is particularly difficult (further discussion below: Example 4). Hence outcomes evaluation is usually used for informing the outside world about your achievements rather than for internal quality improvement.

Who can do audit?

Anyone could audit their practice if they had the will to do so. It does not need any advanced technical or research skills: it can be done just with a pen and paper. More important are an open-minded attitude, systematic application in measuring and analysing your data, people skills and the ability to manage change. Success depends much more on a practitioner’s knowledge of their clinic, patients and therapy, and the motivation to identify and implement appropriate changes.

How to go about doing an audit: stages in the cycle

auditcycle

This cycle equates roughly with the following questions:

  • What am I trying to do?
  • Am I doing it?
  • Why am I doing it?
  • What can I do to make things better?
  • Have I made things better?

Stage 1. Defining the problem and the purpose of the audit: what can I audit?

You can audit any aspect of your practice. You may well already have an issue in mind, which is why you are considering an audit. If not, then the suggested starting point is a systematic examination of your existing practice procedures. This could cover the premises, patient referral pathways, initial contact and booking, patient demographics and presenting conditions, recording, patient management, diagnostic information, monitoring or treatment, adverse events, treatment outcomes, treatment end points, onward referral – in fact any area of practice. Within any of these areas look for things that you think could, or should, be done better. The British Acupuncture Council’s (BAcC) ‘The Standards of Practice for Acupuncture’(SPA)  would provide a useful framework for this. You should consider these points when making your choice:

  • Benefit for patients
  • Benefit for your practice/yourself
  • Potential for improvement
  • Will hold your interest
  • Something that’s under your control; hence it’s feasible for you to take steps to change it
  • It can be measured/recorded without too much trouble

Having identified a problem/topic you should state explicitly what are the aims of the audit: this will drive the rest of the stages

Stage 2. Criteria and standards

Examples

Examples of audits actually carried out by CAM practitioners, with the results, were published by the RCCM in a research pack in 1999: ‘Improving patient care in complementary medicine: using clinical audit’ Although subsequently available on their website, it may no longer be there: contact ARRC for a copy.  
Presented below are several theoretical examples, chosen across topics that may be relevant for acupuncturists, and which illustrate the application of the methods described above and issues that may arise from this.

Example 1. Practitioner punctuality

Let’s say that a practitioner is aware of the fact that (s)he tends to over-run the treatments sessions and be late for the next patient.
Specific criterion to measure: how many minutes late is the practitioner in relation to the due start time.
There is no officially designated standard for this though possibly there is prior information about it from other settings (e.g. GPs). The aim may never to be late by even one minute but this is usually impractical. Your decision on standards will be based largely on common sense and your own experience. For example, you could specify that 90% of appointments should be no more than 5 minutes late.

Good points about this audit:

  • Easy to measure/record the data on the criterion
  • The objective is largely under your control, not dependent on others

Recording your punctuality for a month should provide ample data. Let’s say that you meet the standards in 80% of cases. This falls below the 90% level, so theoretically you should take action to improve the situation. In the absence of existing standards you may decide at this point that in fact 80% is OK, but backsliding on your initial target should not be done without good reason. You may want to get answers to some subsidiary questions before deciding on any actions:

  • What do the patients feel about your time-keeping?
  • How do you compare with colleagues in this respect?
  • What is causing you to be late?

Hence there could be more data collection, e.g. a questionnaire for patients, ideas from self-reflection.
If, on the other hand, your initial results showed being at least 15 minutes late in 80% of cases then the need for action would be very clear.

Conclusions

Despite uncertainties over which criteria to adopt, what standards to set and how to go about improving things, audit is still worth engaging with. You can audit anything you want, in your own way, at your own pace. If you don’t venture into this territory, even just in an exploratory or incomplete manner, then you’ll never know what’s there. Audit is a tool that can be used in an easy but effective way to improve your understanding of your own practice and take steps to change it if you wish. It may also provide you with useful marketing material and valuable experience should you wish to pursue more research-minded activities in future.

References

British Acupuncture Council. Standards of Practice for acupuncture. British Acupuncture Council. 2009 (available from: http://www.acupuncture.org.uk/members-section/members-membership/members-membership-cpd/the-standards-of-practice-for-acupuncture.html. NB. members only access)

Dixon N and Pearce M. Guide to using quality improvement tools to drive clinical audits. Health Quality Improvement Partnership. October 2011

Huber PAJ. Clinical audit of an acupuncture/naturopathy clinic in central London 2007-20012. 14th ARRC Symposium, March 2012, London
Paterson C, Evans M, Bertschinger R, Chapman R, Norton R, Robinson J. Communication about self-care in traditional acupuncture consultations: the co-construction of individualised support and advice. Patient Educ Couns. 2012;89(3):467-75.

Rees RW. Audit or research? A personal view. Complementary Therapies in Medicine. 1997;5:233-7
Research Council for Complementary Medicine. Audit and quality assurance in complementary medicine 1999. Available from http://www.rccm.org.uk/node/15 (Accessed 7.6.2016). NB. Accessible only for members of RCCM

Wadlow G and Peringer E. Retrospective survey of patients of practitioners of traditional Chinese acupuncture in the UK. Complementary Therapies in Medicine. 1996;4:1-7