Core Topics in General & Emergency Surgery: Companion to Specialist Surgical Practice (6 page)
Read Core Topics in General & Emergency Surgery: Companion to Specialist Surgical Practice Online
Authors: Simon Paterson-Brown MBBS MPhil MS FRCS
Clinical guidelines are systematically developed statements to assist practitioner and patient decisions about appropriate healthcare for specific clinical circumstances.
38
EBM is increasingly advocated in healthcare, and evidence-based guidelines are being developed in many areas of primary healthcare such as asthma,
39
stable angina
40
and vascular disease.
41
Over 2000 guidelines or protocols have been developed from audit programmes in the UK alone. An observational study in general practice has also shown that recommendations that are evidence based are more widely adopted than those that are not.
42
The UK Department of Health has also endorsed the policy of using evidence-based guidelines.
43
Guidelines may have a number of different purposes:
to provide an answer to a specific clinical question using evidence-based methods;
to aid a clinician in decision-making;
to standardise aspects of care throughout the country, providing improved equality of access to services and enabling easier comparisons to be made for audit and professional assessment (a reduction in medical practice variation);
to help to make the most cost-effective use of limited resources;
to facilitate education of patients and healthcare professionals.
For clinical policies to be evidence based and clinically useful, there must be a balance between the strengths and limitations of relevant research and the practical realities of the healthcare and clinical settings.
44
There are, however, commonly expressed concerns about the use of guidelines:
There is a worry that the evidence used may be spurious or not relevant, especially in areas where there is a paucity of published evidence.
Guidelines may not be applicable to every patient and are therefore only useful in treating diseases and not patients.
Clinicians may feel that they take away their autonomy in decision-making.
A standardised clinical approach may risk suffocating any clinical flair and innovation.
There may be geographic or demographic limitations to the applicability of guidelines; for instance, a policy developed for use in a city district may not be transferable to a rural area.
The effectiveness of a guideline depends on three areas, as identified by Grimshaw and Russell:
45
How and where the guidelines are produced (development strategy) – at a local, regional or national level or by a group internal or external to the area.
How the guidelines have been disseminated, e.g. a specific education package, group work, publication in a journal or a mailed leaflet.
How the guidelines are implemented (put into use).
In the UK, there are a number of bodies that produce guidelines and summaries of evidence-based advice.
NICE is a special health authority formed on 1 April 1999 by the UK government. The board comprises executive and non-executive members. It is designed to work with the NHS in appraising healthcare interventions and offering guidance on the best treatment methods for patients. It assesses all the evidence on the clinical benefit of an intervention, including quality of life, mortality and cost-effectiveness. It will then decide, using this information if the intervention should be recommended to the NHS.
It produces guidance in three main areas:
public health;
health technologies – including the newly developed Medical Technologies Evaluation Programme (MTEP) and Diagnostic Assessment Programme (DAP);
clinical practice.
Its role was further expanded in 2010 following the NHS White Paper, Equity and Excellence – Liberating the NHS and was tasked with developing 150 quality standards in key areas in order to improve patient outcomes. It is now linked in with NHS Evidence and manages the online search engine that allows easy access to an extensive evidence base and examples of best practice.
The SIGN was formed in 1993. Its objective is to improve the effectiveness and efficiency of clinical care for patients in Scotland by developing, publishing and disseminating guidelines that identify and promote good clinical practice. SIGN is a network of clinicians from all the medical specialities, nurses, other professionals allied to medicine, managers, social services and researchers. Patients and carers are also represented on the council. Since 2005, SIGN has been part of NHS Quality Improvement Scotland.
EPOC is a subgroup of the Cochrane Collaboration that reviews and summarises research about the use of guidelines.
Guidelines also need to be critically appraised and a framework has been developed for this
46
that uses 37 questions to appraise three different areas of a clinical guideline:
rigour of development;
content and context;
application.
ICPs are known by a number of names, including integrated care plans, collaborative care plans, critical care pathways and clinical algorithms. ICPs are a development of clinical practice guidelines and have emerged over recent years as a strategy for delivering consistent high-quality care for a range of diagnostic groups or procedures. They are usually multidisciplinary, patient-focused pathways of care that provide a framework for the management of a clinical condition or procedure and are based upon best available evidence.
The advantage of ICPs over most conventional guidelines is that they provide a complete package of protocols relating to the likely events for all healthcare personnel involved with the patient during a single episode of care. By covering each possible contingency with advice based upon best evidence, they provide a means of both identifying and implementing optimum practice.
There is a traditional hierarchy of evidence, which lists the primary studies in order of perceived scientific merit. This allows one to give an appropriate level of significance to each type of study and is useful when weighing up the evidence in order to make a clinical decision. One version of the hierarchy is given in
Box 1.1
.
21
It must be remembered, however, that this is only a rough guide and that one needs to assess each study on its own merits. Although a meta-analysis comes above an RCT in the hierarchy, a good-quality RCT is far better than a poorly performed meta-analysis. Similarly, a seriously flawed RCT may not merit the same degree of importance as a well-designed cohort study. Checklists have been published that may assist in assessing the methodological quality of each type of study.
21
Box 1.1
Hierarchy of evidence
Systematic reviews and meta-analyses
Randomised controlled trials with definitive results (the results are clinically significant)
Randomised controlled trials with non-definitive results (the results have a point estimate that suggests a clinically significant effect)
Cohort studies
Case–control studies
Cross-sectional studies
Case reports
From Greenhalgh T. How to read a paper: the basics of evidence based medicine. London: BMJ Publications, 1997; Vol. xvii, p. 196. With permission from the BMJ Publishing Group Ltd.
Similar checklists are available for systematic reviews.
21
,
47
,
48
As already discussed, the preparation of a systematic review is a complex process involving a number of steps, each of which is open to bias and inaccuracies that can distort the results. Such lists can be used as a guide when preparing a review as well as in assessing one. One checklist used to assess the validity of a review does so by identifying potential sources of bias in each step (
Table 1.2
).
49
Table 1.2
Checklist for assessing sources of bias and methods of protecting against bias
| Source | Check |
| Problem formulation | Is the question clearly focused? |
| Study identification | Is the search for relevant studies thorough? |
| Study selection | Are the inclusion criteria appropriate? |
| Appraisal of the studies | Is the validity of the studies included adequately assessed? |
| Data collection | Is missing information obtained from investigators? |
| Data synthesis | How sensitive are the results to changes in the way the review is done? |
| Interpretation of results | Do the conclusions flow from the evidence that is reviewed? Are recommendations linked to the strength of the evidence? |
| Are judgments about preferences (values) explicit? | |
| If there is ‘no evidence of effect’, is care taken not to interpret this as ‘evidence of no effect’? | |
| Are subgroup analyses interpreted cautiously? |
From Oxman A. Checklists for review articles. Br Med J 1994; 309:648–51. With permission from the BMJ Publishing Group Ltd.
It is hoped that the results of a systematic review will be precise, valid and statistically powerful in order to provide the highest quality information on which to base clinical decisions or to produce clinical guidelines. The strength of the evidence provided by a study also needs to be assessed before making any clinical recommendations. A grading system is required to specify the levels of evidence, and several have previously been reported (e.g. those of the Antithrombotic Therapy Consensus Conference
50
or that shown in
Table 1.3
).
Table 1.3
Agency for Health Care Policy and Research grading system for evidence and recommendations
From Hadorn DC, Baker D, Hodges JS et al. Rating the quality of evidence for clinical practice guidelines. J Clin Epidemiol 1996 49:749–54. With permission from Elsevier.
The grading of evidence and recommendations within textbooks, clinical guidelines or ICPs should allow users easily to identify those elements of evidence that may be subject to interpretation or modification in the light of new published data or local information. It should identify those aspects of recommendations that are less securely based upon evidence and therefore may appropriately be modified in the light of patient preferences or local circumstances. This raises different issues to the grading of evidence for critical appraisal and for systematic reviews.
In 1979, the Canadian Task Force on the Periodic Health Examination was one of the first groups to propose grading the strength of recommendations.
51
Since then there have been several published systems for rating the quality of evidence, although most were not designed specifically to be translated into guideline development. The Agency for Health Care Policy and Research has published such a system, although this body considered that its level of classification may be too complex to allow clinical practice guideline development.
52
Nevertheless, the Agency advocated evidence-linked guideline development, requiring the explicit linkage of recommendations to the quality of the supporting evidence. The Centre for Evidence-based Medicine has developed a more comprehensive grading system, which incorporates dimensions such as prognosis, diagnosis and economic analysis.
53
These systems are complex; for textbooks, care pathways and guidelines, such grading systems need to be clear and easily understood by the relevant audience as well as taking into account all the different forms of evidence that may be appropriate to such documents.
There are three main factors that need to be taken into account in determining the strength of evidence:
the type and quality of the reported study;
the robustness of the findings;
the applicability of the study to the population or subgroup to which the guidelines are directed.
Meta-analyses, systematic reviews and RCTs are generally considered to be the highest quality evidence that is available. However, in some situations these may not be appropriate or feasible. Recommendations may depend upon evidence from other kinds of study, such as observational studies of epidemiology or natural history, or synthesised evidence, such as decision analyses and cost-effectiveness modelling.
For each type of evidence, there are sets of criteria as to the methodological quality, and descriptions of techniques for critical appraisal are widely available.
21
Inevitably, there is some degree of subjectivity in determining whether particular flaws or a lack of suitable information invalidates an individual study.
The strength of evidence from a published study would depend not only upon the type and quality of a particular study but also upon the magnitude of any differences and the homogeneity of results. High-quality research may report findings with wide confidence intervals, conflicting results or contradictory findings for different outcome measures or patient subgroups. Conversely, sensitivity analysis within a cost-effectiveness or decision analysis may indicate that uncertainty regarding the exact value of a particular parameter does not detract from the strength of the conclusion.
Strong evidence in a set of guidelines must be wholly applicable to the situation in which the guidelines are to be used. For example, a finding from high-quality research based upon a hospital population may provide good evidence for guidelines intended for a similar setting but a lower quality of evidence for guidelines intended for primary care.
The following is a simple pragmatic grading system for the strength of a statement of evidence, which will be used to grade the evidence in this book (and the other volumes in the Companion series). Details of the definitions are given in
Table 1.4
. For practical purposes, only the following three grades are required, which are analogous to the levels of proof required in a court of law:
Table 1.4
Grading of evidence and recommendations
‘Beyond reasonable doubt’.
Analogous to the burden of proof required in a criminal court case and may be thought of as corresponding to the usual standard of ‘proof’ within the medical literature (i.e.
P
< 0.05).
‘On the balance of probabilities’.
In many cases, a high-quality review of literature may fail to reach firm conclusions because of conflicting evidence or inconclusive results, trials of poor methodological quality or the lack of evidence in the population to which the guidelines apply. Where such strong evidence does not exist, it may still be possible to make a statement as to the ‘best’ treatment on the ‘balance of probabilities’. This is analogous to the decision in a civil court where all the available evidence will be weighed up and a verdict will depend upon the ‘balance of probabilities’.
‘Unproven’.
Where the above levels of proof do not exist.
All evidence-based guidelines require regular review because of the constant stream of new information that becomes available. In some areas, there is more rapid development and the emergence of new evidence; in these instances, relevant reference will be made to ongoing trials or systematic reviews in progress.
Although recommendations should be based upon the evidence presented, it is necessary to grade the strength of recommendation separately from the evidence. For example, the lack of evidence regarding an expensive new technology may lead to a strong recommendation that it should only be undertaken as part of an adequately regulated clinical trial. Conversely, strong evidence for the effectiveness of a treatment may not lead to a strong recommendation for use if the magnitude of the benefit is small and the treatment very costly.
The following grades of recommendations are suggested and details of the definition are given in
Table 1.4
:
A strong recommendation, which should be followed.
A recommendation using evidence of effectiveness, but where there may be other factors to take into account in the decision-making process.
A recommendation where evidence as to the most effective practice is not adequate, but there may be reasons for making the recommendations in order to minimise cost or reduce the chance of error through a locally agreed protocol.