 |
| About Bioline | All Journals | Testimonials | Membership | News |
|
||||||
|
||||||
Indian Journal of Surgery, Vol. 68, No. 5, September-October, 2006, pp. 251-256 Review Article The Cochrane database of systematic reviews: A treasure house for evidence-based surgery Saha Saumitra Department of Surgery, North Bengal Clinic, Siliguri, District - Darjeeling - 734 403, West Bengal Code Number: is06071 Abstract Surgeons, like other clinicians, are obliged to keep up-to-date with the rapidly expanding knowledge. The information explosion particularly in the electronic media, methodological problems associated with clinical trials in surgery and failure to appraise critically the existing knowledge has made application of high quality evidence in surgical decision-making difficult. This article examines the functioning and the role of Cochrane collaboration and in particular, the systematic reviews undertaken by it, in the assimilation and dissemination of robust evidence for use in evidence-based surgery.Keywords: Cochrane Collaboration, evidence-based surgery, systematic reviews Widespread computerization and adoption of information technology have allowed clinicians instant access to the vast and rapidly expanding medical information. But gathering information relevant to one′s practice from the print journals or various electronic archives in the Internet has been likened to "trying to get a sip of water from a fire-hose and you don′t even know the quality of the water".[1] Although the practice of evidence-based surgery (E-BS) depends on the critical appraisal of original studies, it is unrealistic to expect a busy surgeon to keep track of the huge volume of publications that appear in the literature. For common therapeutic decisions, an initial search of the most popular bibliographic database - MEDLINE - using appropriate keywords or medical subject headings (MeSH) might yield far more articles or abstracts than a busy clinician can ever pursue. Thus, it is usually necessary to limit the search by applying certain filters such as English language publication, studies within the last five years or randomized controlled trials (RCTs). But even a very specific search with the phrase "stapled hemorrhoidectomy" via the PubMed clinical queries filtering the search for RCTs generated 21 citations.[2] The number of abstracts or articles to peruse gets manageable but it would still be difficult to evaluate certain outcomes of this relatively new technique e.g., long-term functional outcome due to disparate study population and variable follow-up. Individual RCTs may have one or more of the several known biases of selection, detection, attrition and performance.[3] There is also a great risk of missing results of trials published in the non-English literature. Thus, even after limiting one′s search to RCTs, extrapolation of clinically relevant data for use in one′s practice and decision-making is not easy. One way to deal with this problem is to assess integrated results from multiple studies. The Cochrane Collaboration has taken on itself the onerous task of combining systematic identification of all pertinent studies in all languages with a quantitative synthesis of the information where applicable in all aspects of healthcare, many of which are of importance to the surgeons.[4] History and modus operandi of Cochrane Collaboration[4] The first Cochrane centre was established in 1993 in Oxford, UK. This nonprofit, independent organization was named after Archie Cochrane, a British epidemiologist who introduced the concept of controlled clinical trials in the 1960s in the UK amidst stiff opposition from the clinicians. Not only did he revolutionize the way we now advocate treatments but also highlighted the economic benefits that can be derived from practicing E-BS.[5] The first Cochrane Colloquium was held in 1994 which was attended by members from several countries and the Cochrane Collaboration was established as an international body. Currently there are 12 centers spread across the world and run by voluntary groups dedicated to the propagation of evidence-based medicine. China is the only Asian country to have a Cochrane center. There are 51 Cochrane review groups dedicated to a particular topic, of which several are of relevance to surgeons; e.g., Breast cancer, upper GI and pancreatic diseases, colorectal cancer, prostatic diseases and urologic cancers, peripheral vascular disease, injuries, wounds etc. The Cochrane Collaboration aims to provide healthcare professionals with selected, critiqued reviews of the available evidence for specified clinical problems. Using specially developed software tools, systematic reviews and meta-analysis if possible are undertaken and the findings are then stored in a central database. Most studies will have two reviewers and if there is a disagreement on any point, a third reviewer arbitrates. The final reviews are then screened for electronic publication in what is called The Cochrane Library - available on both CD-ROM and the Internet. The guiding philosophy of the Cochrane editors is that a review will only be published by virtue of its quality and not its findings. There are several databases in the Cochrane Library, all of which are unique resources [Table - 1]. [Table - 2] enumerates some of the recently published systematic reviews of importance to surgeons. The topics are wide-ranging and of interest to both the general and the sub-specialist surgeon. Ongoing reviews are listed as "protocols". [Table - 3] lists some of the protocols relevant to the surgeons. Review abstracts are freely accessible on the Internet but full text articles are available on annual subscription basis from John Wiley and Sons Ltd., UK. The entire Cochrane review index is available from http://www.cochrane.org/reviews/en/topics. Why are systematic reviews necessary? The traditional wide-ranging review articles (or overviews) with an assortment of facts from textbooks and publications of mixed quality supplemented with single-institution experiences do not adequately address specific therapeutic dilemmas and tend to present personal preferences. Journals with high impact factor now require review articles to address two key questions to avoid bias. 1) Is there a clearly focused question? 2) Were appropriate criteria used systematically to select studies for inclusion?[6] The answers derived from these questions are useful in several ways for the practice of E-BS. Quality of evidence In the hierarchy of scientific evidence, the highest level (Level 1) is the systematic review with or without meta-analysis of homogeneous prospective RCTs, followed by individual prospective RCTs with narrow confidence intervals.[7] It gives a more precise estimate of treatment effects and more statistical power to detect small but significant effects that may not be obvious in small RCTs. This is well exemplified in the trials of antibiotic prophylaxis in severe acute pancreatitis, where the rationale is to prevent infection of pancreatic necrosis. Routine prophylaxis can lead to an increase in fungal and other lethal resistant organism infection. The evidence is unclear as different end-points have shown benefit for antibiotics in the different RCTs and the trials have been too small to give a definite answer about mortality. But a Cochrane review concluded that antibiotic prophylaxis might reduce mortality in patients with necrosis.[8] It therefore follows that since the rationale for antibiotic treatment is to prevent infected necrosis, it is inappropriate to use antibiotics in patients without CT evidence of hypoperfusion. Level 2 evidence includes systematic reviews of cohort studies, individual cohort studies and outcomes research. Level 3 comprises systematic reviews of case-control studies. Case series and poor quality cohort and case control studies provide Level 4 evidence. The lowest ranking evidence (Level 5) consists of expert opinion without explicit appraisal. Even as recent as 10 years back the majority of publications even in journals with high impact factor were based on Level 4 evidence.[9] Preparation of guidelines It is impossible for the individual surgeon to identify and incorporate into practice the large quantity of rapidly changing knowledge and new technology required to sustain best practices. Professional bodies have now established methods for developing scientifically sound guidelines and technology assessments for common problems and newer operations. The Cochrane systematic reviews and other Cochrane resources are referred to in such exercises.[10] Research and design of trials Even if RCTs are not thought to exist for some unanswered surgical questions, a systematic review may still prove beneficial by confirming the lack of studies and therefore evidence related to the problem in question and highlighting how necessary research might be done. It therefore facilitates the design of the most appropriate trial.[11] Let us take the example of the Cochrane review using meta-analysis on laparoscopic techniques versus open techniques for inguinal hernia repair involving 7161 patients.[12] Stringent inclusion criteria were employed and the open technique group was controlled as best as possible by classifying them into ′mesh′, ′non-mesh′ and ′mixed-open′ group. Both totally extra-peritoneal approach (TEP) and trans-abdominal pre-peritoneal approach (TAPP) procedures were than compared against all three open groups. This allowed comparison of homogeneous groups. The authors concluded that the recurrence rate is lower for laparoscopic mesh repairs versus open non-mesh repair, but is the same for laparoscopic mesh repair versus open mesh repair. Laparoscopic operations were longer with a potential for visceral and vascular injuries in the TAPP procedure but allowed earlier return to normal activity. The conclusions are acknowledged by the majority of surgeons. However, the review did not address the issues of recurrent and bilateral hernias and this prompted researchers to design suitable RCTs.[13] The National Institute of Clinical Excellence in the UK have recommended in 2001, that laparoscopic repair be reserved for recurrent and bilateral inguinal hernia, taking into account that outcome analysis is not only about evaluating recurrence rates but also the learning curve, the potential seriousness of complications and cost of laparoscopy.[14],[15],[16] A difficult but possible way to objectively assess the impact of the learning curve in laparoscopic procedures is to randomize patients to operators in different phases of learning. It remains to be seen whether increasing number of better designed trials with standardized outcome measures will influence the conclusions of a future systematic review on hernia surgery - the commonest operation worldwide. In the Cochrane review on laparoscopic versus open surgery for suspected appendicitis of 54 RCTs (45 in adults), there were several therapeutic outcomes (e.g., negative appendicectomy rate, wound infection, pain, hospital stay) which favored the laparoscopic approach.[17] The rate of intra-abdominal infections was higher in patients undergoing laparoscopic appendicectomy. It could be argued that more RCTs involving children are needed to be included for a future systematic review, for homogeneity in the study population. Strong heterogeneity was found for most outcomes except for wound infection and intra-abdominal abscesses. The authors also sounded a note of caution about the quality of some of the data analyzed and recommended more "blinded" studies (e.g., using identical dressings whilst evaluating postoperative pain and analgesic requirement). Because there is a belief that laparoscopic appendicectomy is less likely to cause adhesions than open operations, longer follow-up studies were recommended to assess the relative incidence of intestinal obstruction due to adhesions. The meta-analysis of preoperative radiotherapy for esophageal cancer on 1147 patients including all published and unpublished RCTs did not show any clear evidence of improved survival. The authors concluded that survival benefit if any, is modest, but this would require a meta-analysis of around 2000 patients (having 90% power and 5% significance level) to detect a 3-4% improvement in absolute survival;[18] an example where outcome analysis can be influenced by studies of inadequate power. This is particularly true for cancer where the principal outcome of survival may not be hugely different between different modes of treatment. Public awareness and informed consent Cochrane review abstracts are accompanied by a plain language summary (bereft of any scientific jargon and an explanation of the background leading to the study) for the layperson and since this is freely accessible, it helps in the dissemination of medical information to the patient and his relatives. In the Western world and even in India, surgeons are seeing more and more patients who feel empowered with evidence from websites which are often superficial, sometimes misleading and occasionally harmful.[19] It is in our interest that such patients′ attention is directed to authentic electronic repositories such as the Cochrane systematic reviews. After all, a well-informed public is the best ally for the surgical profession. Clinical governance and audit The results of systematic reviews can be incorporated into individual practice and may form the basis of an institutional protocol. The Cochrane Library has been used to monitor the usage pattern, decision-making and appropriateness of management by physicians in a busy medical ward, in a novel study in the late 90s whereby computers loaded with Cochrane and other electronic databases (called "electronic carts") were provided in a ward.[20] Teaching institutions, medical libraries and individual clinicians in the UK, Australia, Ireland, South Africa and Scandinavia can access full-text Cochrane reviews for free where a national contract for the Cochrane library exists. In the absence of robust evidence from RCTs, meticulously collected efficacy and safety data on certain procedures forms the basis of institutional or national surgical audits. Medico-legal implications We are being increasingly threatened by malpractice suits. The legal profession is using concepts of evidence-based practice in the management of common problems to support their clients′ claims. In circumstances in which there is an untoward event, failure to have used the best evidence leaves the surgeon open to legal liability.[21] It is therefore essential to embrace the principles and practice of E-BS as far as practicable. Cochrane reviews versus other systematic reviews There have been good systematic reviews conducted outside the Cochrane Collaboration but they were generally restricted to the English language or at the most including one non-English language (resulting in a language bias). Systematic reviews are time-consuming and laborious. Because of the way papers are indexed in electronic databases, some papers will not be found even when all sensible keywords with appropriate Boolean operators or MeSH terms are used. Thus to be complete, thorough manual searches of selected journals, conference proceedings and abstracts are often required. Many studies with unexpected or negative findings are presented at conferences but not published. The workload to collate such vast information will be so large that it is possible only if adequate funding and a dedicated workforce are available. Multilingual and multinational volunteers conversant with the principles of critical appraisal are required to search for relevant literature in both English and non-English languages. There are about 14000 people working for the Cochrane collaboration from nearly 100 countries - a major advantage over an individual or single hospital-initiated systematic review. The extensive network enables establishment of contact with investigators of unpublished trials and helps in updating the review in the event of further accrual of substantive clinical studies in the evidence pool. The review on preoperative hair removal to reduce surgical site infection even required the investigators to contact manufacturers of depilatory creams.[22] [Table - 4] lists some of the reviews which have been recently updated. An updated review does not necessarily differ in its conclusion from the initial review. Application of rigorous scientific methodology in culling suitable trials and publications means that a high quality review is obtained which is as free from bias as possible. A trial with a positive result is more likely to get published. Systematic reviews of such publications are likely to produce a falsely positive treatment effect (publication bias), when in reality if all trials (published, unpublished and in all languages) are combined no effect would be demonstrated. It is in these methodological contexts, that discordance between findings of systematic reviews addressing the same question should be assessed. There has been divergent conclusion of the reviews on the efficacy of albumin supplementation in the critically ill patient. One study concluded this to be clearly beneficial,[23] while another stated this to be harmful.[24] The Cochrane review, however, found no obvious efficacy in human albumin solution for resuscitation and volume expansion in critically ill patients.[25] It should be emphasized that systematic reviews are not ′the last word′ in E-BS and certainly the Cochrane Collaboration do not claim their conclusions to be infallible, as even systematic reviews with meta-analysis have limitations. A valid meta-analysis requires a series of high quality trials of adequate size, follow-up and quantitative and qualitative homogeneity for all groups that are compared in the analysis. The Cochrane reviews therefore, refrain from overstating their results and making definitive recommendation all the time. The review on follow-up on nonmetastatic colorectal cancer found only five trials that met methodological quality. Meta-analysis demonstrated a survival benefit at five years for patients undergoing intensive follow-up. No difference in the number of recurrences was seen. But because of significant study heterogeneity (wide variation of follow-up programs) no clear recommendation was given.[26]Summary The Cochrane Collaboration has contributed immensely to the practice of E-BS by reviewing and assimilating Level 1 evidence and helping in its orderly dissemination to healthcare professionals and patients. Its methodologies for nonrandomized studies are also an important tool and provide the best available evidence in the absence of Level 1 evidence. It is an important educational tool both for the consultant and the trainee surgeon involved in research, writing clinical papers / dissertation or appearing in postgraduate examinations. It is an invaluable resource for the hospital libraries, multidisciplinary teams empowered in decision-making, and surgical associations involved in professional leadership and preparing clinical guidelines. For the practicing surgeons with internet access but no access to quality print journals, the Cochrane library ensures prompt access to all the systematic review abstracts and is worth ′bookmarking′. The more discerning surgeon or researcher would prefer access to the full text systematic reviews which would currently require a subscription in India. The Association of Surgeons of India along with other professional bodies and the Ministry of Health, Govt. of India should explore the feasibility of arranging a national contract with the Cochrane Collaboration which would make full text access free to clinicians. The compendium of the Cochrane database is truly a "library on the desktop". Efforts are already under way to build electronic libraries with "core content" resources such as NLH guidelines Finder specialist library, Cochrane reviews and clinical evidence reviews, all of which have been subject to quality assessment.[27] The supplementation of the frontline print journals with judicious use of the Cochrane database as a rapid and veritable source of evidence ensures that the present-day surgeon does not lose his way on the information superhighway.References
Copyright 2006 - Indian Journal of Surgery The following images related to this document are available:Photo images[is06071t1.jpg] [is06071t4.jpg] [is06071t2.jpg] [is06071t3.jpg] |
| |||||||||
![[Table - 4]](/showimage?is/photo/is06071t4.jpg){kind=link}