BY: CONNIE EVASHWICK, SC.D. AND THOMAS RUNDALL, PH.D.
Now is the perfect time for health care organizations to embrace evidence-based management. The recession has caused health care organizations, like organizations in other fields, to put various programs on hold; re-think competitive strategies for the future; reduce workforce through attrition, retirements or layoffs; and, above all, watch how resources are used. At the same time, the health care reform debate has called into question how hospitals and other organizations can change their care structures and processes to provide more efficient and effective patient care.
Denise Rousseau of the Tepper School of Business at Carnegie Mellon University defines evidence-based management as "a paradigm for making decisions that integrates the best available research evidence with decision-maker expertise and client/customer preferences to guide practice toward more desirable results."1
Incorporating research evidence in decision making helps managers avoid such well known traps as basing important operational and strategic decisions on outdated historical practice, current fears or fads or personal preferences. Also, by assessing the implications of findings from the best available studies, managers can more objectively evaluate the pros and cons of a proposed change in the organization and estimate the likely effects.
The evidence-based approach to management consists of an important set of decision-making concepts and tools that can help health care managers make the changes necessary for future success. Use of evidence-based management requires health care managers to implement new decision-making processes, create organizational structures and allocate resources to support the identification and use of applied management research evidence in decision making.
Physicians have learned over time to use evidence-based medicine. Initially, many rebelled at the idea of using clinical protocols or guidelines for practice. It took time and proof of improved outcomes for the medical professional to embrace the value of evidence-based approaches to patient care.
Now it's time for health care executives to catch up.
THE SIX STEPS OF EVIDENCE-BASED HEALTH CARE MANAGEMENT2
- Frame the question
- Acquire research information
- Assess the validity, quality, applic-ability and actionability of the evidence
- Present the evidence to those who must act on it
- Apply the evidence to the decision
- Evaluate the results
Because the process carries a cost in terms of time, training and sometimes a purchase price for specific data, evidence-based management techniques often are applied only to decisions that deal with important operational and strategic issues in which investment in the process is likely to reap substantial returns to the organization and its patients. To use this process successfully, an organization must commit to providing the resources required. The staff needs time devoted to collecting the information required and becoming proficient at it. All who make decisions must commit to incorporating evidence in the decision-making process rather than short-cutting needed deliberations.
Anthony R. Kovner, David J. Fine and Richard D'Aquila have provided a number of examples of the application of evidence-based management to health care in their new book Evidence-Based Management in Health Care. (See book review, page 88.) Further guidance on the evidence-based decision-making process can be found on the Internet at the Informed Decisions Toolbox: http://toolbox.berkeley.edu.
TYPES OF EVIDENCE
Sometimes the best available evidence for a decision derives from an in-house study to help decision makers understand a particular problem and the likely impact of various solutions. For example, an organization could conduct its own pilot study to assess the effectiveness of assorted types of reminders in different settings or patient populations before it implemented a patient reminder and recall program to improve immunization rates across multiple physician practices.
But local decisions can spring from studies conducted in other organizations, too. A great deal of research on issues of concern to health system managers already exists, and the findings are readily accessible online. This external research ranges from case studies to multi-institutional investigations; for example, the Cochrane Database of Systematic Reviews provides a summary of the findings from 47 studies of the effectiveness of patient reminder and recall systems in improving immunization rates. (http://www.mrw.interscience.wiley.com/cochrane/clsysrev/articles/CD003941/frame.html). Decision makers could use these findings to project the possible effects of implementing such a reminder and recall program in a health system that does not currently have one.
The evidence required for the process comes in several forms:
Qualitative evidence.
Compile qualitative evidence about the feasibility and likely effects of a particular decision by interviewing or surveying patients, employees and other key informants; discussing the proposed decision with executives who have been in a similar situation; and drawing on information provided by consultants and vendors. The process for gathering information should be systematic, unbiased and transparent in order for others to accept the validity of the evidence.
Internal administrative and patient care data.
A great deal of data is generated within a health care organization, but much of it may not ever have been used to aid decision making. Once the organization has framed its questions to explore, internal sources of data are a good starting point. The American Economic Recovery and Reinvestment Act of 2009, the federal stimulus package signed into law last year, has provided millions of dollars for health care organizations to use for electronic medical recordkeeping technology. If designed correctly, such systems can provide information useful for management purposes as well as for clinical care. Management should seize this opportunity — both to secure funds and to determine what internal data from electronic medical records could be applied to management issues.
Pilot studies and demonstration trials.
These can gather information not only to guide new programs but to help decision makers assess specific effects of a change before an organization-wide rollout.
Externally dictated standards.
In health care, these most commonly come up in conjunction with financial performance — with its external regulations — or operations management, as in supply chain processes. Health systems that have developed dash boards have created their own "standards" external to an individual institutional member. Several good sources of data, such as the Centers for Disease Control and Prevention, are available to help managers with tough decisions that bridge clinical and non-clinical programming.
Scientific evidence.
Over the past 40 years, health services and public health research have become highly sophisticated fields of inquiry. An abundance of scientific data is available to guide health care executives in making decisions, if only they knew how to find and interpret those data. When many studies have been conducted on a particular management issue, it is possible to produce a systematic review of the studies, enabling managers to see the overall effects of an organizational change and to understand for what types of organizations the strongest and weakest effects have been observed. Ideally, at least some managers in health organizations will become skilled in performing such research syntheses. Fortunately, websites are proliferating that offer a systematic review of the findings from multiple studies of particular management issues. See for example the Cochrane Collaboration website at http://www3.interscience.wiley.com/homepages/106568753/QuickRef.pdf; the website of the Community Guide to Preventive Services: http://www.thecommunityguide.org/uses/programs_services.html; and the website of McMaster University's Program in Policy Decision Making at http://www.researchtopolicy.ca/Search/Reviews.aspx.
Unique local circumstances, the experience and judgment of the decision makers and the opportunities and constraints present in a given environment also must be taken into account. But, incorporating evidence in the management decision-making process is useful and at times of overwhelming importance to making an effective change in the organization.
EVALUATING EVIDENCE
Any of the above types of information may form the basis for a better management decision than one made in the absence of concrete evidence. But the evidence must be assessed before it is accepted and used. Institutions can learn to use the four A's to judge evidence: it must be accurate, applicable, available and actionable.
Accurate.
To know if data are accurate, staff must probe for the source of the data and know enough about health care research to assess its validity. High-quality research clearly reports the research design, study context, sample definition and size and data collection methods. The presentation of the data should provide a complete and balanced viewpoint and should be from a credible source, free of potential conflicts of interest. For example, annual salary surveys are published in a variety of sources for various health care jobs. Many state hospital and professional associations keep track of local market basket adjustments to national salary data. Given these reputable sources, one would assume that the methods are valid and the results accurate.
Applicable.
The data must apply as closely as possible to the decision at hand. At times — but not always — research on a similar management question completed on a different population or in a different setting can reasonably be used to help make a decision. For example, can statewide averages of nursing salary increases be applied to the largest metropolitan area in the state?
Available.
Though there are many sources of data, such factors as cost, time frame and geographic unit may interfere with obtaining or using the information when decision makers need it. A state professional association or union may survey nurses every other year, for example, but will two-year-old data gathered before the recession still be useful? Is anything more current available for the immediate employment area?
Actionable.
Finally, can the evidence lead to action? If it cannot, then it's not worth expending the resources to acquire it, whether they are staff time or dollars. For example, does a salary report identify the implications of increasing nursing salaries by a given percent? Does the report indicate whether taking the action will attract new employees, decrease the costs of temporary staff and increase patient satisfaction scores? If not, the information is too incomplete.
EXAMPLES
Two examples illustrate how evidence can be used to inform management decisions. One approach uses data the organization has collected internally. The other applies data from external sources.
Catholic Health Care West, which operates 41 hospitals in California, Arizona and Nevada, developed the nation's first standardized Community Need Index in partnership with the Evanston, Ill.-based information products company Solucient, LLC. In developing this tool, they applied to public health the same level of scientific rigor that they insist upon in the practice of medicine. The Community Need Index identifies the severity of health disparity for every ZIP code in the United States based on specific barriers to health care access. In doing so, CHW has demonstrated the link between community need, access to care and preventable hospitalization for conditions that, if effectively diagnosed and managed, should be treatable in an outpatient setting.
To augment the program selection process, the corporate office used internal data to provide each facility with three years of trended data for overall community benefit expense, uncompensated care and utilization of health care services for ambulatory care sensitive conditions. CHW asked each facility to identify a health issue in a neighborhood with disproportionate unmet health-related need, as indicated by the Community Need Index, community needs assessments and hospital-specific data related to ambulatory care sensitive conditions. Each facility was then asked to plan and develop, or enhance an existing, preventive health program, using evidence-based intervention strategies to address the identified disproportionate unmet health-related need, focusing primarily on the uninsured and populations covered by Medicaid, Medicare/Medicaid or other government-funded insurance programs for the indigent.
This three-year initiative includes an ongoing evaluation process to ensure strategies are effective for their patient populations and the goals and objectives of the programs are met. The initiative has also been elevated to a system-wide metric goal that includes compensation incentives for successful accomplishment of the established goals. In short, from community needs assessment to program selection to staff incentives, all decisions pertaining to community benefit are based on compiling and evaluating current evidence.
Patient handoffs, that is, transitions, during hospitalization from one provider to another are critical points in patient care. Poor quality handoffs have been identified as a cause of adverse events for hospitalized patients. In 2006, the Joint Commission issued a national Patient Safety Goal that requires providers to adopt a standardized approach for handoff communications. In response to these issues, the Society of Hospital Medicine convened a task force comprised of six hospitalists with backgrounds in education, patient safety, health communication, evidence-based medicine and handoffs. The task force performed a systematic review of the literature, using PubMed to search for articles using "handoff" and related keywords. After applying study inclusion and exclusion criteria, 10 articles were reviewed and key features of patient handoffs associated with patient outcomes were identified. This literature review was supplemented by a review of expert consensus papers. The task force commissioned the papers from a panel of four content experts selected for their work on handoffs in the fields of nursing, information technology, human factors engineering and hospital medicine.
The task force used the evidence from the literature review and the experts' reviews to draft recommendations that were then presented to an audience of approximately 300 hospitalists and other patient care professionals for feedback. Further review was carried out with the Society of Hospital Medicine board.
In its final recommendations, the Society of Hospital Medicine advised organizations to institute a formally recognized handoff plan at the end of a shift or change in service and adopted 12 specific points regarding the way in which the handoff is performed, the nature of the verbal exchange between providers and the content of the exchange. Any health care organization that decides to revise its patient handoff procedures will have a well researched template to guide its new system. This will likely make the change easier to accept and more targeted at areas likely to result in the greatest improvement of current processes.
PROS, CONS AND OUTCOMES
What evidence demonstrates the impact of using an evidence-based approach to health care management? In truth, the field is still young.
Benefits.
Both examples above show how using an evidence-based approach helps management pin decisions about new activities to objective data rather than to the personal preferences of a hospital staff member, making it easier to convince others — from the board to the immediate staff involved — to accept the new initiatives. Similarly, with evaluation pegged to expectations set by an objective external standard, it will be easier for everyone involved to measure success or recognize shortfalls that could be corrected. In short, the long-term outcomes make it worth the upfront effort to find data and then build a program accordingly.
Considerations.
At the same time, organizations should embark on evidence-based management with realistic expectations. Moving toward an evidence-based approach is likely to incur costs both to start and to operate on an ongoing basis. The approach may require extra time to train staff and extra time for staff to spend searching for evidence, not to mention the possible expense of purchasing data. Moreover, there is always a cost to change: Emotions, attitudes and the errors of start-up all may come into play. Starting with one key project rather than trying to change the entire organization at once minimizes the negatives inherent in change.
Outcomes.
Organizations that have tried evidence-based management to date have done so more often because they believe in the concept rather than because they have seen vast savings or improved patient care in relatively short periods of time. Those organizations that have implemented evidence-based decision making by changing organizational culture and establishing new processes have seen it embraced over time because it provides a systematic and unbiased approach to problem solving. The earlier an organization explores the possibilities of an evidence-based approach, the sooner it will be able to assess the benefits and start building a culture that embraces this approach.
While dollars are scarce and change is in the air, why not try evidence-based management now?
NOTES
- Denise Rousseau, "Is There Such a Thing as 'Evidence-Based Management'?" Academy of Management Review 31 (2006): 256-69.
- Vineet M. Arora et al., "Hospitalist Handoffs: A Systematic Review and Task Force Recommendations," Journal of Hospital Medicine 4, no. 7 (September 2009): 433-40.
CONNIE EVASHWICK is professor, departments of health management and policy, and community health, School of Public Health, Saint Louis University, St. Louis, and THOMAS RUNDALL is executive associate dean and professor of health policy management, School of Public Health, University of California, Berkeley, Calif.
Copyright © 2010 by the Catholic Health Association of the United States
For reprint permission, contact Betty Crosby or call (314) 253-3477.