Comparative Performance Data

May-June 2002


Mr. Gift is president, Systems Management Associates, Inc., Omaha, NE; Ms. Jones is project administrator, mission services, Catholic Health Association, St. Louis.

In 1998, after strategy sessions with membership from across the nation, the Catholic Health Association (CHA) launched its breakthrough initiative entitled "Living Our Promises, Acting On Faith: A National Program of Performance Improvement for the Catholic Health Ministry." The impetus for this initiative came from a growing desire for accountability in the ministry and to sustain "who we are."

Grounded in the Ethical and Religious Directives for Catholic Health Care Services, this effort focuses on helping CHA members more effectively articulate and realize their Catholic identity. The overall purpose of this work is performance improvement for the Catholic health ministry. The approach designed to help achieve this purpose required gathering facility performance data and compiling that data into a comparative report. The first round of acute-care data was collected in February 2000 and published in June 2000; the second round of data collection occurred in November 2001 and will be distributed to members in May 2002.*

* Copies of these reports are available to CHA members at no cost; visit our online Resource Catalog at www.chausa.org/RESOURCES/ or call 314-253-3458.

The Role of Comparative Data
Comparative performance data serve several useful purposes. First, the data describe the ministry's overall performance in quantifiable measures, which allows the ministry to discuss these topics in concrete terms. Second, the data inform the entire ministry about collective opportunities for improvement. Analysis reveals potential performance gaps and provides direction for moving the entire ministry. Third, potential sources of successful practices that lead to improved performance are revealed. Comparative data indicate which factors influence performance, which is vital to the benchmarking process. Fourth, comparative data offer individual facilities a wealth of information with which they can stimulate internal performance improvement.

By comparing individual facility performance with ministry-wide and peer group performance, facility leaders can incorporate performance improvement of the measures included in this dataset into the organization's overall performance improvement initiatives.

Comparative data provide powerful motivation for organizational change by identifying opportunities for improvement and assisting in setting improvement goals. With the outcomes reported in the "Living Our Promises, Acting On Faith" comparative dataset, leaders can establish improvement goals that are motivational, realistically achievable, and rooted in the demonstrated performance of others in the ministry.

A Two-Part Approach
The "Living Our Promises, Acting On Faith" report provides two types of interconnected data.

Performance Measures Data The data collection tool included 21 different measures of organizational performance. These ranged from satisfaction measures (the percent of employees indicating satisfaction with their involvement in decision making) to volume-related measures (the percent of total pastoral care visits performed in acute inpatient care settings) to selected financial measures (long-term debt to capitalization ratios).

The Table below presents a hypothetical facility's comparative data for a selected performance measure: "The percent of patients who died in the facility in the last calendar year who received palliative care." This hypothetical hospital is a 250-bed urban facility that belongs to a health care system. As such, the comparative peer groups include all participants, system participants, urban facility participants, and those participants that have between 200 and 299 beds.

Characteristics Data Characteristics used in the performance comparison are factors believed to influence performance, as gauged by the measure. The characteristics applicable to each measure were developed from discussions with practitioners active in each field.

The characteristics are affirmative statements about the presence of some attribute within the facility, such as "The facility provides education on supportive services such as palliative care and/or hospice for all physicians." Each performance measure in the data collection tool has a group of related characteristics statements.

When data were collected, study participants responded to the characteristics statements with either a "yes" or "no." The final report presents the hospital's response alongside the four peer group comparisons listed. The percent of participants in each peer group who responded "yes" appears for each characteristic.

When comparative performance measures data are supported by detailed comparative characteristics data, hospitals can use the composite information to analyze differences in organizational policies, processes, and practices that influence performance. Discerning these differences and acting on them unleashes the power of comparative data in the hospital's internal improvement process.

Incorporating Comparative Data in Existing Performance Improvement Efforts
Once a management team decides to incorporate comparative data in the organization's improvement efforts, the team needs a pathway to guide its application. Without such a pathway, the management team risks missed opportunities or misapplication of the data. To accomplish this, hospital leadership must treat improving performance in demonstrating its ministry the same way it treats improving performance in clinical and operational areas.

The comparative data report now available for acute-care facilities can assist hospital leadership with setting improvement goals and can provide direction for better performance. This effort, however, does not require an entirely new initiative. Today, virtually all hospitals have some type of performance improvement structures in place.

In a manner similar to clinical or operational improvement, leadership can create a performance improvement team. The performance improvement team, in turn, uses the hospital's designated process to address the challenge. As it works to accomplish its mission, the improvement team reports its progress using the existing reporting channels.

An Approach to Using Comparative Data
The Figure below presents an approach for using comparative data to drive internal improvement. Figure 1 presents the hypothetical comparative data to use in the algorithm.

Assessment The approach begins with the management team reviewing the data and determining the hospital's comparative standing. As the sample report shows, in palliative care, the hospital compares unfavorably with all its peer groups except for bed size: Only 35 percent of patients who died in the facility received palliative care. The important point to take away from that is that 65 percent of dying patients did not receive this care. The hospital performs below the overall study, system, and setting participants in this important service to patients.

The management team then asks if performance, as assessed by this measure, is consistent with the organization's values, mission, strategy, and operating model. Given the measure and the organization's mission and values, along with its pledge of the fidelity to the Ethical and Religious Directives for Catholic Health Care Services, the management team affirms that improvement in this measure is consistent with its core values.

The management team takes this information, adds detailed knowledge of its own operation, and discusses the appropriateness of setting an improvement goal. In this instance, given the nature of the measure, 100 percent represents a desirable long-term goal. However, the team wishes to set some short-term objectives for incremental improvement over the next three years. For the purpose of the example, the team chooses to increase this number to 60 percent by the end of the first year, 85 percent by the end of the second year, and 100 percent at the end of the third year.

Performance Improvement Initiatives Once the management team reaches agreement on an improvement goal, it incorporates the project into the organization's existing performance improvement (PI) initiative. The practices of the hospital's PI initiative call for establishing a PI committee to analyze performance and develop recommendations for actions that will accomplish the set improvement target.

The PI committee reviews the improvement goal and comparative data. It then analyzes the characteristics to determine how the hospital performs when compared with the peer groups. As the sample report demonstrates, the hospital performs consistent with the peer groups in characteristics C100, C104, C105, and C106. The hospital appears to differ from the majority of peer group participants in characteristics C98, C99, C101, C102, and C103. These differences may represent factors that contribute to the performance measure.

For example, characteristics C98, C99, and C103 relate to education and training. In each of these characteristics, the hospital differs from the majority of its peer groups. The hospital does, however, provide training about the influence of culture and ethnic background (C106), which is consistent with its peer groups. The PI committee reviews existing training practices and discusses the appropriateness of expanding education and training offerings into more areas and broadening the audiences for those offerings. The PI committee recommends an objective for the first year to increase education and training programs on palliative care.

The hospital also differs from its peer groups in the evaluation of the effectiveness of supportive services by bereaved families. Approximately half the participants in each peer group responded that they conduct such evaluations. The PI committee discusses these results and researches an appropriate mechanism with which to conduct evaluations of the effectiveness of supportive services by bereaved families. It ultimately recommends the use of structured telephone interviews as the assessment mechanism.

The third difference revealed by the comparative data lies in the inclusion of pastoral care interventions in clinical care paths. The PI committee discusses the hospital's ineffective attempts to develop and deploy clinical care paths and the reluctance of the clinical staff to engage in that developmental process. Although the PI committee recognizes this may represent a method to improve performance in this measure, the current organizational climate precludes acting effectively on this potential improvement idea. The PI committee identifies the work required to create an accepting atmosphere to act on the idea, but defers a specific objective.

Action Plans Once the PI committee reaches agreement on the actions it will take to improve performance, it develops detailed action plans. These action plans identify the specific steps to be taken to transform the action from an idea to reality. The action plans also identify the time frame within which the steps will be completed and the individuals responsible for completion.

The PI committee carries these recommendations back to the management team. Once approved, implementation of the action plans begins. Once implemented, performance is monitored to determine the level of improvement achieved.

As depicted in the Figure, after a reasonable amount of time has elapsed the committee takes new measurements and determines if performance has reached the desired level. If it hasn't, the team cycles through the approach again, seeking new insights into mechanisms for change. If it has, the team turns its attention to other areas needing improvement.

Incorporating the "Living Our Promises, Acting On Faith" comparative data by using existing resources and methods, takes advantage of the flexibility and rigor in the current improvement structures and processes and involves more staff in a broader array of areas. In addition, incorporation precludes development of a new structure to address these opportunities for improvement, thereby promoting organizational alignment and integration.

A Call to Action
One of the recurring lessons from "Living Our Promises, Acting On Faith" has been that hospitals that have worked to align their values, mission, strategy, and operating model tend to perform more effectively. And most importantly, they serve their patients better. The data resulting from the project provide a rich, informed description of the current status of the ministry. This information has begun to drive collaborative benchmarking studies at the national level through CHA and at regional levels through individual systems. In turn, these benchmarking studies are beginning to indicate successful practices that can help improve performance across the ministry.

Much more can be done, however, to foster breakthrough improvement within Catholic health care. If each study participant conducted one improvement project over the next year, the collective improvement across the ministry could be astounding. If the entire CHA membership launched one such internal improvement project using the data to help set an improvement goal, the ministry-wide impact would increase three-fold.

When does your improvement project begin?



Copyright © 2002 by the Catholic Health Association of the United States
For reprint permission, contact Betty Crosby or call (314) 253-3477.

Comparative Performance Data

Copyright © 2002 by the Catholic Health Association of the United States

For reprint permission, contact Betty Crosby or call (314) 253-3490.