Steven J. Squires
Director Mission and Ethics
Trinity Health
Novi, Mich.
[email protected]

Is it working? That is the question Trinity Health sought to answer when it began a critical examination of competencies for its acute care ethics committee and consultation service members in the fall of 2008. With significant resources invested in strengthening the role and effectiveness of ethics committees and consultants, the organization listened when committee leaders expressed the desire to create a new standard for excellence — something to ensure that physicians navigating a challenging ethical terrain would continue to call on them for assistance. After considerable study and much thought, I began to develop PEERS, the "Programmatic Ethics Evaluation Reciprocity System," as one answer to the question. Still in development, PEERS is a theoretical approach for evaluating ethics programs and their effectiveness in a more objective manner than is typically the case.

The first step in developing PEERS involved working with the committee to survey the literature, other health care systems, and existing — as well as developing — standards within Trinity Health. Next, we developed new competency standards for ethics committees and consultation services, and then created a method for evaluation and assessment.

The direction we chose for the latter was atypical. Usually, ethics program evaluation and assessment do not receive much attention either in practice or in the literature. When it is discussed, the discussion is sparse with regard to its necessity as well as the methodology to be employed. Generally, ethics program evaluation and assessment are overlooked.

The "Core Competencies" report from the American Society for Bioethics and Humanities (ASBH) provides a pertinent example. The beginning of the very short section on competency evaluation asserts that a complete discussion of evaluation goals and techniques is beyond the report's scope.¹ Evidently, endorsing a specific method is also outside the bounds of the report.² While the report does address the need for evaluating ethics consultations, namely, consultants themselves, the consultation process, and consultation outcomes as well as barriers to evaluation (e.g., the absence of stipulated consultation goals), it does not even hint at a potential methodology.³

Development of PEERS is designed to fill the gap.

When considering ethics committee evaluation and assessment, common themes emerge from the literature. One source elaborates on the distinction between formative and summative evaluation. Formative evaluation focuses on processes to improve ethics program effectiveness, whereas summative evaluation focuses on program goals or products in view of determining whether the program should improve, continue, be cut back, or cease.⁴ It seems intuitively, coherently, and empirically justifiable that ethics programs should have both process and product evaluations, even if one disagrees with the approach of this particular source.

Why? Process and product are both critical to ethics program functioning. Take, for example, the questions that Bernard Lo regards as important in an ethics committee evaluation:

1. Are patients and surrogates able to access committees when they like?
2. Are committee recommendations, including the reasons for them, given to those who request assistance?
3. Are recommendations consistent with legal and ethical frameworks?
4. Are disagreeing or inquiring parties satisfied with the ethics review process and product (i.e. recommendations)?⁵

Answering these questions requires both process and product (goal) evaluation.⁶ Other essays either implicitly⁷ or explicitly⁸ make this assertion as well.

While evaluating process and product are themselves important, the interconnectedness or relation between ethics programs' process and product is even more crucial. Describing an ethics committee's effect, or outcome, without a process effectively disconnects the committee from the attainment of that outcome or goal.⁹ At minimum, it is much more difficult for a committee to accurately "take credit" for a particular goal without defining its involvement in the process of achieving the goal.

For instance, consider an ethics committee that established the goal of reducing by 10% in a six-month time period the number of inpatient admissions of patients over 65-years-of-age who did not have an advance directive. At the end of the six months, reviews reveal an 11% reduction. Without a process and ethics intervention, the committee has a substantial burden-of-proof to claim that the actions of its members were part of the solution. Put differently, achieving or exceeding this particular goal without an identified process or actual intervention is not indicative of ethics program effectiveness (formative) or comparative worth (summative). Any number of other initiatives and interventions could have achieved this result. There is even the possibility, however improbable, that this improvement occurred on its own.

Ethics committees should be intentional in their planning process when it comes to linking specific goals with particular processes designed to achieve them. John Mitchell suggests the use of a planning instrument so that committee members consider goals, a program (process) to realize each of the goals, accountability for interventions, timetables for interventions, expected outcomes, and methods for assessing successful interventions right from the start.¹⁰ By establishing goals and processes, ethics committees make evaluating an intervention's effectiveness and worth easier, as evaluating both is important.¹¹

My study of the current approach to program evaluation and assessment — clinical and organizational — indicated, without doubt, that it is flawed. The lack of any guidance or standards enables members to self-regulate and many of the evaluators have a vested interest in the success of their programs because their program's success could ultimately impact their employment. Organizational bias and conflicts-of-interest abound as well. Those who evaluate or assess committees may not know they have a conflict-of-interest or bias and may dismiss their presence precisely because they exist. Externally evaluated clinical or organizational ethics programs are, therefore, simply more credible — to individual professionals, entire organizations, and to both patients and the surrounding community.

The PEERS Approach
Because credibility builds trust, it was a top priority in designing the PEERS approach. It combines both internal and external evaluators and can be implemented in a variety of ways.

In one implementation, called "the low objectivity and reciprocal accountability model," ethics program leaders constitute a PEERS group. Members meet periodically and discuss their respective program goals and objectives. PEERS group members make three agreements:

1. Each member sets clear, specific goals and objectives for his or her own ethics program;
2. Members provide feedback about respective goals;
3. The group reviews the progress of programs before constructing new goals.

This method does not leave much discernment to the individual members of the group. Evaluation and assessment are largely the purview of the internal ethics leader. Not much changes, other than the amount of open, external feedback in constructing goals.

In a "moderate objectivity and reciprocal accountability model," ethics program leaders discuss minimal standards for programs during PEERS meetings. All programs represented by PEERS group members adhere to the methods used to measure the standards and evaluation according to the standards. Other members of the group are accountable as representatives of the public interest, which may take the form of an internal scorecard or dashboard. PEERS group members are "coaches" motivating program leaders and providing specific suggestions for improving program performance.

A model that promotes "high objectivity and reciprocal accountability" is one that may be the most uncomfortable for program participants and leaders whose performance is under scrutiny. PEERS group participants randomly select names of organizations whose ethics programs are to be evaluated. The person selecting the name is the evaluator. Participants may wish to agree on four conditions:

1. Avoid the appearance of mutual bias. For instance, Saint Joseph Mercy Oakland (SJMO) hospital should not evaluate Beaumont Hospital if Beaumont is evaluating SJMO.
2. Increase objectivity and accountability. Each participant could randomly select two names, in accordance with the first condition, to ensure program evaluation teams of two.
3. Persons from the organization being evaluated do not bias the PEERS evaluation, other than organizing discussions and visits.
4. Senior leadership should fully support the process. The PEERS evaluation team presents results to senior leaders of the organization being evaluated.

None of these conditions precludes the possibility of self-evaluated programs still being helpful and useful components of overall evaluations. All of the suggestions and conditions are dependent upon ethics programs establishing specific, measurable goals and objectives from the start. This is imperative. As stated earlier, organizational improvements not linked to measurable objectives, goals and interventions cannot be shown to be associated with the ethics program.

Succeeding with PEERS
The PEERS model assumes a high level of trust and mutuality among group members. Those who carry out evaluations and assessments must take care to respect the privacy of information to safeguard persons, as described in regulations such as the Health Information and Portability and Accountability Act (HIPAA). A good start for a PEERS group may be the low objectivity and accountability model, with the eventual adoption of a medium or high model. There is no reason to suggest one exclusive model for one PEERS group. Different PEERS groups should experiment with different models. Finally, none of this precludes partnerships between health care organizations and academia, especially institutions of higher education with programs in health care ethics. In fact, health care organizations using PEERS could make arrangements with universities and colleges to, for instance, allow for internship opportunities for students, in exchange for program evaluation.

No organization should fear the ramifications of external evaluation. It can only make them stronger. External regulation and evaluation are far from foreign in health care. Acute and long-term care organizations are highly regulated, in almost every sphere of operations,¹² and external regulation is normative for clinicians, as well as executive leadership. Those in health care expect regulation, whether or not it is the best way to ensure good standards. In light of this, is it not logical for our health care colleagues to expect objective standards and evaluation for ethics programs as well? Opening ethics programs to external standards and review would lend them more credibility in the eyes of our colleagues and makes good sense.

So, what should we say to those in ethics who protest on the grounds that objective standards for ethics programs do not exist? They are actually partially correct. The Joint Commission's requirement that organizations have an ethics program does not detail components and goals,¹³ nor does the American Society for Bioethics and Humanities (ASBH) either mandate standards for ethics programs, including committees and consultation services, or certify them. Still, their objections are not ultimately persuasive. In fact, several sets of standards for ethics programs exist from a variety of sources which are congruent with one another (e.g., the "Core Competencies" by ASBH and "Deciding to Forego Life-Sustaining Treatment" by the President's Commission). Other literature offers additional standards, as do newer approaches to ethics programs such as the Veteran Administration's IntegratedEthics program and "Next Generation" ethics.¹⁴ There are, in fact, many suggested standards — and they are sound. PEERS, we think, when put into

practice, will be a powerful tool that can help ethics committees meet those standards.

###

This article is a condensation of two articles that first appeared in Ethics-in-Formation (May 2009 and September 2009), a publication of the Medical Ethics Resource Network of Michigan. Used with permission.

References

American Society for Bioethics and Humanities. Core Competencies for Health Care Ethics Consultation: The Report of the American Society for Bioethics and Humanities. Glenview, IL: American Society for Bioethics and Humanities, 1998.

Bernt, Francis, Peter Clark, Josita Starrs, and Patricia Talone. "Ethics Committees in Catholic Hospitals." Health Progress 87, no. 2 (2006): 18-25.

Fletcher, John, and Diane Hoffmann. "Ethics Committees: Time to Experiment with Standards." Annals of Internal Medicine 120, no. 4 (1994): 335-38.

Heeley, Gerry. "A System's Transition to Next Generation Model of Ethics." Health Care Ethics USA 15, no. 4 (2007): 2-4.

Hoffmann, Diane. "Evaluating Ethics Committees: A View from the Outside." The Milbank Quarterly 71, no. 4 (1993): 677-701.

Joint Commission for Accreditation of Healthcare Organizations. Accreditation Manual for Hospitals. Oakbrook Terrace, IL: The Joint Commission for Accreditation of Healthcare Organizations, 1992.

"Ethics Issues." The Joint Commission for Accreditation of Healthcare Organizations, http://www.jcrinc.com/Chapter-1-Defining-Main-Components/Developing-and-Implementing-an-Ethical-Infras/Ethics-Framework/

Lo, Bernard. "Answers and Questions About Ethics Consultations." Journal of the American Medical Association 290, no. 9 (2003): 1208-10.

"Behind Closed Doors: Promises and Pitfalls of Ethics Committees." New England Journal of Medicine 317, no. 6 (1987): 46-50.

"Behind Closed Doors: Promises and Pitfalls of Ethics Committees." In Bioethics: An Introduction to the History, Methods, and Practice, edited by Nancy Jecker, Albert Jonsen and Robert Pearlman, 236-44. Sudbury, MA: Jones and Bartlett Publishers, Inc., 2007.

Mitchell, John. "New Challenges for Ethics Committees." Health Progress 81, no. 6 (2000): 22-25.

Murphy, Kevin. "A 'Next Generation' Ethics Committee." Health Progress 87, no. 2 (2006).

Ross, Judith, John Glaser, Dorothy Rasinski-Gregory, Joan Gibson, and Corrine Bayley. Health Care Ethics Committees: The Next Generation. U.S.A.: American Hospital Publishing, 1993.

Slosar, John Paul. March 16 2009. Communication about Ascension Health's Suggested Ethics Committee Plans, Goals, and Assessment.

Squires, Steven. "Trinity Health's Next Generation Model of Ethics." Ethics-in-Formation 13, no. 2 (2009): 4-5.

Copyright © 2010 CHA. Permission granted to CHA-member organizations and Saint Louis University to copy and distribute for educational purposes. For reprint permission, contact Betty Crosby or call (314) 253-3490.