BY: BILL MCCREADY
"One of the great mistakes is to judge policies and programs by their intentions rather than their results." —Milton Friedman
The Ministry Leadership Center, based in Roseville, California, initiated its three-year leadership formation program in 2005 and a separate, senior leadership forum for executives in 2011. From the very outset, both programs mandated evaluation as an integral part.
The purpose of evaluation is to create greater understanding of a program or activity by documenting resultant improvements. The Ministry Leadership Center uses theoretical models to guide data collection and analyses, measures individual formation outcomes at both the end of each session and at the end of the program, and assesses system-level formation outcomes with pilot studies. In addition, the center is designing and testing an assessment tool.
LEADERSHIP FORMATION
The leadership formation program meets in 12 sessions. The content is based on a dozen foundational concerns that the Ministry Leadership Center considers essential in maintaining the institutional identity of Catholic health care.
They fall into three domains of leadership:
Personal identity — vocation, heritage, spirituality and discernment.
Caring — care for the poor, responding to suffering, whole person care and Catholic social teaching.
Decision-making — values integration, clinical ethics, organizational ethics and collaboration with church agencies.
The Ministry Leadership Center uses both quantitative and qualitative methods to focus on the content, presentation and dynamics of each session and to assess what participants have learned and applied over the program's three-year duration. The evaluation process uses theoretical models to guide data collection and analyses, and it measures participants' individual formation outcomes.
Feedback is important — participants reflect on and articulate their own sense of having received working knowledge and skills that enhanced their abilities to lead the mission and ministry. Typically, after three years of the program, a majority of participants say they want ongoing formation, have linked their formation with leadership, have acquired theological underpinnings and feel they had formed a community of leaders.
The evaluation information:
- Guides the program into the future
- Keeps the program relevant to each new cohort of participants
- Measures the return on investment made by the supporting systems
- Promotes the program's credibility to the broader world of management science
THEOLOGY
As leaders in Catholic health care analyze complex situations and fashion strategies to address them, they consult the mission and values that are theologically grounded. However, this theological grounding often is bracketed or articulated in a sentence or two, clearly a thin description.1 The result is leaders who are not sure about underpinnings, do not know the convincing rationales for certain positions, cannot make the link or follow the logic and are not fully aware of the bigger picture.
Leaders strive for knowledge. The leadership program's feedback indicates participants want to know the theological underpinnings. They want to be able to identify and articulate the driving forces that influence the organization — including the ultimate motivations, rationales and perspectives of the faith-based heritage. Participants in the Ministry Leadership Center's program have said they appreciate the content, feel that it changed their leadership style in substantial ways and perceived personal growth as the greatest benefit from participating.
GOALS
The evaluation program has three goals: to measure the participants' abilities to articulate, reflect, and apply the insights they have acquired; to detect and measure formational changes that are linked to the program's foundational concerns; and to measure participants' abilities to articulate and execute a vision of leadership in accord with the foundational concerns.
We conduct both pre- and post-session surveys and an "after three years" assessment using qualitative and quantitative measures, since methodologists recommend a dual approach when studying outcomes that have multiple goals.
The pre- and post-session surveys help us determine what works, and we explore sessions over time for consistency and the effects of innovation. We also explore how individuals experience the program and develop their formation over time. In addition to the session evaluations, we use a variety of mechanisms to ask participants to recall significant things they have learned over the entire three years.
SENIOR LEADERSHIP FORUM
In 2011, the Ministry Leadership Center initiated the senior leadership forum in response to a call to take leadership formation more explicitly into the executive suite. The resulting program was developed for chief executives with responsibilities including regional oversight, mission integration, human resources, clinical services, strategy, public affairs/marketing and more.
The value of the forum to the organization is the creation of powerfully cohesive executive teams that bond together in interpersonal ways and allow for the sharing of both individual and organizational challenges. The process forms small executive teams whose members are deeply connected to one another and who can communicate quickly, clearly and responsively with each other.
Evaluation of the forum consists of qualitative one-hour interviews with each executive by a sociologist and a management professor. Feedback indicates that the forum engages executive teams in formation dialogue, development and shared discernment.
LEADERSHIP ALIGNMENT TOOL
All of the Ministry Leadership Center's evaluation experiences, quantitative and qualitative data have come together in the development of a leadership alignment tool called the M-LAT. In pilot testing, the M-LAT produces an "alignment index" that reflects the congruence of the attitudes and practices of health care leaders with the 12 foundation concerns.
The M-LAT can generate a report on personal growth in formation that tracks an individual's development over the course of the program. It also can generate a report tracking the aggregated development of executives and managers within a specific system.
The tool measures five leadership dimensions:
- Leadership links to heritage
- Spiritual orientation to leadership
- Personal development and vocation
- Secular orientation to leadership
- Caring and Catholic social teaching
In addition, there are two qualitative items that produce "imagery of leadership styles," such as bureaucratic, imaginative, and religious, and "imagery of how others perceive your leadership style" such as rational, inspirational and personal.
The next step for the M-LAT is to develop its use in classic 360-degree evaluations in which an employee's manager, peers and direct reports offer their feedback about the individual. The Leadership Ministry Center is actively seeking systems to partner in the effort.
Another possible use for the M-LAT is in executive recruiting. Candidates for leadership positions in Catholic health care used to be mostly "home grown" from within the faith community. Today, executives for Catholic systems come from diverse backgrounds, and although they bring many technical skills, they frequently need further development to acquire a deeply formed perspective toward the traditions and ethos upon which the health care system is based.
The M-LAT can be a tool to help recruiters screen leadership candidates by measuring an individual's progress in terms of leadership ethos. It can generate measures of such areas as vocation, spirituality, Catholic social teaching, collaboration with sponsoring organizations and the process of decision-making.
CONCLUSION
Formation is key to institutional character because leaders articulate the institutional vision. Institutional identity proceeds from the top down, and the real return on investment for formation programs is not measured by the financial investment in formation. Rather, it is measured by the continuation of the Catholic identity and character of our health care institutions. Evaluations that produce evidence-based measurements are essential to the survival of institutional identity.
BILL MCCREADY is senior fellow, evaluation and strategic planning, the Ministry Leadership Center, Roseville, California.
NOTE
- Clifford Geertz, The Interpretation of Cultures: Selected Essays (New York: Basic Books, 1973) 3-30.