A type of artificial intelligence application that has been widely adopted in recent years and that will surely proliferate is agentic AI, or AI applications that take actions across multiple virtual platforms with little human involvement.
Becket Gremmels, CommonSpirit Health system vice president, theology and ethics, says the lack of human oversight poses myriad ethical concerns. He sees great potential benefit from agentic AI, including in healthcare, but says organizations that use it should proceed cautiously and should put in place protections to prevent negative fallout.
In May, Gremmels presented "Agentic AI and Health Equity," the most recent in CHA's We Are Called webinar series. CHA launched the We Are Called initiative in 2021 to formally recognize the profound effect racism has on the health and well-being of individuals and communities. We Are Called challenges the Catholic health ministry to address systemic causes of health disparities among underserved and vulnerable populations.
During the webinar, Gremmels said there are profound potential health equity concerns involved with the use of agentic AI. A top concern, he said, is that people seem to think "the technology is morally neutral to some extent, (and yet) it takes on the moral character of those who use it and design it and create it and build it."
Given the stakes of agentic AI use in healthcare — stakes that could be life or death in some cases — organizations must use thorough ethical discernment in determining how to use AI in a just way, Gremmels said.
"Start yesterday"
When it comes to ethical discernment around AI usage, healthcare organizations should "start yesterday," Gremmels told webinar participants, explaining that agentic AI already is proliferating at most healthcare organizations.
He cited several examples of agentic AI applications, including one that can process thousands of insurance denials per week, one that handles patient scheduling and other administrative tasks, one that completes medical coding, and one that coordinates prescription refills.
He noted that at CommonSpirit alone, there are hundreds of AI applications in use and many more in the queue for rollout. CommonSpirit has 138 hospitals in 24 states.
Technology with great promise
Gremmels said that healthcare providers already are using agentic AI to accomplish great feats, and the technology holds much promise for the future.
The technology can enable tasks to be accomplished faster and more efficiently. From a health equity perspective, Gremmels said, this means that some AI applications could help improve healthcare access and make it faster and easier for people, including the marginalized, to get care.
He said another potential health equity benefit is that agentic AI could improve chronic disease management, easing care coordination. AI also could speed the diagnosis of illnesses, which could mean fewer treatment delays.
AI — particularly through the use of large datasets and complex data analysis — could greatly improve population health work.
“I am not trying to fearmonger. But, we need to understand: These agentic models have great potential, but we must move cautiously." — Becket Gremmels
Gremmels said healthcare providers already are seeing success using AI in these and many other ways to promote health equity. Such advancements are sure to continue in the years ahead.
Cause for concern
And yet, this new frontier for technology poses many potential pitfalls, Gremmels said, noting that Pope Leo XIV covered many areas of concern in the Magnifica Humanitas encyclical that the Vatican released May 25.
Gremmels said some of the practical concerns are that people could — intentionally or unintentionally — use agentic AI in a way that causes great harm to others. Also, human oversight of AI is only as good as the attention span and capabilities of the humans filling the oversight role. Also, AI can make mistakes. Additionally, no one yet knows the implications of novelties associated with AI functioning, such as how the AI agents will use virtual environments designed for them to work freely only with one another, with no human involvement.
There are many additional threats. Gremmels said AI could essentially replace human decision-making, leading to a deskilling. It also is becoming increasingly difficult to detect when AI is mimicking real people, and so it can be used ever more effectively for deception.
Also, as Pope Leo explains in the encyclical, AI has the veneer of neutrality. But in reality, the biases of the people programming and using the AI are dictating the applications' "morals" and "values."
When it comes to health equity, such threats around AI use can put the marginalized at even greater risk than they already are.
Broad oversight
Gremmels said organizations are responsible for getting ahead of such threats by using ethical discernment to develop both broad oversight and specific guidelines for AI tools to promote good outcomes and ward off bad outcomes.
At the very outset, before AI is even deployed, organizations' leadership must involve ethicists in undertaking an open-eyed analysis of the goals, moral boundaries, dangers and potential good involved. All people involved with using the AI applications should be formed to understand the ethical landscape.
Gremmels noted that there are insights from the section of the pope's encyclical that covers AI use in war that are applicable to healthcare. For instance, an organization should not give AI decision-making power in matters that have direct, life-or-death consequences for humans. And, there should be clear chain-of-command documentation of AI decision-making, and clear lines of human responsibility for all decisions made. There also should be an effective way for the humans who are impacted by the actions of AI agents to challenge what the AI is doing.
With such precautions in mind, Gremmels said, technology experts should program AI agents with very specific goals and with instructions that mimic moral boundaries. The leadership should be explicit about what would trigger a hard stop of each AI agent and how that would be done. And the technology team should test all of the AI agents' work in a simulated environment before allowing them to access real-world data.
He said everyone involved should understand that AI agents can destroy, corrupt, delete or release any data they have access to.
"I am not trying to fearmonger," Gremmels said. "But, we need to understand: These agentic models have great potential, but we must move cautiously."