Using artificial intelligence (AI) to predict the probability of missed appointments in a busy clinic—followed up with personal outreach to at-risk patients—improved “show rates” for patients, especially those who were Black, according to a new study led by researchers at Case Western Reserve University and MetroHealth.
Although patients receive appointment reminders through text messages and other means, researchers found additional outreach proved beneficial because of wide disparities in access to technology throughout Cleveland communities.
The AI model can potentially be used to improve access to care in other clinics and health systems, the researchers concluded in the study to be published in the Journal of General Internal Medicine.
“We used the AI technology to figure out who needed additional support or an alternative low-tech outreach solution,” said the study’s lead author Yasir Tarabichi, an associate professor at the Case Western Reserve School of Medicine and MetroHealth’s medical director of the Virtual Care Enterprise and director of Clinical Research Informatics. “When we use automatic tools for reminders in a community with a huge digital divide, we are making assumptions those reminders are reaching everyone. That is not true. Minorities have less access to reliable internet and are less likely to use patient portals to engage with care. It is a complex problem we are trying to solve.”
The study’s co-authors are: David Kaelber, a professor at the School of Medicine and chief medical informatics officer at MetroHealth; MetroHealth’s Jessica Higginbotham, and Nicholas Riley; and Brook Watts, formerly of MetroHealth and now with the University of Michigan.
The study was supported by the Clinical and Translational Science Collaborative of Cleveland, a collaborative among Case Western Reserve and its affiliated hospital systems, including MetroHealth.
Using patient data, researchers built an AI model in Epic, MetroHealth’s electronic health-record system, to predict the chances a patient would miss an appointment. The study targeted adult internal medicine patients who had a probable no-show rate of at least 15%.
Between January and September 2022, a random selection of those patients received phone calls from MetroHealth schedulers. If patients indicated barriers that may prevent them from making their appointments, schedulers offered resources such as transportation or telehealth options.
Black patients who received phone calls had a 36% reduction in no-show rates, compared to those who did not receive a call. Although race and ethnicity were not included as variables in the design build, MetroHealth has a sizable Black patient population.
The reason for not calling everyone who had scheduled appointments was due to limited human resources and the need to ensure equity, the researchers said.
“The fear was implementing a model that gave more opportunities to patients who were not in dire need of better access, which would widen disparity gaps,” said Tarabichi, who explained that AI allowed them to create a fair model to avoid that. “We want to provide an outreach mechanism that is fair and provides a level of equity. People at a higher risk for not showing up are the ones we are helping the most. Minority patients were more likely to pick up the phone when called, and we met them where they were.”
The study is significant because racial and ethnic disparities in missed appointments are common in safety-net health systems like MetroHealth. These disparities in access risk harming the patients that safety-net systems are designed to serve, the researchers said.
“This is an amazing example of using technology, in this case machine learning, to have our staff work smarter and not harder, to help close no-show disparities in our patient population to improve care,” Kaelber said.
Due to its success, MetroHealth’s internal medicine clinic continued using the Epic AI model to call potential no-show patients. And the model can be tailored to use in other MetroHealth clinics—and possibly other health systems—to reach at-risk patients.
For more information, contact Bill Lubinger at william.lubinger@case.edu.