by A recent clinical trial conducted at Vanderbilt University Medical Center has revealed that an artificial intelligence (AI) tool accurately predicts the risk of blood clots in pediatric patients. However, the trial did not show any significant difference in outcomes when compared to a control group. The researchers reported their findings in the journal JAMA Network Open.
One possible reason for the lack of difference in outcomes is that the treating physicians were hesitant to follow the recommendation to begin blood-thinning therapy in high-risk patients, which occurred in only 26% of cases. The physicians expressed concern that the therapy could lead to major bleeding, although no such complications were observed during the study.
Despite the unexpected results, the clinical trial, known as CLOT (Children’s Likelihood of Thrombosis), demonstrated the safety and effectiveness of the AI tool in a healthcare setting. It also provided valuable insights into how to successfully integrate this tool into clinical practice.
Dr. Shannon Walker, the first author of the paper and an assistant professor of Pathology, Microbiology and Immunology and Pediatrics at Vanderbilt University Medical Center, stated, “There is going to be more and more AI in healthcare. Having a system established where we can assess these (models) will allow us to provide safer and more effective care to our patients.”
The study highlighted the importance of pragmatic patient-level, randomized, controlled trials in assessing the safety and effectiveness of AI tools. According to Daniel Byrne, the director of AI Research at the Advanced Vanderbilt Artificial Intelligence Laboratory (AVAIL) and the Department of Biostatistics, “This study demonstrates that a pragmatic patient-level, randomized, controlled trial is the most ethical and effective way to assess whether AI tools are safe and effective.”
Although blood clots in pediatric patients are rare, they can significantly increase hospital stays and the risk of complications and death after discharge. To identify high-risk patients, the researchers analyzed the electronic medical records of over 110,000 admissions to the Monroe Carell Jr. Children’s Hospital at Vanderbilt.
The researchers identified 11 factors associated with the risk of blood clots, including lab values, diagnoses, and whether the patient had undergone surgery or had cardiology or infectious disease consults. Using this information, they developed a predictive model that calculated a risk score daily for every pediatric hospital admission, allowing medical teams to focus on patients with the highest likelihood of developing blood clots.
The 15-month trial included 17,000 hospitalizations, with patients randomly assigned to either a study group or a control group. The study group received risk scores and recommendations to initiate anti-thrombolytic therapy, while the control group did not receive risk scores but still received blood thinners based on their treating physicians’ judgment.
No bleeding complications were observed in any of the patients who received blood thinners according to the study recommendations. However, at the end of the trial, there was no significant difference in the rate of blood clots between the study and control groups. Further analysis revealed that the recommendation to begin blood thinners in high-risk patients was followed only 25.8% of the time.
Dr. Walker emphasized that without conducting the trial, it would have been impossible to identify potential reasons for the lack of success with the intervention. She also emphasized that it was not a failure of the AI model itself but possibly due to the reluctance of physicians to accept the recommendations.
The implementation of AI in clinical practice has faced resistance, with concerns over time-consuming implementation and evaluation of the algorithms and their impact on already limited clinical resources. However, the CLOT trial showed that rapid results can be obtained without burdening the treatment team’s time or resources by automatically enrolling patients in a clinical trial using existing electronic medical records.
While the trial demonstrated the feasibility of assessing the value of predictive AI models in healthcare, further research is still needed. Dr. Walker and her colleagues at Vanderbilt are planning another trial to better understand why providers are reluctant to accept the recommendations and how to overcome this hesitancy.
Dr. Byrne emphasized the importance of conducting rigorous science to ensure that AI tools are safe and improve outcomes before claiming their effectiveness, noting that other hospitals and AI vendors may skip this step. Dr. Walker added that the infrastructure established in this trial will allow for larger study populations to determine the safety and efficacy of AI interventions and to identify the patients who may benefit the most from them.
*Note:
1. Source: Coherent Market Insights, Public sources, Desk research
2. We have leveraged AI tools to mine information and compile it
