October 20, 2025
Burying the Lead: Adjusting Goals to Manage Functional Limitations of AI Tools in Healthcare
Artificial intelligence based tools are being developed for decision support in healthcare, however, they are frequently found to lack the required functionality to achieve the clinical goals for which they were built. This results in wasted time, money and resources for hospitals attempting to implement and operate such tools. To determine how functionality issues can be resolved prior to tool implementation, it is necessary to understand why such tools are being designed and then built.
Abstract
Our research focuses on clinical decision support tools with functionality issues arising from target variable invalidity. In this paper, we analyze published articles which present clinical decision support tool designs related to clinical goals. These tools use machine learning models trained on electronic health record data. We find that design decisions driven by data availability can introduce construct invalidity in clinical decision support tool designs, leading to an inability of the tool to address the clinical goal. We observe that alternative goals to the main clinical goal are used to justify continued development. We show that functional limitations of the tool related to the clinical goal can be obscured by imprecise terminology in the model’s stated functionality. Finally, we highlight the need for reconsidered approaches to dataset creation, defining success criteria, and the reporting and transparency of research outcomes as they relate to clinical goals.
To continue reading please visit: https://doi.org/10.1609/aies.v8i2.36640
This open access article was published in the Proceedings of the AAAI ACM Conference on AI, Ethics, and Society on 20 October 2025.
Kernahan, J., Bartels, R., de Reuver, M., Oberski, D., & Dobbe, R. (2025). Burying the Lead: Adjusting Goals to Manage Functional Limitations of AI Tools in Healthcare. Proceedings of the AAAI ACM Conference on AI, Ethics, and Society, 8(2), 1401–1412. https://doi.org/10.1609/aies.v8i2.36640
Keywords: Clinical decision support, AI in healthcare
Authors
Health
Dr. Richard Bartels
Assistant Professor in Data Science and Senior Data Scientist / Universitair Medisch Centrum Utrecht
University Profile
Health
Prof. Mark de Reuver
Professor in Digital Platforms / Delft University of Technology
University Profile
Health / sector leader
Prof. dr. Daniel Oberski
Professor of social and health data science / Utrecht University
University Profile
Health
Dr. ir. Roel Dobbe
Assistant professor of data-driven and algorithmic systems / Delft University of Technology
University Profile
More results /
AI insiders warn it could end humanity. Does the public agree?
By Ernesto de León • Claes de Vreese • September 17, 2026
By Natali Helberger • June 16, 2026
By José van Dijck • April 30, 2026
Loss of judicial sight? The impact of assistive AI technologies on judicial perception — beyond the question of discretion
By Iris van Domselaar • Isabella Banks • September 23, 2026
By Tynke Schepers • July 09, 2026
By Corinne Cath • May 29, 2026
By Martijn Logtenberg • November 20, 2025
By Jacqueline Kernahan • Richard Bartels • Mark de Reuver • Daniel Oberski • Roel Dobbe • October 20, 2025
The Label Paradox: Can AI transparency create more distrust?
By Natali Helberger • September 21, 2026
By Jin Wan • Theo Araujo • Natali Helberger • Claes de Vreese • September 17, 2026
By João Pedro Quintais • September 03, 2026
