Automated Decision-making in Healthcare: Making and Contesting the Rules

Partners: Tilburg University & Erasmus University Rotterdam
Type: Postdoc
Duration: 2023—2026

Automated decision-making systems (ADS) are increasingly thought to be able to positively contribute to the quality, accessibility and affordability of healthcare in the Netherlands. These three public values are nowadays increasingly under pressure due to scarcity of personnel and financial resources. As a result, solidarity among Dutch citizens, which is commonly seen as the anchor of the Dutch healthcare system, is also put under pressure. There are many ADS which hold a promise to positively contribute to these values. Examples are predictive tools, tools that aid decision-making for treatment and diagnosis, and tools that aid efficient allocation of resources. A large and increasing amount of  commercial vendors sees opportunities in the market for healthcare-related ADS-products. In order to ensure that commercial ADS can live up to their promises and actually positively contribute to public values, rulemaking plays an important role.

This project therefore studies what role rulemaking plays in the development and use of ADS in different healthcare practices. Rulemaking can be seen as a broad concept which includes dynamics around European legislation, national guidelines, and organizational protocols. More specifically, this project studies how various actors such as EU notified bodies, national regulatory agencies, professional organizations, and policymakers and professionals within healthcare organizations perceive certain rules and how they try to create, maintain, or disrupt them. It also analyses what type of rules influence the development and use of healthcare-related ADS by these actors and how they do so.

Rulemaking is a relevant concept to study since perspectives on rules and regulation around ADS in and also beyond healthcare are diverging. On the one hand, regulation is thought of as a tool to foster innovation in a context of slow progress due to limited professional trust and issues concerning financing and interoperability of systems. On the other hand, regulation can prevent the uptake of ineffective ADS with limited added value for clinical practices. Furthermore, it can stimulate alignment of ADS with public values such as safety, equity, and patient-centeredness by preventing risks. The focus of this research derives from an ecosystems perspective on ADS. It centers on networked relations between different actors at the European, national, organizational, and at the individual patient-professional level. Besides legal aspects, it also considers economic, cultural, institutional, and social aspects related to rulemaking. In doing so, the project is based on a qualitative research design. Data collection is conducted through interviewing, observations, document analysis, and literature reviewing. We thus primarily study how actors in local practices within the Dutch healthcare system give meaning to ADS in healthcare, how they give meaning to rules and regulation, and how they try to govern these technologies through rulemaking.

The first study of this project aims to conceptualize ADS in its manifold different forms, and studies how the conceptualization of ADS by various actors influences how, why, and by whom they are tested. In doing so, we distinguish among commercially developed ADS that are marketed to healthcare organizations and ADS which were initiated by healthcare professionals themselves before their commercialization. Also, we distinguish among ADS which are defined as medical devices, and ADS which are not developed for medical-, but for other purposes within healthcare organizations. Although evidence on effectiveness is considered important when it comes to health technologies, ADS need to be trained with relevant data and used in specific practices to improve their usefulness in clinical settings. The focus on testing thus allows us to further explore this relevant tension.