This year, ICTRecht visited the 'Zorg & ICT beurs', held from 14 to 16 April at the Jaarbeurs in Utrecht.* Our main objective was to gather insights into ongoing and new healthcare initiatives, with a focus on legislation and its practical application in the sector.
* The 'Zorg & ICT beurs' is an annual Dutch health tech event focusing on healthcare and information technology, held at the Jaarbeurs convention centre in Utrecht.
AI is everywhere
On the exhibition floor, AI's presence in healthcare was unmistakable. We saw all kinds of smart solutions, most of which pursue the same goal: greater efficiency in healthcare processes. The result? More time and attention for patients. The possibilities of AI are diverse and significant. However, there is still a gap between a demonstration or promising pitch and successful implementation within a healthcare organisation.
"AI readiness"
During one of the sessions we joined, we noted that AI readiness is an important topic within the healthcare sector. AI readiness means being fully prepared to implement and use AI successfully. This requires looking at the overall picture, not just a single aspect.
This includes:
-
Focusing on data availability within the healthcare organisation.
-
Investing in training healthcare professionals for AI literacy.
-
Focusing on AI compliance by complying with laws and regulations.
It's no coincidence that these are exactly the topics we deal with on a daily basis. For example, we help (healthcare) organisations with the legal and compliance aspects of their AI use. This enables us to support organisations that want to keep pace with current developments or develop innovative applications by themselves.
For this reason, we have outlined a number of key considerations below. The aim is to provide you more concrete and practical guidance when your healthcare organisation is procuring and implementing AI.
Practical considerations when procuring AI
Define the purpose of the system
Before actually procuring and implementing an AI application within your organisation, it is advisable to establish internally whether and for what purpose the organisation intends to use AI. For example, the goal might be to reduce administrative burden in a particular workflow or to support diagnostic processes.
This may sound obvious, but consider what problem AI is intended to solve. Given the requirements and obligations involved, a simpler form of automation might suffice.
Consider the impact of use
If a so-called high-risk AI system is used, it is required that the individuals to whom the system is applied are informed. In line with the previous point, consider the impact of using the system in advance. Who will be affected, both internally and externally?
Pay attention to training and input data
Is the chosen system being deployed to support diagnostics? Then it is particularly important to have insight into the training data to develop the system. For example, a system trained exclusively on hospital data from abroad, may produce different outcomes when deployed in Dutch hospitals. It is therefore worth considering whether the training data is aligned with the patient or client population. This can be verified with the relevant AI system supplier.
AI systems used for diagnostic or therapeutic purposes may also fall under the Medical Device Regulation (MDR).
Additionally, in the case of a high-risk AI system, your healthcare organisation may need to ensure that the input data is relevant and representative for the intended purpose.
Ensure the supplier provides transparency
Verify whether the AI system supplier can provide transparency about how the system works. As a healthcare organisation using an AI application, you must be able to explain, both internally to your staff and externally to patients and clients, how the system works in broad terms. The supplier is therefore obliged to provide clear, accessible information about the system.
Obtain AI literacy
A key component of the previously mentioned AI readiness is AI literacy. This is essential to ensure that the person who works with the AI system understands it in broad terms. They must be able to understand how the system works, recognise its capabilities and limitations, critically assess whether outputs are correct, and escalate issues relating to the use of the system. You can read more about this in our earlier blog: AI literacy: what is it and how do you prepare your organisation?
It is also worth noting, under the AI Act, various specific obligations apply depending on the classification of the system and the role of the healthcare provider. Furthermore, other legal issues also arise when implementing AI in healthcare. Consider requirements or obligations under the MDR, GDPR, and potentially the European Health Data Space Regulation (EHDS). The list above should therefore not be regarded as a comprehensive legal overview of requirements.
Are you planning to procure and implement AI within your healthcare organisation? Or perhaps you have already started, but are facing legal issues? We are happy to help you identify the legal and organisational obligations under the AI Act or other relevant legislation. Please feel free to contact us.
