How do users’ perceptions of AI influence adoption into value-based radiology departments?
Internship research by Taylor Arient
AI in value-based healthcare
Artificial intelligence (AI) developments in healthcare present time-saving benefits for physicians, by increasing productivity and improving efficiency in executing specific tasks. At the same time, for hospitals looking to adopt AI solutions, buying decisions increasingly rely on considerations of their concrete added value to the health system. AI algorithms have proven that they can automate some of the tedious tasks in clinical practice. But do radiologists feel that adopting AI is really worth the investment?
This question is essential within value-based healthcare systems. The concepts of value-based healthcare have first been described by Prof. Porter in his famous article in the New England Journal of Medicine. In this delivery model, healthcare providers and hospitals are compensated based on patient outcomes. Hospitals and healthcare systems are also expected to follow value-based purchasing regarding any new technical innovation purchases. In doing so, research shows that hospitals evaluate clinical outcomes, person and community engagement, safety, and efficiency and cost reduction.
When it comes to AI adoption, how do these evaluations occur, and what role do the perceptions of AI ultimately play in deciding to implement specific solutions? As part of her master’s study at the VU University (Vrije Universiteit) Amsterdam, Taylor Arient focused on this topic during a six-month research internship at Aidence.
Posing a research question
The study aimed to contribute to the development of a value proposition for AI solutions by evaluating stakeholders’ perceptions of the adoption of AI in radiology. The contextual background was value-based healthcare systems.
The research was guided by this overarching question:
“How do radiologists’ and implementation leaders’ perceptions on AI influence the adoption of AI into radiology departments within value-based health systems?”
The concepts in brief
To understand the process of adopting new technology or innovation, Taylor built on the diffusion of information (DOI) theory, developed by E. M. Rogers in 1983. DOI identifies five stages in the innovation-decision process: 1) knowledge, 2) persuasion, 3) decision, 4) implementation, and 5) confirmation. The focus of this research was placed on the persuasion stage. This is where most stakeholders in radiology are currently at; they already know the types of AI solutions available and of the opportunities to use AI in radiology, but have not yet decided to fully adopt or reject the innovation.
The DOI theory states that potential adopters’ perceptions of an innovation’s characteristics are more important than the objectively measured parts of the innovation. Taylor, therefore, looked at five perceived characteristics of an innovation:
Relative advantage: the perceived advantage of using an innovation versus having none in place, or the perceived advantage of the considered innovation over another, if applicable
Compatibility: the perceived ease of integration into existing workflow, software, and health systems
Complexity: the perceived ease of use, operationalized to the time needed to learn how to use the solution
Trialability: the perceived ability to test the solution in practice before adopting
Observability: the degree to which the benefits of the innovation can be observed by the user or adopter of the innovation
Getting the insights
To answer the research question, Taylor interviewed 12 healthcare professionals from eight hospitals located in the Netherlands, Sweden, Denmark, and the UK. The participant pool was made up of six cardiothoracic radiologists, a PACS manager, a clinical physicist, a medical technology engineer, two members of the strategy and legal department, and an AI implementation specialist. All of the participants have either used AI in radiology (routine practice or lung cancer screening) or have been involved in the implementation process.
The innovation discussed was Veye Lung Nodules, Aidence’s lung nodule management solution.
Three key findings
The results of the research study brought Taylor to reflect on these main findings, in the context of her original research question:
1. It all depends on the reason for using AI
The first conclusion was that the way that a hospital was using AI in radiology influenced how participants viewed the adoption process. The participants who used AI as part of a lung cancer screening programme viewed improved reporting time and accuracy as more important. Users of AI in routine clinical practice spoke more about adopting an AI solution that could integrate seamlessly with their workflow first and foremost. The use case also influenced the adoption of AI because if it was being used as part of a lung cancer screening programme participants vocalized a greater need for AI.
2. Workflow integration is essential
How compatible an AI solution is with a hospital’s workflow and current policies is one of the main factors that influence adoption. The radiologists I interviewed are willing to use new technology but are not willing to change their current behaviour or workflow to use new technology. Many participants stated that they would like to avoid any disruptions in their workflow as much as possible since these add time to their already busy days. This finding is consistent with the importance Aidence assigns to workflow integration.
3. Expectations on AI uses and capabilities are high
Interviewees felt that a proper understanding of AI capabilities is very important. For some radiologists, taking the time to gain the knowledge of what the AI solution can detect and its accuracy had a negative influence on their perception of AI adoption. Others, however, say they can get back this time once they are using the solution.
Additionally, using Veye Lung Nodules to assess just one area in the body was perceived as limiting by some participants. Indeed, AI models are not currently capable of generalising their results to all findings in one scan. At the same time, Aidence is expanding its suite of clinical applications, addressing some of the other clinical needs indicated by radiologists.
What users are saying about Veye Lung Nodules
In this last section, Taylor gathered some of the quotes about Veye Lung Nodules obtained during her research. Whilst we cannot share their sources, they act as a reminder of how a good AI solution can make a difference for healthcare practitioners:
“It is so user friendly, it doesn’t complicate things, so to analyze what it’s worked out with all the nodules it doesn’t increase my time and the fact that it’s incorporated with the PACS and there’s no separate screen to go to. With Veye it really does save time because it’s a pain having to go through the computer and pull something else up.” (Radiologist, UK)
“I just use the measurements supplied to me by Veye Lung Nodules which saves me a lot of time and I feel is more accurate and more consistent.” (Radiologist, the Netherlands)
“Benefits to patients are quality based. It just makes sure the minimum quality delivered for the reporting of nodules is up to standards.” (Radiologist, the Netherlands)
“I think it will help us to do our job better and maybe focus more on other stuff like talking with our clinicians and ultrasound.“ (Radiologist, Denmark)
“It became apparent that it’s actually really good and reassuring as a radiologist to have a second pair of eyes.” (Radiologist, UK)
“It’s the first AI solution that we’ve implemented so it has given me and colleagues confidence that it adds to our workflow and adds to the way we work. And it has given the hospital a view on a different way to work with a software vendor that is easy to contact, helps us out and there is pleasant communication.” (Radiologist, the Netherlands)
Taylor on her internship experience at Aidence
I have learned so much while having the opportunity to intern at Aidence! First of all, working at Aidence has taught me how to work in a multi-national team which was such a diverse and unique experience. It was great to have lunchtime chats and even Friday quizzes learning about everyone’s home countries. I also think it was always an exciting and insightful experience to be able to work in a startup that is at the forefront of innovation in its industry. You really learn about the work that goes into the early stages of developing a product and the decisions that have to be made. I loved the environment at Aidence where everyone has the ability to collaborate, no matter which background or department you’re coming from. Overall my experience has been a positive one that makes me excited to work with more startups and new health technologies in the future.
Catalina Barzescu is the Content Manager at AidenceAfter graduating from the Erasmus University Rotterdam with a master’s degree in media and journalism, she made her way to MedTech via e-learning, travel, and humanitarian aid. Her interests are storytelling and data science with a purpose. The articles she signs reflect the work and ideas of the various teams at Aidence – medical, tech, regulatory, and business.