Artificial Intelligence for Health – Data and Model Deployment Determine Real World Solutions

  • How do we move from our proof of concept then, to a product in practice using AI in dermatology ?
  • For this we need to ask the right questions after understanding the true needs of our patients and providers in our health systems.
  • Only then can solution be created, which is specific for those use cases and those users.
As you can see Data is at the center of it all, it trains the model and fine tunes it when the model it rolled out to the real world. Source of image: www.fda.gov/digitalhealth

The first thing is to get the right Data

  • Data is the fuel of this new economy.. In the real world, it strikes as being quite messy meaning it can be incomplete and incorrect.
    • Labelling and annotation is really human-capital intensive, so getting clean, high-quality data is a challenge.
  • However usable data is critical for the:
    • initial development of models
    • retraining of models
    • fine tuning models, so that we have an ever valid model.
  • Then continued flow new data is required to assess the clinical validation of how well it performs in clinical settings and in the real world.
  • Challenges:
    • However, a lot of our data lacks population diversity:
      • There was just a recent study (2020) that showed that most of the US data cohorts are based in California, Massachusetts and New York.
      • There is also lack of racial diversity and economic diversity in our data sets.
    • In addition, we also have siloed data, and these are unsharable as being part of the electronic health records.
    • Altogether messy data, homogeneous data and siloed data, can perpetuate and amplify and exacerbate bias and healthcare disparity. Indeed the developed model may only function so well in the real world, depending after all only on the real-world data it has received.

The second thing is to deploy the model

  • When we think about clinical deployment, so when we’re thinking about moving the model outside of the “in silico” setting to the in vivo setting, we want to think first of “primum non nocere” (first, do no harm).
    • Is this AI intervention safe?
    • Is this AI intervention effective?
    • Does it translate to the clinical setting ?
    • And then once we think about translation into the clinical setting, does it then translate into the real world?

Guidelines

  • There have been recent guidelines (2020) that have been published in Nature Medicine, and these are reporting guidelines for reporting monitoring of AI solutions implemented in the clinical trial setting:
    • CONSORT-AI guidelines
    • SPIRIT-AI guidelines

Note that the presenter has used the concept of Augmented Intelligence instead of Artificial Intelligence. Please refer to the introductory article of this term.

Ivy Lee, MD. Augmented Artificial Intelligence in Dermatology. Core concepts and clinical decision making. 8th World Congress of Teledermatology, Skin Imaging and AI in Skin diseases – November 2020

Categories

Scroll to Top