I am working in the Software as a Medical Device (SaMD). Software / AI is heavily regulated already since 3 yrs. In US 182 AI models are approved and in EU so far 52, some of which I certified for customers I consulted. In VERY short words, you need to have a lot of documentation ready, a valid QMS for developement, verification and validation. You need to have unbiased and evaluated (diverse) clinical training data and then for the validation of your trained LLM a SECOND independent Data Set with independent assessment (usually by doctors). You then need to have a study and test it in the wild with predefined expected outcomes and trust levels. Many companies are currently working on it or are in certification. My current assignment we got Class IIb EU MDR (the first company) which allows using algorithms for any kind of disease and patient condition and treat or diagnose for up to serious condition patients.
I am happy to guide everybody through it. For a start, pls. Google „MDCG2019-11“
Rudolf (Quality and Regulatory affairs expert ;-) )
Isn't that just like Med-PaLM 2 Google is working on? In my opinion a medical LLM is the most complicated as a single hallucination or wrong answer could potentially kill a person.
Allow me one question: Did you work on a project in healthcare in the past? You are definitely having some expert level knowledge on this topic! I can tell this because we did pretty much exactly what you suggest as our thesis project together with a research team from a leading German hospital back in 2020 😅
However I have to say that this was done with „traditional“ machine learning models in mind (that require structured data - which you don’t have in clinics usually) and I don’t know how generative models like GPT4 come into the equasion here.
Top brainstorming. Would love to see how you could apply this process to economic theory. Could AI be the leapfrog that economics needs to remove ideology from a so-called science?
I am working in the Software as a Medical Device (SaMD). Software / AI is heavily regulated already since 3 yrs. In US 182 AI models are approved and in EU so far 52, some of which I certified for customers I consulted. In VERY short words, you need to have a lot of documentation ready, a valid QMS for developement, verification and validation. You need to have unbiased and evaluated (diverse) clinical training data and then for the validation of your trained LLM a SECOND independent Data Set with independent assessment (usually by doctors). You then need to have a study and test it in the wild with predefined expected outcomes and trust levels. Many companies are currently working on it or are in certification. My current assignment we got Class IIb EU MDR (the first company) which allows using algorithms for any kind of disease and patient condition and treat or diagnose for up to serious condition patients.
I am happy to guide everybody through it. For a start, pls. Google „MDCG2019-11“
Rudolf (Quality and Regulatory affairs expert ;-) )
Isn't that just like Med-PaLM 2 Google is working on? In my opinion a medical LLM is the most complicated as a single hallucination or wrong answer could potentially kill a person.
Allow me one question: Did you work on a project in healthcare in the past? You are definitely having some expert level knowledge on this topic! I can tell this because we did pretty much exactly what you suggest as our thesis project together with a research team from a leading German hospital back in 2020 😅
https://ux-design-awards.com/winners/aliado
However I have to say that this was done with „traditional“ machine learning models in mind (that require structured data - which you don’t have in clinics usually) and I don’t know how generative models like GPT4 come into the equasion here.
Ongoing efforts in exactly this direction: https://www.mosaicml.com/blog/introducing-pubmed-gpt
Top brainstorming. Would love to see how you could apply this process to economic theory. Could AI be the leapfrog that economics needs to remove ideology from a so-called science?
Lawyers are loving this idea
This is brilliant!
I have read the original article mentioned. What are the uneducated statements?