I am working in the Software as a Medical Device (SaMD). Software / AI is heavily regulated already since 3 yrs. In US 182 AI models are approved and in EU so far 52, some of which I certified for customers I consulted. In VERY short words, you need to have a lot of documentation ready, a valid QMS for developement, verification and validation. You need to have unbiased and evaluated (diverse) clinical training data and then for the validation of your trained LLM a SECOND independent Data Set with independent assessment (usually by doctors). You then need to have a study and test it in the wild with predefined expected outcomes and trust levels. Many companies are currently working on it or are in certification. My current assignment we got Class IIb EU MDR (the first company) which allows using algorithms for any kind of disease and patient condition and treat or diagnose for up to serious condition patients.
I am happy to guide everybody through it. For a start, pls. Google „MDCG2019-11“
Rudolf (Quality and Regulatory affairs expert ;-) )
And final word - you are responsible and accountable for patient safety and reporting of adverse events to authorities. Believe me, there are hundreds of companies working on that but failed or got removed by authorities (alone 10 in 2020 by US FDA)
Isn't that just like Med-PaLM 2 Google is working on? In my opinion a medical LLM is the most complicated as a single hallucination or wrong answer could potentially kill a person.
Interesting, need to read up on Med-PaLM 2. And agree, it should not be the primary weight in a diagnosis, but I can see a lot of good being made with this if targeting the most low hanging fruit first. Nothing should get released without rigour and fallbacks. I'm only hinting at whats possible, and that we could use AI for tremendous good here.
Allow me one question: Did you work on a project in healthcare in the past? You are definitely having some expert level knowledge on this topic! I can tell this because we did pretty much exactly what you suggest as our thesis project together with a research team from a leading German hospital back in 2020 😅
However I have to say that this was done with „traditional“ machine learning models in mind (that require structured data - which you don’t have in clinics usually) and I don’t know how generative models like GPT4 come into the equasion here.
I have zero healthcare expertise. Never worked in or close to healthcare. I'd say I'm got at connecting dots on a blank sheet of paper. I might also be terribly wrong here, hence the disclaimer and call to tell me I'm wrong. 🙏🏻
Seeing incredible opportunity to do good with AI in the medical field, and by the looks of the comments, so are many others too. I just did not know where to look. the internet is to big of a place.
Top brainstorming. Would love to see how you could apply this process to economic theory. Could AI be the leapfrog that economics needs to remove ideology from a so-called science?
I am working in the Software as a Medical Device (SaMD). Software / AI is heavily regulated already since 3 yrs. In US 182 AI models are approved and in EU so far 52, some of which I certified for customers I consulted. In VERY short words, you need to have a lot of documentation ready, a valid QMS for developement, verification and validation. You need to have unbiased and evaluated (diverse) clinical training data and then for the validation of your trained LLM a SECOND independent Data Set with independent assessment (usually by doctors). You then need to have a study and test it in the wild with predefined expected outcomes and trust levels. Many companies are currently working on it or are in certification. My current assignment we got Class IIb EU MDR (the first company) which allows using algorithms for any kind of disease and patient condition and treat or diagnose for up to serious condition patients.
I am happy to guide everybody through it. For a start, pls. Google „MDCG2019-11“
Rudolf (Quality and Regulatory affairs expert ;-) )
And final word - you are responsible and accountable for patient safety and reporting of adverse events to authorities. Believe me, there are hundreds of companies working on that but failed or got removed by authorities (alone 10 in 2020 by US FDA)
Isn't that just like Med-PaLM 2 Google is working on? In my opinion a medical LLM is the most complicated as a single hallucination or wrong answer could potentially kill a person.
Interesting, need to read up on Med-PaLM 2. And agree, it should not be the primary weight in a diagnosis, but I can see a lot of good being made with this if targeting the most low hanging fruit first. Nothing should get released without rigour and fallbacks. I'm only hinting at whats possible, and that we could use AI for tremendous good here.
Allow me one question: Did you work on a project in healthcare in the past? You are definitely having some expert level knowledge on this topic! I can tell this because we did pretty much exactly what you suggest as our thesis project together with a research team from a leading German hospital back in 2020 😅
https://ux-design-awards.com/winners/aliado
However I have to say that this was done with „traditional“ machine learning models in mind (that require structured data - which you don’t have in clinics usually) and I don’t know how generative models like GPT4 come into the equasion here.
I have zero healthcare expertise. Never worked in or close to healthcare. I'd say I'm got at connecting dots on a blank sheet of paper. I might also be terribly wrong here, hence the disclaimer and call to tell me I'm wrong. 🙏🏻
Seeing incredible opportunity to do good with AI in the medical field, and by the looks of the comments, so are many others too. I just did not know where to look. the internet is to big of a place.
Ongoing efforts in exactly this direction: https://www.mosaicml.com/blog/introducing-pubmed-gpt
Top brainstorming. Would love to see how you could apply this process to economic theory. Could AI be the leapfrog that economics needs to remove ideology from a so-called science?
Lawyers are loving this idea
This is brilliant!
I have read the original article mentioned. What are the uneducated statements?