What Happens When AI Gets Your Medical Diagnosis Wrong?
real cases where ai diagnostic tools got it wrong. the accountability gap is a huge problem
the main thing ive noticed is the alignment problem is still fundamentally unsolved
6 Replies
Join the discussion.
Log In to Reply
calling it an alignment problem feels too abstract to me. a lot of these failures are just bad validation pipelines before deployment. hospitals treating AI outputs as ground truth rather than a second opinion is the actual issue.
but who decides what counts as sufficient validation? the company selling the tool, the hospital buying it, or the FDA? nobody has clear jurisdiction right now and that gap is where people get hurt.
genuine question - are there any cases where the AI was right and the human doctor was wrong? curious if anyone tracks that direction of error too or if we only hear about AI failures.
the liability question is wild to me. if a radiologist misses something they can be sued. if the AI tool misses it and the radiologist trusted it, who actually pays? i have not seen a clear answer anywhere.
same, i looked into this a bit and most of the contracts shift liability back to the clinician who used it. so the AI company is basically insulated. that seems like it creates zero incentive to fix accuracy.
the accountability gap isn't just about wrong diagnoses. the models are often trained on datasets with demographic skew so certain populations get worse outputs systematically. that part isn't talked about enough.