‘AI’ Is Supercharging Our Broken Healthcare System’s Worst Tendencies
from the I'm-sorry-I-can't-do-that,-Dave dept
“AI” (or more accurately language learning models nowhere close to sentience or genuine awareness) has plenty of innovative potential. Unfortunately, most of the folks actually in charge of the technology’s deployment largely see it as a way to cut corners, attack labor, and double down on all of their very worst impulses.
Case in point: “AI’s” rushed deployment in journalism has been a keystone-cops-esque mess. The fail-upward brunchlord types in charge of most media companies were so excited to get to work undermining unionized labor and cutting corners that they immediately implemented the technology without making sure it actually works. The result: plagiarism, bullshit, a lower quality product, and chaos.
Not to be outdone, the very broken U.S. healthcare industry is similarly trying to layer half-baked AI systems on top of a very broken system. Except here, human lives are at stake.
For example UnitedHealthcare, the largest health insurance company in the US, has been using AI to determine whether elderly patients should be cut off from Medicare benefits. If you’ve ever navigated this system on behalf of an elderly loved one, you likely know what a preposterously heartless shitwhistle this whole system already is long before automation gets involved.
But a recent investigation by STAT showed the AI consistently made major errors and cut elderly folks off from needed care prematurely, with little recourse by patients or families:
“UnitedHealth Group has repeatedly said its algorithm, which predicts how long patients will need to stay in rehab, is merely a guidepost for their recoveries. But inside the company, managers delivered a much different message: that the algorithm was to be followed precisely so payment could be cut off by the date it predicted.”
How bad is the AI? A recent lawsuit filed in the US District Court for the District of Minnesota alleges that the AI in question was reversed by human review roughly 90 percent of the time:
“Though few patients appeal coverage denials generally, when UnitedHealth members appeal denials based on nH Predict estimates—through internal appeals processes or through the federal Administrative Law Judge proceedings—over 90 percent of the denials are reversed, the lawsuit claims. This makes it obvious that the algorithm is wrongly denying coverage, it argues.”
Of course, the way that the AI is making determinations isn’t particularly transparent. But what can be discerned is that the artificial intelligence at use here isn’t particularly intelligent:
“It’s unclear how nH Predict works exactly, but it reportedly estimates post-acute care by pulling information from a database containing medical cases from 6 million patients…But Lynch noted to Stat that the algorithm doesn’t account for many relevant factors in a patient’s health and recovery time, including comorbidities and things that occur during stays, like if they develop pneumonia while in the hospital or catch COVID-19 in a nursing home.”
Despite this obvious example of the AI making incorrect determinations, company employees were increasingly mandated to strictly adhere to its decisions. Even when users successfully appealed these AI-generated determinations and win, they’re greeted with follow up AI-dictated rejections just days later, starting the process all over again.
The company in question insists that the AI’s rulings are only used as a guide. But it seems pretty apparent that, as in most early applications of LLMs, the systems are primarily viewed by executives as a quick and easy way to cut costs and automate systems already rife with problems, frustrated consumers, and underpaid and overtaxed support employees.
There’s no real financial incentive to reform the very broken but profitable systems underpinning modern media, healthcare, or other industries. But there is plenty of financial incentive to use “AI” to speed up and automate these problematic systems. The only guard rails for now are competent government regulation (lol), or belated wrist slap penalties by class action lawyers.
In other words, expect to see a lot more stories exactly like this one in the decade to come.
Filed Under: ai, automation, chat-gpt, coverage denied, healthcare, language learning models, medicare
Companies: unitedhealthcare
Comments on “‘AI’ Is Supercharging Our Broken Healthcare System’s Worst Tendencies”
” using AI to determine whether elderly patients should be cut off from Medicare benefits.”
Sounds like one of those ‘Advantage’ plans which have nothing to do with medicare. Medicare benefits are determined by the government agency, not a private insurance company.
They should stop calling their private insurance medicare.
Re:
Medicare Advantage plans are Medicare, just a form that incentivizes the insurance companies to maximize profits by denying care.
Re: Re:
That is incorrect.
Medicare advantage plans are Medicare Approved private insurance.
There is no part C to original medicare.
Re: Re: Re: medicare part
Medicare has parts A to N
https://www.medicare.gov/health-drug-plans/medigap/basics/compare-plan-benefits
Re: Re: Re:2
Part C is private amendment to the government plan. It’s supposed to be regulated within the government process but rarely is.
Ever tried getting a google decision reversed? A youtube strike …
There’s a reason why this https://en.wikipedia.org/wiki/Computer_says_no rings true, or, not yet but watch this sapce https://en.wikipedia.org/wiki/Computers_Don%27t_Argue
Re:
Card Says Moops
https://www.urbandictionary.com/define.php?term=%22The%20Card%20Says%20Moops%22%20Maneuver
I’d love to see all of these “pro-life” GOP legislators go after insurance companies that sloppily use AI to deny necessary medical care. Unfortunately, the only consistent aspect of right-wing ideology is its inconsistency, and that’s by design because the cruelty is the point. I fully expect most “pro-life” legislators to sit on their hands and do nothing about this situation while squawking endlessly over culture war nonsense–all while their own constituents’ health continues to decline.
Re:
You know, I’d be quite ok if the right decisions and policies affected only the right. Their constituents would be reduced to a few hundred super rich people quite fast. Sadly they affect us all.
Public Healthcare
The solution to this kind of fuckery is am universal, public, free healthcare. It is feasible and it is affordable if you are willing to deliver the bare minimum of tax equality and wealth distribution.
Perhaps that’s why private actors are trying so hard to destroy said system here in Brazil. When health insurance plans and private hospitals started going the US route people simply dropped out and used the public system and that’s why prices of private healthcare here are outrageous but not inhumanly and insanely outrageous like in the US.
Same for pharma. If they force their hands they simply lose the patents. Along with generics.
When the government actually rules including the people in the equation and not only the filthy rich things work. And Brazil is a cesspool of inequality so we are talking about the bare minimum. I’m fairly sure the US can do it instead of warmongering and appeasing to the few dozen hundred super rich.
This comment has been flagged by the community. Click here to show it.
Re:
There is no such thing as “Free Healthcare”, only someone else paying for the healthcare. With “free” healthcare, people have no incentive to use healthcare less, and will take riskier life decisions. However in many socialized countries everyone gets free rationed care, which effectively means that you have to wait weeks or even months for procedures.
Re: Re:
Is this why the US spends far more per capita on healthcare than any other OECD country, yet continues to be near the bottom in life expectancy and quality of care?
From my experience, expensive healthcare actually encourages risky behavior. By charging a premium for preventative care and checkups, people are more likely to skip these measures. Sometimes, these problems get far worse, which is when they will certainly be more expensive to address. The whole system is designed to maximize executives’ profits. I’m not going to spend $90 to visit the doctor’s office only to be told to wait something out, but the next time I get those same symptoms, it might not be the same story.
Re: Re: Re:
“From my experience”
What might that be?
Are/were you a medical professional?
Maybe an insurance adjuster.
“expensive healthcare actually encourages risky behavior”
I’d be interested in the data from which this conclusion was drawn. I always thought it was the alcohol.
Re: Re: Re:2
It’s not hard to understand that. If you have to pay for a routine review, most people skip it. Save the money. Minor aches and pains are ignored. And major medical problems are missed at early, inexpensively treatable timing because of it.
People skip the basics because the basics cost the same as the emergency. The. You have more emergencies, which cost the industry more but the people the same. That drives up costs overall.
The way the system is here more people cost more money because nobody (generic) goes for the basics. More people get sick. More people die. All because the government would rather send tanks and bombs to other countries, rather than spend a small bit of tax dollars on basic care needs here.
Cut off Ukraine. We funded healthcare for a year. Close Gtm. We funded healthcare for a year. End USAid. We funded healthcare annually.
Part of what makes the social care system work is more people get the basics and then less people have expensive medical issues.
Re: Re:
Call me a bleeding heart if you want, but I’m okay with my tax dollars going to pay for my fellow human beings’ healthcare.
As opposed to these great United States, where you’re always seen promptly and clinics are never booked out 7 months.
Re: Re: Re:
I had to wait six months to see a cardiologist this year, even though I was having symptoms that could have possibly been a clogged artery. (Turned out to be my blood pressure meds causing side effects, despite my taking them for many years.) The cardiologist simply had no earlier slots available for a new patient.
People dragging up the waits bullshit clearly don’t use medical care much. Or they’re lying. Can’t rule that one out.
Re: Re:
Yes there is. Taxes are paying for it. But it is effectively free as in you won’t spend directly. And it can be done if taxes are applied proportionally.
As for the riskier behavior it’s actually the opposite for the vast majority of the population. Most people tend to do regular check ups and will actually call an ambulance if they have exposed fractures instead of yelling at people not to do it because they don’t have money to afford it.
Finally, rationed care is not the problem you make it seem to be. You don’t need to promptly serve everybody, you just need readily available general practice to act as a triage. You don’t need to have prompt medical attention to check your eyes for new glasses for example. Queues are not the end of the world. And even then, it’s better than NOT having healthcare because you can’t pay.
Re: Re: Re:
As an example, a friend in Switzerland gave birth and she receives regular visits from a nurse to keep tabs on her son’s health. Here in Brazil we have healthcare agents that regularly visit households that request such services to check on the family and forward any family members in need of medical attention. There are many more vulnerable and hardly accessible places where doctors themselves visit the households (they are called family doctors). Again, in a country within the worst inequality levels of the world.
It can be done. And it’s less expensive than your right wing economics say it is. Ans systemically speaking it actually saves dollars for the economy.
Re: Re: Re:2
I went from being diagnosed with a torn retina to laser-surgery to being home in 4 days. It cost me less than $100-120 which I can be reimbursed for (but I couldn’t be bothered to file the paperwork). I got paid sick-leave for 2 weeks to allow the eye to heal enough before going back to work. I have had 2 free checkups since then.
That is what universal healthcare/single payer system allows. Thank the Holy Noodleyness that I don’t live in the US.
Re: Re:
wow, you managed to reply in a way that completely contradicts reality yet again!
places that have universal healthcare pretty universally have higher life expectancy and less utilization of healthcare for minor issues.
Re: Re:
Your right, there’s no such thing as truly “free” healthcare; instead, taxes pay for that. But my logic for this is this: Free healthcare is paid for in taxes, but free healthcare is beneficial to all individuals. So, it is reasonable to expect all individuals to be willing to pay for said healthcare in taxes given that said healthcare benefits not just them, but everybody else. Maybe I’m a bleeding heart, but I do know that free healthcare can be done. It’s already been done in many countries already. And, personally, I’d rather have free healthcare than get charged $300000 for a medical bill and have to deal with insurance agencies that, somehow, get the right to dictate to me what healthcare I can and can’t get, on the basis that if I get something they say I can’t get, they won’t pay for it, which gives them absolute control over what I can and can’t get unless I somehow get the money to pay for it out of pocket.
Re: Re:
Free at point of service. We understand how taxes work, Einstein.
This comment has been flagged by the community. Click here to show it.
Karl Bode is a liar
AI didn’t do any of this, it was human managers, who decided to misuse AI, as the bode explains the AI wasn’t told that the patients had covid, yet bode blames AI for the humans not telling it those details. Its like complaining that Databases are bad, because the databases were used for decisions, and someone didn’t update the database.
Karl Bode of course has no interest about how AI has folded every single human protein, or is doing drug discovery, or is finding the single nucleotide polymorphisms in cancer genomes for diagnosis, because it doesn’t align with his “CAPITALISM BAD” viewpoint.
Re:
so, AI didn’t fold every single protein, or we wouldn’t still be doing folding.
as for the rest of your nonsensical rant, you missed the fact that the entire paragraph preceeding the single mention of covid was a quote from someone else, and no where did it say that the AI wasn’t informed, it states that the AI didn’t take it into account.
You really need to english harder if you wanna rile people up 🙂
Here is the quote for you:
“It’s unclear how nH Predict works exactly, but it reportedly estimates post-acute care by pulling information from a database containing medical cases from 6 million patients…But Lynch noted to Stat that the algorithm doesn’t account for many relevant factors in a patient’s health and recovery time, including comorbidities and things that occur during stays, like if they develop pneumonia while in the hospital or catch COVID-19 in a nursing home.”
where does that say the AI wasn’t informed? right. Keep makin shit up, eventually you might accidentally make a truthful statement! 🙂
Re: Re:
As an aside, the protein folding was done by supercomputers, then by things like Folding@Home, which are not exactly AI, but rather, algorithms using distributed computing power to do a very specific thing. (Which could be construed as AI, but not the procedural content generator scam masquerading as AI.)
As for revenge porn man, he’s not even wrong.
He’s not even addressing the topic.
What their "AI" likely is
Their model: select avg(recoveryTime) from patient_care … maybe if they are being super accurate they will use median in place of average.
To paraphrase, the solution to a bad guy with AI is a good guy with AI. So, I see more AI as the solution.
We look at the situation, the insurance company used AI to violate Medicare standards. But different AI could check compliance with these standards. CMS needs its own AI to monitor Medicare Advantage.
It still boils down to what the standards are. AI doesn’t change that. It is still a true-false situation. Either the Medicare Advantage plan complies with the standards, or it doesn’t.
The piece that is missing is, the referee (CMS) doesn’t have AI. It doesn’t have to stay that way.
They have been teaching us, for a LONG TIME
Insted of having a Nice and Fair system, over the many years Social sec. and Medical has been around.
YES, MEDICAL and PRIVATE hospitals WERE CREATED, After it was almost free.
The Systems created are teaching us WE HAVE TO CHEAT. Many Doctors Know how it works and WILL back you up.
But the Corps keep pointing to the 1-5% that Fake it, and “They Say” are costing them More money.
“THERE IS A SOLUTION”
We can do as they want. Cut all the people out of the jobs. LET the computers have them.
Its inevitable. So lets Shove it down their Throats.
As with NO jobs, and NO ONE PAYING US, Everything BECOMES FREE. Correct?(NOT ON THEIR LIVES)
They recently got a taste of this with the Flu, and 2 years of it. And workers NOT wanting CHEAP wages when they went back.
Its Fine. Lets take out the bottom Rung of Capitalism. The Lowest consumers. The Bottom employees.
How much are the willing to pay for the NEXT rung up, of workers to do those jobs?
All of this is like the OLD Miner Songs, Work hard get paid, the town belongs to the Owners, Pay them for Food so they can pay you Next week..
THEN if you get sick/hurt. Dont give a Care.
Hire someone Else, and get rid of the other.
Ever hear of the Persons that pay for Insurance for Years, HaVE AN ACCIDENT, be it the house, Medical, Car, Kids, And the agency Drops them? It does happen.
Life insurance, the Forgotten thing. You never make a KIT, in case Something happens. And the Life insurance Never gets claimed.
THERE ARE companies that Pay for Life insurance on EMPLOYEES, and keep it even after they quit/fired/Leave.
Something making the ‘wrong’ call 90% of the time isn’t broken, it’s working as intended. The executives are directing the company to act illegally and using the computer program to disguise it.
Re:
Are we describing the healthcare system or the RIAA?