AI In Healthcare: Who's Responsible in America?

On May 28th, 2026, the AMA, the American Medical Association, released a consumer guide on how patients could use AI for health. There was no actual recommendation of a specific product. Right, there was no ChatGPT, no Claude, no Gemini. Just use AI to supplement your doctor, not replace them. The obvious read or interpretation on that is that the AMA doesn't want to play favorites.

That might be part of it, but the other one is the AMA is a body of licensed physicians, not a federal regulator. They do not have authority over those AI companies. Which begs the question, who does? Is it the FDA? Is it the FTC? Is it the HHS? Is it your doctor? Is it the AI company itself? [snorts and gasps] I set out to find the answer to this question in this video.

Now, I built AI software for e-living in health care, and I'm going to walk you through every candidate party that I mentioned and more in this tree, starting with you. So, let's be honest first. Before AI even showed up, there were some things that you were responsible for. You've always been the one orchestrating your health care.

You schedule appointments. You decide which doctors to see, which doctors to trust, and even how to judge what they say in your own context. That role has always been there, your orchestration role. AI did not create that role. But, what AI changes is the territory or what that role orchestration actually involves.

And this is because the other parties in this whole system are using AI for health reasons. They're trying to make their processes more efficient. And so, how does them changing their processes affect you? Let's look at that. So the first branch is the federal regulatory agencies. Which agencies oversee AI in health and what is each one actually responsible for?

There are four candidates. The FDA's reach is defined by 60-year-old statute. Section 201H of the Food, Drug, and Cosmetic Act defines a medical device by its intended use. Five trigger verbs: diagnose, cure, mitigate, treat, prevent. If a software's product marketing uses those verbs, it meets the intended use test and the FDA does regulate it.

The consumer AI health products use those verbs only in negation. So for example, Microsoft co-pilot's health terms from its May 13th launch is says it is not intended to diagnose, treat, or prevent medical conditions. So that is three of the five verbs in negation. The disclaimer is a legal positioning move, not a statement of what the product actually does or delivers.

Now on the other hand, the FTC reaches deceptive marketing under section five. Its operation AI comply, which launched in 2024, went after false AI capability claims. Now that's real authority. But the scope limit is critical. So the FTC can act when a company falsely claims its AI is clinically validated.

It cannot act when a tool gives wrong medical advice, but does not claim to be accurate. So advice quality and marketing honesty are separated. As for the HHS, they reach federally funded entities through section 1557 of the ACA. OCR, which is a subdivision, enforces HIPAA on covered entities. So consumer chat GPT is not a covered entity.

A patient who uploads their own lab results to chat GPT has voluntarily stepped outside the HIPAA framework. So, each agency reaches its own statutory lane. None of them reaches a consumer chatbot answering a health question for you at home. So, the boundary isn't a policy gap. It is structural. Each agency's authority stops at a statutory edge.

So, federal agencies can reach the app that you're using directly. What about the bodies that govern your doctor? We've got two flavors here. We've got the state medical boards and they have statutory authority. And we've got the AMA and specialty boards and they have professional authority but no enforcement.

So, state boards can discipline a physician who relied negligently on AI tool. They can sanction a physician who failed to disclose that AI played a role in their diagnosis and they can sue AI companies for impersonating licensed physicians or practitioners. Pennsylvania filed exactly just that case on May 1st of this year 2026 against Character Technologies or Character AI.

Uh so, is extending a pre-AI doctrine that boards have used for decades against companies facilitating unlicensed medical practice. So, that is real authority that state boards are exercising. The AMA, on the other hand, published the consumer infographic that you saw at the start. It's written eight principles about AI and it's created CPT codes for AI augmented services so that the insurance could use.

It sent a letter to Congress in April 2026 for stronger safeguards on mental health chatbots. Um every one of those mechanisms is real and they've affected us in some way. But every single one of those get operationalized or executed by somebody else. State boards, courts, Congress, CMS. The AMA has no enforcement authority.

So, state boards can discipline your doctor for negligent AI reliance and they can sue an AI company when pretends to be a doctor or a licensed practitioner. That's a Pennsylvania case. But the consumer AI that you use at home isn't impersonating a doctor. ChatGPT [snorts] answers your questions without claiming to be a psychiatrist with a license number.

That is outside state board reach. The AMA can publish guidance, but it has no mechanism to make any company comply. Boards regulate licensees and impersonators. The AMA has no enforcement at all. So that's professional bodies. But what about the actors who decide whether your AI touch treatment gets paid for?

Of course, here I'm talking about insurance. So insurance play two roles in America. And they were separated. The first role is that AI deployers. Insurers are using algorithms to make coverage decisions. The canonical use case is United Health NH Predict algorithm. And it was used to determine discharge dates for Medicare Advantage patients in skilled nursing facilities.

A class action filed in 2023 and allowed to proceed in February 2025 alleges the algorithm produced systematically short recommendations with 90% of denials reversed on appeal. California passed SB 1012 effective January 2025 requiring licensed physician review of any AI based coverage details. So So AI is being used to deny you care, uh but some regulation like from California is closing in on that.

The second role that insurers play is AI constraints. So insurers shape which clinical AI gets deployed in the first place by deciding what gets reimbursed. Now, when a code exists and gets paid, that AI gets used in clinics. When it doesn't, the AI doesn't get deployed at scale. So insurers also gatekeep which clinical AI gets used in practice.

But both of those are inside the institutional system. AI denying your care or AI being deployed in your clinic. Neither touches the consumer AI tool that you're consulting at home. So we've walked three parties so far, the federal regulators, physician bodies, insurers. All three operate up around the patient.

They're the regulators, the credentials, and the financers. But none of them touches you directly. Now let's look at the three parties that do touch the patient directly. The providers who treat you, the AI companies whose products you actually use, and the advocates raising the alarms on your behalf.

So let's start with the health care providers. When your doctor or hospital is using AI, what are they responsible for? Well, when the provider is in the loop, four legal doctrines apply. Negligence is standard of care. So once a physician recommends or relies on AI tool, they accept responsibility for responding appropriately to its output.

Informed consent. So failing to disclose that AI played a role in diagnosis is a distinct breach even if the diagnosis was correct. Enterprise liability. So a hospital owes a non-delegable duty to vet and credential the AI tools that it deploys. So the hospital has to check that it works, basically. And four is vicarious liability.

So the hospital inherits the physician AI-related negligence. AI-implicated malpractice claims went up about 14% from 2022 to 2024. So every one of these doctrines requires a physician-patient relationship as the starting point. So when you act alone with a consumer AI tool home, the chain breaks. There is no physician in the loop, the standard of care doctrine has nothing to land on.

So the chain only reattaches when an institution, like your clinic, builds the AI into an official patient-facing product. So like Hartford Healthcare's Patient GPT. Outside of that, the accountability gap opens. So providers can reach AI in the clinic. So that's providers. What about the actors making the AI themselves?

The companies. All four major consumer AI companies shipped dedicated consumer health products in 2026. We got ChatGPT health on January 27th. Claude personal health on January 11th. Microsoft Copilot health on May 13th. And Google health coach also on May 19th. Each one of those uses your medical records.

Each one gets personalized health information about the specific conditions and data. And each one carries a disclaimer that positions it outside FDA's intended use test. The Copilot's terms, for example, from May 13th say it is not intended to diagnose, treat, or prevent medical conditions. We went over this previously.

And that is three of the five device verbs. In negation in a product that connects to your electronic health records. And here's the part that actually gets me. These same companies also ship fully compliant clinical products. Like OpenAI has ChatGPT for clinicians with peer-reviewed validation and HIPAA business association agreement.

Anthropic has Claude for healthcare with a human in the loop requirement for high-risk use cases. Microsoft has Copilot Studio with fully HIPAA-compliant documentation. So, they have the engineering capability built. They have done the validation. They have signed the legal agreements. They just choose to not do that on the consumer side.

That's because the consumer market doesn't require it. So, AI companies are enforceably responsible for The consumer side is what the FDA's intended use case catches. The disclaimer is positioned for those companies to stay outside the intended use case. The consumer health AI market is self-regulated.

The only enforcement mechanism is whatever the company decides to enforce on itself. AI companies regulate themselves and don't do the same validation that they do for consumers that they do for clinicians. So, who's left to make noise? It's the patient advocacy groups and accountability media have been very active on this.

The Consumer Federation of America led a coalition of 22 organizations filing complaints with the FTC. All 50 state AGs and all 50 state mental health licensing boards in June 2025 targeting AI tools impersonating therapists. Common Sense Media and Stanford published an assessment calling consumer AI chatbots fundamentally unsafe for teen mental health support in November 2025.

Stat News has been running investigations into AI cover shin dials. Reuters exposed Meta's content trust standards. Those are real actions. What advocacy and media cannot do is compel anything directly. So, every documented AI behavior change in this period from OpenAI's parental controls to United Health's scrutiny.

It happened through voluntary corporate response or civil litigation. It's never through the direct authority of any advocacy groups or news outlets. They can build the record. They cannot enforce it. And that brings us back to you. So, we've walked through six branches. Each of them has a clean story about what they're responsible for.

So, let me show you what the whole picture looks like because there are three patterns running through this tree and where they land tells you exactly what you've asked at the beginning, what are you responsible for? So, pattern one is clear boundaries. Some harms have one party with a clean lane and then action happens.

We saw this most clearly at the federal regulators branch. FTC reaches deceptive marketing. FDA reaches product that satisfies section 215 intended use. OCR reaches HIPAA-covered entities. HHS reaches federally funded discrimination. Each one is a clear boundary, one party, one lane. Action happens when triggered.

So, about four zones across this tree look like this. And where these exist, somebody else has this zone. The second pattern is diffuse boundaries. So, multiple parties have partial reach. But no one owns the harm end to end. And when no one owns it end to end, non-action is the structural result. The clearest example on this tree is the AI mental health chatbot harm.

So, eight parties weighed in on the same problem. The AMA wrote Congress. The American Psychiatric Association published a position statement. The FTC issued an information gathering order to seven major AI companies. Coalition of 22 organizations filed complaints with all 50 state attorney generals and all 50 mental health licensing boards.

Advocacy groups published assessments. Journals ran investigations. Eight wrongful death suits were filed in California. Compelled enforcement actions across all eight parties. So, across all those eight parties, there were zero compelled actions. The only change came from OpenAI voluntarily parental controls.

If OpenAI had decided not to, nothing would have moved. So, we see this pattern at the physician governing bodies branch, where the AMA has eight mechanisms and no enforcement. And at the advocacy branch, where every documented behavior change routed through voluntary corporate response. So, it's a classic bystander effect, where everybody has a PhD and a press release.

The third pattern is the gaps. So, no party in this tree has authority over the harm at all. Not partial, none. The example here is Microsoft Copilot Health, launched May 13th, 2026. The product connects to your EHR via health exchange, Health X. It offers personalized information about your health and wellness based on your specific data.

The disclaimer says it is not intended to diagnose, treat, or prevent medical conditions. So, who can compel either commitment to mean anything? Not the FDA. the disclaimer keeps the product outside the intended use case. Not the FTC, Microsoft didn't claim accuracy, so there's no deceptive marketing claim to act on.

Not state boards, Microsoft isn't a licensee. Not the AMA, there's no enforcement. Not your doctor, the chain doesn't reattach because you're not going through your doctor or your doctor's clinic. Not advocacy because they don't have direct authority, so the accountability map runs out of paper. About five zones across the street look like this.

So, back to the question we came in with, what are you responsible for? The clear boundary zones, somebody has those. Diffusing the gap zones, you're responsible for those, in my opinion, cuz no other party is. That's the expanded orchestration territory with AI. You've always orchestrated your own healthcare.

What's changed with AI is that the orchestration role now includes verifying things no institutional party is verifying for you. The diffuse zones and the gap zones are where your burden is, and that is the structural fact about how this accountability landscape was built. Not a complaint about any one of those parties.

The AMA's cautions at the beginning of this video, never use AI in emergencies, don't rely on AI for diagnosis, protect your privacy, use AI to supplement your doctor. Those are real, useful cautions. The reason they're general, with no specific tool named, is the structural admission that we just mapped.

The AMA cannot reach the gap zones to evaluate specific tools. Neither can the FDA for tools positioned outside the intended use test. Neither can the FTC the for products that disclaim accuracy rather than claim it. No party industry can do that evaluation for you at scale. So, that's a big ask. You should have to build a full accountability framework from scratch for every consumer AI product that you use.

So, that's what I'm trying to do with the NAS Health Trust list. I'm trying to fill some of those diffuse areas of responsibilities as some of those gaps. Obviously, it's a limited tool. It's, you know, it's outside the jurisdictions like we talked about. And it's, you know, not per patient, so you can't really trust it for your own specific context, right?

That's really important. It's just me doing some of the work, and it's definitely not the full answer. If that's something that you think might be valuable to you, I'm going to leave a link in the description and comment. But, if you want to know how to run validations yourself, I made a separate video about that with a bunch of products that you probably already know ranked from F to S.

So, the link for that is right here.