Nobody Told Them Not to Paste It: AI Chats, Connectors, and Your HIPAA Risk Analysis (2026)

It is 4:40 on a Thursday at the billing company you hired. A rep named Dana has a stack of denials and a payer deadline in the morning. She opens an AI chat in another browser tab and pastes in a full denial letter: the patient's name, date of birth, member ID, and the diagnosis code the payer rejected. She types, "Write me an appeal." Thirty seconds later she has a clean, professional letter. She does it eleven more times before she logs off.

Across town, at your front desk, Marcus is behind on reminder calls. He copies tomorrow's schedule, every name with a phone number and a reason for the visit, into the same kind of chat and asks for a friendly reminder text for each patient. It works beautifully.

Neither of them is careless. Both are trying to do right by patients. And nobody, at your organization or at the vendor, ever told them in writing that this was a problem. (Dana and Marcus are illustrations, not real people or client cases. You may still recognize them.)

The gap is not a bad actor. It is a rule nobody wrote down, a vendor contract older than the tool, and a risk analysis that has never heard of it.

Nobody said no out loud

Most organizations under 1,000 employees I talk to have one of two AI policies: silence, or one sentence in an acceptable-use document nobody has opened since onboarding.

Silence is not neutral. Your workforce must be trained on your PHI policies as needed for their jobs, the training must be documented, and a material policy change means training on the change. (Source: 45 CFR 164.530(b).) With no written AI rule, there is nothing to train on, and people fill the gap with whatever seems helpful.

Vendors are a different lever. "Workforce" means people whose work is under your direct control. (Source: 45 CFR 160.103.) Dana is not your workforce. You do not train her. You govern her through the business associate agreement, which must limit what the vendor may do with your PHI, require safeguards and reporting, and bind its own subcontractors to the same terms. (Source: 45 CFR 164.504(e)(2).)

So the question for a vendor is not "did you take our training?" It is "what does our agreement say about AI, and what have you told your people?"

What happens when someone hits send

You do not need to be technical to follow this.

The chat window is an app sitting on top of the AI. The AI is the part that writes. The app is everything around it: the login, the history, the settings, the buttons.

The app does not send just your sentence. It bundles your message with its own instructions, any instructions you saved, any memories it keeps about you, any files you attached, and the conversation so far.

That bundle leaves your building. It goes to computers run by the vendor or its partners.

Some apps hand work to other software. A connector or plugin lets the AI pass text to a file drive, an email account, or a marketplace add-on, which does the job and sends results back.

Every step above is a place PHI can go, and your risk analysis has to know about each one.

A new chat is not empty, and neither is the next message

This is the part most people get wrong. The AI has no memory of its own. So every time someone sends a new message, the app reminds it of everything by sending the whole session again: the full conversation up to that point, plus the saved memories, the instructions, the attached files, and whatever the connectors pulled in.

Go back to Dana. The denial letter goes in her first message. Then she asks five more questions about appeal deadlines and payer rules that have nothing to do with that patient. That patient's name, birth date, and diagnosis ride along with every one of those five messages. Pasted once. Sent again and again, every turn, for the rest of that session. How a given vendor stores or logs those turns is a question for its documentation and your agreement. The resending is simply how the conversation works.

And a "new chat" may not start clean. Saved memories, custom instructions, and project files can be packed into the next conversation too.

Two rules you already follow apply directly.

Minimum necessary. When you use or disclose PHI, you must make reasonable efforts to limit it to the minimum necessary for the purpose. (Source: 45 CFR 164.502(b).) A question about appeal deadlines does not need a patient's name. A long chat with a name in the first message keeps sending it long after the question changed.

Risk analysis. This is a Required implementation specification: an accurate and thorough assessment of the potential risks to the confidentiality, integrity, and availability of the electronic PHI you hold. (Source: 45 CFR 164.308(a)(1)(ii)(A).) An analysis that pictures a chat as one sentence sent once is not accurate. The honest version names the tool, its memories and files, its connectors, and the fact that everything is resent every turn.

Connectors are separate software, and often separate companies

This is where "we have the enterprise plan with a BAA" stops being the whole answer.

Say you have the enterprise workspace and the signed BAA. That agreement covers the services it names. How to read what a BAA actually covers is in Is ChatGPT HIPAA Compliant? A Badge Is Not a BAA, so I will not repeat it here.

Now someone switches on a connector to a note app, a transcription add-on, or a plugin from a marketplace. The AI hands text to that software. If another company runs it, your patient information just went to that company.

HHS has been clear: an outside service that creates, receives, maintains, or transmits electronic PHI for you is a business associate and needs a BAA, even if it cannot read the data. Using one without a BAA is a violation. (Source: HHS, Guidance on HIPAA & Cloud Computing, content last reviewed December 23, 2022.)

So sort every connector into one of three plain buckets:

• A third-party app someone at your organization turned on. Treat it like any new vendor. No BAA, no PHI. (Source: 45 CFR 164.502(e)(1)(i); 45 CFR 164.308(b)(1).)
• A company your AI vendor uses behind the scenes. Your vendor must bind it to the same terms. You do not sign with every company in that chain, but your agreement should require it. (Source: 45 CFR 164.502(e)(1)(ii); 45 CFR 164.504(e)(2)(ii)(D).)
• A system you already cover, such as your own file drive under its own BAA. The question there is whether that system is approved for PHI and whether the AI tool may read it.

If nobody can tell you which bucket a connector is in, that is a finding.

The same logic reaches your vendors. If Dana's employer lets staff paste your patients into a consumer chat, that is likely a use your agreement does not allow. If you learn of a pattern like that, take reasonable steps to cure it, and if that fails, end the arrangement if feasible. (Source: 45 CFR 164.504(e)(1)(ii).)

What OCR has said, and what it has not

As of October 19, 2026, I could not find an OCR resolution agreement or civil money penalty on HHS's public list that is about AI. (Source: HHS, Resolution Agreements and Civil Money Penalties, content last reviewed October 9, 2026.) That is not permission. The lessons come from rules OCR already enforces.

A vendor holding records with no agreement. The Center for Children's Digestive Health, a small pediatric practice in Illinois, paid $31,000 and agreed to a corrective action plan. It shared PHI with a records-storage vendor starting in 2003, and neither side could produce a signed BAA before October 12, 2015. (Source: HHS, No Business Associate Agreement? $31K Mistake, April 20, 2017.)

A vendor nobody vetted. Advanced Care Hospitalists agreed to pay $500,000 and adopt a corrective action plan. HHS described the case as sharing protected health information with an unknown vendor without a business associate agreement. (Source: HHS, Advanced Care Hospitalists, December 4, 2018.)

Settlements resolve potential violations, not court findings, and neither case involved AI. Both describe an organization that could not show who held its patient information or under what contract. A connector nobody listed is the same problem with a newer screen.

Where AI tools and connectors belong in the program

None of this needs a new HIPAA rule. It needs the tool written into the rules you already follow. (For the federal and state AI rules beyond HIPAA, see AI in healthcare: the rules, in plain English.)

Risk analysis: Required. Name every AI tool that can reach PHI, including personal accounts people actually use, the memories and files those tools keep, and every enabled connector. (Source: 45 CFR 164.308(a)(1)(ii)(A).) A form you fill out once is not this; see why a questionnaire is not a risk analysis.

Risk management: Required. Reduce the risks you found to a reasonable and appropriate level, and keep the proof. (Source: 45 CFR 164.308(a)(1)(ii)(B).)

Vendor due diligence. Get satisfactory assurances first; the written contract documents them. (Source: 45 CFR 164.308(b); 45 CFR 164.502(e).) Ask every vendor that touches PHI which AI tools and connectors it uses with your data. More in A Signed BAA Is Not Satisfactory Assurance.

Policies. Write the AI rule down, keep it six years, make it available to the people who use it, and review it when things change. Each of those is Required. (Source: 45 CFR 164.316(b).)

Training. Security awareness and training is a standard you must meet for all workforce members, including management. Its four implementation specifications, including security reminders, are Addressable. (Source: 45 CFR 164.308(a)(5).) Addressable does not mean optional: you implement it if reasonable and appropriate, or document why not and adopt an equivalent alternative if that is reasonable and appropriate. (Source: 45 CFR 164.306(d).) A reminder that names the AI tools people actually have meets it cheaply.

Sanctions and incidents. The sanction policy and security incident response and reporting are both Required. (Source: 45 CFR 164.308(a)(1)(ii)(C) and (a)(6)(ii).) A sanction for breaking a rule nobody was told about will not hold up. Write the rule first.

The audit-side view of the same rules is in The HIPAA Audit Protocol Has 180 Questions. None of Them Say AI.

What to put in place this week

You do not need a committee. You need five things, in this order.

1. Write a one-page AI rule. Which tools are approved for PHI, and which are not. Say plainly that patient names, birth dates, member IDs, schedules, and chart text do not go into any tool that is not on the approved list. Send the same page to your vendors.
2. List the tools and the connectors. Every AI tool your staff use, including personal accounts on work devices, and for each approved tool, every connector that is switched on, who turned it on, and which company runs it. This list becomes a new section of your risk analysis.
3. Use the admin controls. On business plans, an administrator can usually limit which connectors users may turn on and whether memories are kept. Turn off what you have not vetted, and confirm the exact settings in your vendor's current documentation.
4. Review the BAAs, including the companies behind them. Confirm a signed BAA for each AI vendor, the services it names, and how its subcontractors are bound. Ask your billing, transcription, IT, and answering services in writing what AI tools they use with your PHI.
5. Train, and remind. A short session on the one-page rule, with the Dana picture: pasted once, sent every turn. Document it, then add a recurring reminder that names the tools.

If you can only do one this week, make the list. Everything else depends on knowing what is switched on.

The tool is new. The program is not.

Your risk analysis lists the EHR. It probably does not list the chat window, the memories inside it, or the connector someone turned on to save time. Nobody chose that gap. It opened while everyone was busy.

That is what a gap scan and a real risk analysis are for. The gap scan shows where your program stands on the areas an auditor checks first. A risk analysis with our team puts your AI tools, their connectors, and the vendors who use them in the same record as your policies, training, and BAAs. The next time a Dana or a Marcus finds a shortcut, the rule is already written.

We have done this work since 2010, across 500+ healthcare organizations, with a 100 percent audit success rate. Audit-ready in 60 days, or we keep working at no additional cost until you are.

Get My Audit-Readiness Score, or Talk to a Specialist about a risk analysis that includes the AI tools already in use across your organization and your vendors.