If you run a law firm, a medical practice, an accounting shop, or any business bound by professional confidentiality rules, you have probably had a quiet conversation with yourself about the AI tools your team is using. The tools are useful. The team likes them. But every time client information goes into one of them, something nags at you, because the data is leaving the building.
This post is for the people having that conversation. It is not about saving money on AI. It is about a structural problem with the way most AI is delivered today, and a quietly mature alternative that sits inside your office instead of inside someone else’s data center.
The Premise
Public AI services run on someone else’s computers. When your team uses one, the contents of every prompt and every document they paste in travel across the internet, get processed somewhere else, and come back. The vendor will tell you they don’t train on your data, that they have strong security, that they will sign a business associate agreement or a data processing addendum. All of that is generally true. None of it changes the underlying fact that your client’s information has, however briefly, lived on a system you do not own and cannot audit at the level a strict reading of your professional obligations would prefer.
For an unregulated business, that trade is fine. For a law firm, a medical practice, or any practice that sits inside a confidentiality regime, it isn’t really a trade you should be making at scale, no matter how reasonable the vendor sounds.
Why “The Vendor Will Sign a BAA” Isn’t a Compliance Strategy
A signed business associate agreement, or its analog in your industry, shifts liability in the event of a breach. It does not eliminate the breach. It does not change the fact that the data left your control. And it does not give you the kind of audit story that an opposing counsel, a state bar, or a regulator wants to hear when they ask exactly where privileged communications or protected health information have traveled.
The other thing a BAA doesn’t do: protect you from a future policy change at the vendor. Today’s data handling promises are not contractually permanent. A vendor can change a default, get acquired, restructure their data retention, or quietly start logging more than they used to. You will hear about it on a Wednesday in a release-notes email, and the only honest answer to “where has our client data been?” will be “we’ll have to ask them.”
For sensitive professions, that is not a defensible posture.
What Owning Your AI Changes
When the AI doing the work lives in your office, on a machine you own, the data flow gets dramatically simpler to describe and defend. The case file your associate is drafting from doesn’t travel anywhere. The patient note your nurse is summarizing doesn’t pass through anyone else’s system. The client documents your accountant is processing stay inside your own four walls.
For a law firm, that means privileged communication stays privileged in the most literal possible sense: the only entity that has ever seen the content is the firm itself. There is no third-party log to subpoena. No vendor staff who could, in some hypothetical scenario, see something they shouldn’t.
For a medical practice, HIPAA compliance stops being a question of vetting an external processor and starts being a much shorter conversation about the security of your own network. The data flow you’re documenting starts and ends inside the practice.
For accountants, financial advisors, and anyone else governed by client-confidentiality duties, the same logic applies. The audit story collapses from “we trust these vendors and here’s the paperwork that says so” to “the data doesn’t leave.”
What It Actually Looks Like
The picture most people have of running AI in-house is a server room with cooling fans and a full-time staff to babysit it. That picture is years out of date. For a small practice today, owning your AI looks like one quiet computer in a closet, plugged into a normal wall outlet, connected to your office network. Your team uses it through a normal browser or through their existing software, the same way they’d use any rented AI tool. The only difference is where the work happens.
The open-source models that run on these boxes are commercially free to use, and a properly built setup can handle the usual professional jobs: drafting routine documents, summarizing long files, transcribing voice notes, extracting data from PDFs, answering questions about your own internal knowledge, helping with research. Roughly the same set of jobs people are using public AI for today, performed against the same kinds of documents, but without the documents ever leaving the building.
When This Is Worth Considering
For very small operations using AI lightly, owning a system is overkill, and the simpler answer is to use AI sparingly or not at all on sensitive material. For larger practices where AI has become part of how the work gets done, continuing to send privileged or protected information across the public internet is a position that gets less defensible every year, regardless of what any individual vendor’s privacy policy says today.
If you’d like to talk about whether owning your AI makes sense for your practice, including an honest read on whether your usage pattern justifies it, get in touch. We will tell you straight if it doesn’t, because the worst outcome for both of us is a system you bought but didn’t need.