How to verify that an AI is really private
Unplug the network cable and ask it a question. If it still answers, the model runs on the machine. Then plug it back in and confirm the firewall blocks all outgoing traffic, and watch the network for any connection leaving the office while it works. A private system passes all three.
1. The cable test
Disconnect the server from the internet and use it from a computer on the same switch. A system that depends on a cloud service stops working. This is the one test a non-technical person can run and trust.
2. The firewall rules
The server should refuse outgoing connections except to the office network. On Linux with UFW:
sudo ufw status verbose
Look for deny (outgoing) as the default, and only your office network among the allowed destinations.
3. Watch the traffic
While someone uses the assistant, record any packet going anywhere except the office network:
sudo tcpdump -n -i any 'not (dst net 192.168.0.0/16 or dst net 10.0.0.0/8 or dst net 127.0.0.0/8)'
On a private system this stays empty apart from blocked attempts by the operating system to check for updates. For a stronger check, do the same on the office router, which the server cannot influence.
4. What is listening
sudo ss -ltnp
This lists every program accepting connections. Expect the chat screen and the model runner, and nothing you cannot name. The model runner should listen on the local address only, behind the chat screen.
5. Ask where each part runs
A document assistant has several parts. Ask the builder, for each one, where it runs:
- The main model.
- The embedding model used for search.
- Character recognition for scanned pages.
- Speech-to-text, if it takes dictation.
- Usage statistics and error reports.
- Software updates.
“On the server” should be the answer to all but the last, and updates should be something you start by hand. A common weak spot is a product that runs the model locally and still sends search or scanning to a cloud service.
What “private” does not cover
These checks show nothing leaves the building. They do not protect against someone inside reading files they should not. That needs accounts, access by role and a record of who asked what.
Questions and answers
- How can I tell if an AI tool sends my data to the cloud?
- Disconnect it from the internet and see if it still answers. Then check that the firewall blocks outgoing traffic and watch the network for connections leaving the office while it is in use.
- Is an AI that runs locally always private?
- No. Some products run the model locally but send search, scanning, statistics or error reports to outside services. Ask where each part runs.
- Does a private AI server ever need the internet?
- Only to download updates, and only when you choose to. It answers questions with no connection.