Can several people use a private AI server at the same time?
Yes. The model runner answers several questions side by side on one graphics card. Each question in progress takes a share of the card’s memory for the text it is reading, and each answer comes a little slower when the card is busy. A small office is well served by one card; a large one needs more memory or a second card.
How it works
The model is loaded once. The runner then keeps several slots, each holding one conversation. With llama.cpp:
llama-server -m model.gguf -c 65536 -np 4 -ngl 99
-np 4 opens four slots. The context given by -c is shared among them, so this example gives each person about 16,000 tokens, roughly 35 pages in view at a time.
The trade
- More slots means more people answered at once and less text each can have in view.
- Fewer slots means longer documents per question and a short wait when everyone asks together.
When all slots are busy, the next question waits in line. For a question that takes six seconds, the wait is rarely noticed.
What staff notice
With the card answering one person, text appears faster than anyone reads. With several at once, the card’s total output goes up and each person’s share goes down. In an office, people rarely ask in the same few seconds, so most questions get the whole card.
Sizing
| Office | Setup |
|---|---|
| Up to about five people | One 16 or 24 GB card, two to four slots |
| Ten to twenty people | 24 GB or more, four to eight slots, or two cards |
| Long documents for everyone | More memory rather than more slots: 48 GB or two cards |
These are starting points. The honest way to size it is to count how many questions your office asks in its busiest ten minutes during a trial.
Keeping people’s work apart
Slots share the model, not the conversations. One person’s documents and questions are never placed in another person’s slot. Who may ask about which files is decided by accounts and permissions, outside the model: see how to secure a private AI server.
Heavy jobs
Reading a whole archive, or character recognition on thousands of scans, can fill the card for hours. Run those at night so the daytime slots stay free.
Questions and answers
- How many people can use one private AI server?
- A single 16 or 24 GB graphics card serves an office of about five comfortably. Larger offices need more graphics memory or a second card.
- Does a private AI get slower with more users?
- Each answer is somewhat slower when several questions run at once, because they share the card. Since people rarely ask at the same moment, most questions run at full speed.
- Can users see each other’s questions on a shared AI server?
- No. Each conversation is kept in its own slot, and access to documents is controlled by each person’s account.