Field notes. John Arndt, Soxoa. Published .
Before a business puts confidential documents into an AI pilot, someone should be able to draw where those documents go. That drawing needs more than a box labeled private model. Include the upload, document parser, search index, model, application, logs, backups, and any service that receives a request.
My starting rule is simple: approve a defined use of defined information. Private AI should be a set of inspectable decisions about access and processing, with named owners. A product label or reassuring answer from a chatbot cannot make those decisions for your organization.
Define what private means for this task
Write the requirement in ordinary language. For example: only the authorized proposal team may search these approved documents; processing stays in the approved environment; drafts require review before sharing. Then record exceptions, prohibited data, and who can approve changes.
Local execution is one deployment choice. A dedicated environment or an approved hosted service may be another, depending on your requirements. Local AI vs cloud AI covers that choice. This checklist applies to either because the application around the model can expose information independently of where inference runs.
Six checks to complete before real data enters
Ask the builder and the people responsible for your systems to complete these together. An unanswered item becomes a pilot limitation, not an assumption that everything is covered.
Data scope and authority
List the source systems, document categories, and intended use. Confirm who is authorized to provide the information and which records must be excluded. Start with synthetic or approved examples while the design is unresolved. Give each document collection an owner who can approve additions and removal.
Every processing destination
Trace uploads, parsing or OCR, embeddings used for search, model requests, monitoring, support tools, and exports. Record the operator and location of each step, plus any subprocessors or external dependencies that need review. Inspect normal traffic and failure paths; an automatic fallback can change the destination.
Identity and document permissions
Define who can sign in, administer the system, search a collection, and open a cited source. Preserve document restrictions in retrieval rather than depending on the model to decide what a user may see. Test with two users who have different permissions, including a user whose access has just been removed.
Retention and deletion
Set rules for prompts, answers, source files, extracted text, search indexes, logs, and backups. Ask what deletion removes immediately, what remains until a backup expires, and who can demonstrate the difference. Treat a search index derived from confidential documents as part of the data inventory.
Tools and human authority
Write down which systems the AI may read and which actions it may propose or perform. Separate a drafted reply from permission to send it. Keep credentials outside the prompt and restrict tools to the scope they need. Define who reviews consequential output and what happens when that person is unavailable.
Operations and change control
Name the people responsible for updates, monitoring, incident handling, access reviews, and restoration. Record the tested model and application versions. Agree which changes require the acceptance cases to run again. The handoff should let your operator stop the service and return the workflow to a known manual path.
Check the settings, then check the behavior
Ollama's documentation distinguishes local processing from cloud features and describes a local-only setting. A deployment using it should record that configuration and verify the surrounding application. A local model does not make an external document parser or cloud backup local.
Also test hostile or misleading source content. OWASP's prompt injection guidance describes how instructions in external material can influence a model. A document is evidence to process; it should not grant permission to reveal another collection, send a message, or change an approval rule. Local hosting does not remove that issue.
For hosted components, have the appropriate people review the actual service terms and controls for the intended account and configuration. Ask separately about training use, storage, retention, access, and deletion. One favorable answer does not establish the others.
An illustrative pilot acceptance record
Suppose a proposal coordinator wants a draft answer drawn from approved past material. The coordinator must verify every claim before the proposal lead approves external use. The system may find and draft; it cannot make new commitments on the organization's behalf.
Agree these example checks before running the pilot, then preserve the inputs, observed behavior, and reviewer's decision. A failed check needs a change or a narrower scope before the affected use proceeds.
- Access boundary
- A user outside the proposal team cannot retrieve restricted source text or get it restated in an answer. Revoked access is tested again after the documented synchronization interval.
- Data route
- The operator records the destinations used during an ordinary request, a parser failure, and a model outage. No unapproved fallback processes the documents.
- Review boundary
- Unsupported claims and conflicting sources are visible. The draft cannot send itself or update an approved statement without the required human decision.
- Removal and recovery
- The owner demonstrates removal from the active collection, explains backup retention, and restores the approved configuration without opening broader access.
Leave with evidence and an owner
A useful checklist ends in a decision: proceed within a documented boundary, resolve specified gaps, or stop. Keep the data map, configuration record, acceptance results, review instructions, and owner list together. They support ongoing operation; they do not certify security or compliance.
An AI Build Sprint can make those deliverables part of the scope. Bring one workflow and its data owner to the private AI discussion, and use our AI evaluation buying guide to ask what the builder will prove before the pilot expands.
Common questions
- Does a private AI system need access controls?
- Yes. Define user and administrator access, preserve source-document permissions during retrieval, and test users with different rights. A private deployment still needs boundaries between people and collections.
- Does a no-training promise mean no data is stored?
- No. Training use and storage are separate questions. Review the relevant terms and configuration for prompt retention, application logs, support access, backups, and deletion as well.
- Who approves a private AI pilot?
- Your organization names the workflow owner and the people responsible for data, systems, and any required legal or compliance review. The builder supplies evidence and explains limitations; the organization retains the approval decision.