A document assistant needs to read business material. Sometimes that material also contains instructions aimed at the assistant itself: change the answer, ignore other evidence, or take an unrelated action.
This is called indirect prompt injection. The attempted instruction arrives inside something the assistant was asked to read, rather than from the person directing the work. A retrieved passage can be relevant to a question and still contain an attempt to redirect the system.
We recommend treating this as a design and testing problem before giving an assistant access to a working document collection. A good demonstration with ordinary files does not show how it will handle a hostile one.
Start with a concrete failure
Consider a hypothetical parts distributor using an assistant to compare supplier documents. A staff member asks which quoted items meet a required specification.
One supplier’s document includes a passage telling an automated reviewer to rank that supplier first and omit any missing specifications. The assistant should evaluate the quote against the staff member’s criteria. It should not let the quote rewrite those criteria.
The failure does not require an email to be sent or a record to be changed. A misleading comparison could still affect a purchasing decision. For this example, we would check whether the answer identifies missing evidence and whether its conclusions remain tied to the actual specifications.
Separate evidence from instructions
Document text should enter the system as source material, clearly separated from the application’s instructions. That separation helps communicate the intended role of each passage, but labels and a stronger prompt are not sufficient enforcement.
Keep access checks in the application. A document should not grant access to another collection, authorize a tool, or change what a user may see. If the assistant only needs to compare quotes, it should not receive tools for placing orders.
Our RAG AI systems work includes retrieval, prompt design, evaluation, and the interface around the answer. For this use case, we recommend reviewing those parts together: what reaches the model, what it can do with that material, and what the person reviewing the comparison can inspect.
Decide what happens after a warning
A screening service can flag suspected document attacks. Its result still needs a defined response in the application, such as excluding the affected passage or sending the document for review.
A clean screening result does not establish that a document is accurate or that it came from the claimed sender. Attack detection, source verification, and answer checking address different questions. Keep those checks separate rather than letting one green indicator stand for all three.
In our supplier example, a warning could leave the comparison incomplete until someone reviews the document. We would prefer that visible gap to a confident recommendation that silently relies on disputed material. The screen should explain which source needs attention without presenting its instructions as something the reviewer should follow.
Test ordinary documents alongside hostile ones
Use a separate test collection with synthetic business details and no live actions. Write down the expected result before running the assistant.
For the distributor example, a small test set could include:
- A normal quote with all required specifications.
- A quote missing one required specification.
- A quote containing an instruction to hide that missing specification.
- A staff training document discussing prompt injection as a topic.
- A normal quote containing legitimate installation instructions.
The last two matter because documents often contain instructions that are simply part of their subject matter. The assistant should still be able to summarize an installation procedure without treating it as permission to operate equipment.
Check both the final answer and the evidence shown beside it. Record whether the assistant followed an unrelated instruction, omitted an important limitation, or rejected an ordinary document unnecessarily. Repeat these checks when changing the model or the way passages are supplied.
Keep the first version narrow
Research on prompt injection shows that improved defenses do not eliminate the problem. Giving an assistant more ways to act increases what a successful redirection could affect.
For an initial comparison tool, we would keep purchasing outside the assistant and make the original quotes available to the reviewer. If the workflow later needs actions, define those permissions separately rather than inheriting them from the document-reading feature.
Not every document task needs generated answers. A searchable folder and a comparison form may be enough when the collection is small and the criteria are fixed. If staff need help interpreting varied documents, talk with us about a limited workflow and the failures it needs to withstand.