Act as an AI application security engineer. Build The AI Prompt Injection Boundary Tester, a defensive testing workbench for applications that combine user instructions, retrieved documents, and tool access.
Allow developers to define a simulated system policy, user request, untrusted document, available tools, and permitted actions.
The application must:
Distinguish trusted instructions from untrusted content in the test configuration.
Generate safe test cases involving malicious instructions embedded in documents, emails, and retrieved text.
Simulate attempts to override policy, disclose protected data, or trigger unauthorized tool actions.
Evaluate whether the simulated agent follows the configured permission boundaries.
Identify cases where untrusted content is incorrectly treated as an instruction.
Test least-privilege tool access and explicit approval requirements.
Produce a pass/fail matrix with the evidence supporting each result.
Suggest mitigations and generate regression tests.
Export test cases and results in JSON.
Use a simulated agent and synthetic confidential data; do not connect to real accounts or execute destructive operations. Explain that passing the included tests does not establish that an AI system is secure against every attack. Deliver a usable testing interface with repeatable scenarios and clear remediation guidance.
0 Comments