AgentShield

AI Agent Security Gateway & Red-Team Platform

"An AI agent should never be trusted simply because the model says an action is safe. Every sensitive action must pass identity, policy, data, tool and risk controls before execution."

Observe. Evaluate. Authorize. Stop unsafe AI-agent actions.

---

What this solves

Modern AI agents can autonomously access sensitive data, invoke tools, call APIs, and act on a user's behalf. Traditional application-security controls (auth, WAFs, RBAC at the API layer) don't fully address AI-agent-specific risks:

• Prompt injection — instructions hidden in user input or retrieved content that try to override the agent's behavior.
• Context / memory poisoning — the same kind of attack, but planted in the agent's own memory/context store instead of the live prompt.
• Malicious or unauthorized tool invocation — the agent calling a tool it shouldn't, or with parameters it shouldn't.
• Identity misuse & privilege escalation — a low-privilege agent talked into acting like an admin.
• Excessive agency — an agent chaining several individually-plausible steps into something no human approved.
• Sensitive-data exfiltration — customer data, secrets or tokens flowing to an external destination.

AgentShield is a security gateway that sits between an AI agent and its tools/data. Every proposed action is inspected, scored, and either allowed, blocked, sent for human approval, or rejected outright (if the agent has been killed) — before it ever executes. A companion red-team engine runs controlled attacks against the same pipeline and measures how well it's actually working.

Why AI agents need additional security controls

A conventional API gateway checks "is this caller allowed to hit this endpoint." An agent gateway has to also ask: is the content that produced this request trustworthy, does this specific role/agent get to do this specific thing with this specific data, and should a human see this before it runs. That's what the pipeline in backend/app/gateway/core.py does on every single request — deterministically, not by asking an LLM if it's fine.

Architecture

flowchart TB
    User[User] --> Agent[AI Agent]
    Agent -->|proposed tool call| Gateway

    subgraph Gateway[AgentShield Security Gateway]
        direction TB
        G1[1. Identity & RBAC] --> G2[2. DLP / Sensitive-Data Scan]
        G2 --> G3[3. Prompt-Injection & Memory Check]
        G3 --> G4[4. Tool Guard / Destination Check]
        G4 --> G5[5. Rate Limit & Kill Switch]
        G5 --> G6[6. Risk Engine]
        G6 --> G7[7. Policy Engine]
        G7 --> G8[8. Audit Logger]
    end

    Gateway -->|ALLOW| Tools[Local Demo Tools]
    Gateway -->|REQUIRE_APPROVAL| Approvals[Approval Queue]
    Gateway -->|BLOCK / TERMINATED| Denied[Denied — nothing executes]
    Approvals -->|approved| Tools

    Tools --> Customer[(Customer DB - synthetic)]
    Tools --> Email[Email Tool - mock outbox]
    Tools --> File[File Tool - in-memory sandbox]
    Tools --> Search[Knowledge Search]
    Tools --> HTTP[HTTP Mock Tool - no real network]

    RedTeam[Red-Team Engine] -->|synthetic attacks| Gateway
    Gateway --> Dashboard[Observability Dashboard]

Loading

See docs/architecture.md for the full breakdown and docs/security-controls.md for what each stage actually checks.

Features

• Security gateway — identity/RBAC, DLP, prompt-injection detection, destination classification, deterministic risk scoring, human-readable policy engine, rate limiting, and a kill switch — all evaluated on every request, all logged.
• Explainable decisions — every decision (ALLOW / BLOCK / REQUIRE_APPROVAL / TERMINATED) ships with a risk score, a risk-factor breakdown, matched/unmatched policies, and plain-English reasons.
• Human approval workflow — REQUIRE_APPROVAL actions sit in a queue until a human approves or denies them; approval can never bypass a hard BLOCK.
• Emergency kill switch — terminate an agent instantly; every subsequent call from it is denied until it's resumed.
• Red-team engine — 15 controlled attack/baseline tests (12 malicious across all required categories + a 4-step attack chain + 3 legitimate baselines) run straight through the same gateway pipeline a real agent would use, with automatically computed detection coverage and false-positive rate.
• Observability dashboard — live decisions, risk distribution, attack categories, top blocked tools, top triggered policies, latency distribution, and a full request-investigation timeline per event.
• Threat model & reports — an interactive threat model and three downloadable reports (security assessment, red-team evaluation, threat model), computed from live data, in JSON and Markdown.
• Works with zero API keys — the LLM provider interface defaults to a deterministic MockLLMProvider; the gateway's authorization decisions never depend on an LLM.

Tech stack

Layer	Choice
Backend	Python 3.12, FastAPI, Pydantic v2, SQLAlchemy 2.0, SQLite, pytest
Frontend	React 19, TypeScript, Vite, Tailwind CSS v4, Recharts, React Router, TanStack Query, Vitest
Security	Deterministic rule-based DLP & prompt-injection detection, RBAC, policy engine, in-memory rate limiting, structured audit logging
Infra	Docker, Docker Compose, Makefile

No Redis, no external DB server, and no LLM API key are required to run the full application.

Setup

Option A — Docker Compose

docker compose up --build

• Frontend: http://localhost:5173
• Backend API: http://localhost:8000 (docs at /docs)

The backend container seeds demo data automatically on startup.

Option B — local (no Docker)

macOS / Linux:

./scripts/setup.sh      # creates backend venv, installs deps, seeds the DB, npm installs frontend
./scripts/run_demo.sh   # starts both servers and runs the one-click security demo

Windows PowerShell:

cd backend
python -m venv venv
.\venv\Scripts\Activate.ps1
pip install -r requirements.txt
python ..\scripts\seed.py
deactivate

cd ..\frontend
npm install
npm run dev   # in one terminal

cd ..\backend
.\venv\Scripts\Activate.ps1
uvicorn app.main:app --reload --port 8000   # in another terminal

Then open http://localhost:5173.

Configuration

Copy .env.example to .env and adjust as needed (all values have sane defaults — nothing is required to run locally). See docs/architecture.md for what each variable does.

Demo mode

Click "Run Full Security Demo" in the top bar (or POST /api/demo/run, or ./scripts/run_demo.sh). It seeds data, runs the full red-team suite, triggers a legitimate request, an approval-required action, and a kill-switch walkthrough — then reports real numbers computed from what just happened:

Requests evaluated: 21
Blocked: 14
Allowed: 5
Approval required: 1
Attack tests: 18 (from 15 catalog entries — the attack chain expands into 4 steps)
Passed: 18
Detection coverage: 100%
Average gateway latency: ~2 ms

Red-team suite

./scripts/run_redteam.sh

or POST /api/red-team/run / the Run Full Suite button on /red-team. See docs/red-team-methodology.md for the full test catalog and how pass/fail and detection coverage are computed.

API usage

Full reference in docs/api.md; interactive docs at /docs (Swagger UI) once the backend is running. Quick example:

curl -X POST http://localhost:8000/api/gateway/evaluate \
  -H "Content-Type: application/json" \
  -d '{
    "agent_id": "<agent-id-from-/api/agents>",
    "tool_name": "search_customer",
    "action": "search",
    "parameters": {"query": "premium"},
    "prompt_text": "Please find customers on the premium plan."
  }'

Testing

make test              # backend + frontend
make test-backend      # pytest (63 tests: unit + integration + red-team + API smoke)
make test-frontend     # vitest (component tests)

Metrics & their formulas

See docs/evaluation.md for full definitions. Summary:

• Detection coverage = blocked malicious tests / total malicious tests × 100
• False-positive rate = legitimate baseline tests incorrectly blocked / total legitimate tests × 100
• Gateway latency = wall-clock time from gateway entry to decision (measured per request)
• Policy effectiveness = requests a given policy's condition matched, tracked as trigger_count per policy

Security limitations

• Prompt-injection and DLP detection are rule/pattern-based. A sufficiently novel phrasing can evade them — by design, the system never lets an LLM make the authorization call instead, but pattern-based detection has real false-negative risk that a production system would want to layer additional signals on top of.
• All tools operate on local synthetic data and in-memory mock stores. No real system, credential, or customer data is ever involved, and demo tools have no real network access.
• Rate limiting and behavior tracking are in-process, per-instance state — a multi-instance deployment would need a shared store (e.g. Redis).
• The LLM provider is a deterministic mock; nothing in this repo calls a real language model.

Future improvements

See the Recommendations section of the generated Security and Red-Team reports (/api/reports/security, /api/reports/red-team), and docs/security-controls.md.

Project docs

• docs/architecture.md
• docs/threat-model.md
• docs/security-controls.md
• docs/api.md
• docs/red-team-methodology.md
• docs/evaluation.md
• docs/demo-script.md
• docs/resume-bullets.md
• PROJECT_OVERVIEW.md
• FINAL_VERIFICATION.md

License

MIT — synthetic demo project for security research and portfolio/interview use.