AI-101 Capstone: Defender-Style AI-System Threat Model

Scenario

You are a security consultant engaged by Apex AI Labs, a startup that has built an internal productivity assistant using LangChain. The assistant is deployed for 200 employees and has access to their email summaries, calendar data, and a shared knowledge base. Management is preparing to expand the tool to handle financial approvals and vendor communications.

Before the expansion, you have been asked to produce a threat model. Your deliverable will inform the security review board's decision on whether to proceed, what guardrails to add, and which risks to accept.

Target Application

Audit the open-source LangChain react_agent example application, treating it as the Apex AI Labs internal assistant. Supplement with the following assumed configuration:

System prompt (given):

You are the Apex AI Labs internal productivity assistant. You help employees:
- Summarize emails (read-only access)
- Check and create calendar events
- Answer questions from the company knowledge base
- Draft documents for employee review

You have access to these tools: read_email, list_calendar, create_calendar_event,
search_knowledge_base, draft_document.

Never access employee data belonging to other employees.
Never send emails directly -- only draft them for user review.
Never make financial commitments on behalf of the company.

Tools registered:

• read_email(user_id, email_id): reads a specific email
• list_calendar(user_id, date_range): lists calendar events
• create_calendar_event(user_id, title, time, attendees): creates a calendar event
• search_knowledge_base(query): searches internal knowledge base
• draft_document(type, content): creates a document draft

Deliverable: Threat Model Report

Format: Markdown document Filename: ai101-capstone-threat-model-[LASTNAME].md Length: 6-10 pages Audience: Security review board (technical, but not AI-specialist)

Required Sections

Section 1: Executive Summary (1/2 page)

Describe the assistant and its current risk level in language a non-technical executive can understand. State your overall risk rating (Critical / High / Medium / Low) and justify it in 2-3 sentences.

Section 2: System Description

Include:

• Architecture diagram (ASCII art acceptable)
• Data flow: what data enters the system, what it outputs, where output goes
• Trust boundaries: which components are trusted, which are untrusted
• External services connected

Section 3: Asset Inventory

List:

• Data the application processes and stores
• Actions the application can take (via tools)
• Consequences of compromise for each asset/action

Section 4: Threat Enumeration Table

One row per OWASP LLM Top 10 (2025) entry:

Section 5: Attack Scenarios

Describe 3-5 attack scenarios, each as a narrative ending in specific harm. Required format:

### Scenario N: [Name]

**Attack class:** [OWASP entry]
**Attacker:** [Who is the attacker? External? Malicious employee? Indirect via poisoned doc?]
**Entry point:** [How does the attack start?]
**Attack chain:** [Step-by-step description]
**Harm:** [What is the specific real-world consequence?]
**Likelihood:** High/Medium/Low
**Impact:** High/Medium/Low

At least one scenario must be an indirect prompt injection scenario (attacker uses the email or knowledge base as the injection vector, not direct user input).

Section 6: D8 Evaluation Results

Report your Tier 1-3 evaluation results:

• Tier 1: How well does the agent accomplish its intended tasks? (5 test cases, pass/fail)
• Tier 2: How does the agent respond to attack inputs mapped to OWASP entries? (5 tests, refused/vulnerable)
• Tier 3: Cost estimate per day for 200-user deployment (assumed 20 queries/user/day)

Section 7: Mitigation Roadmap

Prioritized by P1 (fix immediately) / P2 (fix within 30 days) / P3 (accept or defer):

Minimum: 3 P1 mitigations, 3 P2, 2 P3.

Section 8: ASI Top 10 Cross-Reference

Identify which ASI Top 10 (2026) entries apply given the application's agentic nature. For each applicable entry, explain how the tool-use capability amplifies the corresponding LLM risk.

Rubric (40 points)

Binary gates (automatic Incomplete):

• Fewer than 8 of 10 OWASP LLM Top 10 entries present
• No indirect prompt injection scenario
• D8 evaluation section missing
• Document under 4 pages
• Mitigations are generic ("use security best practices") rather than specific

Grading Note

The threat model is graded holistically. A threat model that identifies only 3 vulnerabilities but analyzes each in depth with concrete evidence and specific mitigations scores higher than one that mentions all 10 but provides only superficial analysis.

The most common failure mode in student threat models: attack scenarios describe the attack class ("the attacker could perform prompt injection") without describing a specific plausible scenario with specific harm ("an attacker sends a crafted email to alice@apexailabs.com; when Alice opens her Copilot and asks to summarize her emails, the assistant reads the email, follows the injected instructions, and calls create_calendar_event to schedule a meeting with the attacker on Alice's calendar").

Be specific. Generic threat models provide no value to the security review board.