AI Agents - Missing Tools in Instructions

Hunt Hypothesis

This hunt targets adversaries exploiting misconfigured AI agents where unused or hidden tools create an attack surface for unauthorized capability execution and potential data exfiltration (T1499, T1562). Proactively hunting these discrepancies in Azure Sentinel is critical to prevent attackers from leveraging silent tool configurations to bypass instruction-based security controls and manipulate agent behavior.

KQL Query

let IdentityIdtoUPN = materialize (
    IdentityInfo
    | where isnotempty(AccountObjectId) and isnotempty(AccountUpn)
    | summarize arg_max(Timestamp, AccountUpn) by AccountObjectId
    | project AccountObjectId = tostring(AccountObjectId), AccountUpn);
let LatestAgents = materialize (
    AgentsInfo
    | summarize arg_max(Timestamp, *) by AgentId
    | where LifecycleStatus != "Deleted");
let AgentToolNames =
    LatestAgents
    | where array_length(DeclaredTools) > 0
    | mv-expand Tool = DeclaredTools
    | extend ToolName = tostring(Tool.name)
    | where isnotempty(ToolName)
    | summarize ToolNames = make_set(ToolName) by AgentId;
LatestAgents
| where isnotempty(Instructions) and Instructions != "N/A"
| join kind=inner AgentToolNames on AgentId
| mv-apply ToolName = ToolNames to typeof(string) on (
    where Instructions !has ToolName
    | summarize MissingTools = make_list(ToolName))
| where array_length(MissingTools) > 0
| extend OwnerId = tostring(Owners[0])
| join kind=leftouter IdentityIdtoUPN on $left.OwnerId == $right.AccountObjectId
| project-rename OwnerUpn = AccountUpn
| project-away AgentId1, ToolNames, OwnerId, AccountObjectId
| project-reorder CreatedDateTime, AgentId, Name, Platform, Instructions, MissingTools, OwnerUpn

Analytic Rule Definition

id: 4bfbe654-d961-4712-b2a3-f1fd7eaa9da3
name: AI Agents - Missing Tools in Instructions
description: |
  This query identifies AI agents that have tools configured which are not mentioned in their instructions.
  When tools are enabled without being explicitly referenced, the agent may use capabilities that are not clearly governed or expected, increasing the risk of unintended behavior or misuse.
  Recommended Action: Ensure all configured tools are clearly documented in the agent's instructions, including their purpose and usage constraints. Regularly review instructions and tool configurations to maintain alignment and reduce security risks.
requiredDataConnectors: []
tactics:
  - Impact
  - DefenseEvasion
relevantTechniques:
  - T1499
  - T1562
query: |
  let IdentityIdtoUPN = materialize (
      IdentityInfo
      | where isnotempty(AccountObjectId) and isnotempty(AccountUpn)
      | summarize arg_max(Timestamp, AccountUpn) by AccountObjectId
      | project AccountObjectId = tostring(AccountObjectId), AccountUpn);
  let LatestAgents = materialize (
      AgentsInfo
      | summarize arg_max(Timestamp, *) by AgentId
      | where LifecycleStatus != "Deleted");
  let AgentToolNames =
      LatestAgents
      | where array_length(DeclaredTools) > 0
      | mv-expand Tool = DeclaredTools
      | extend ToolName = tostring(Tool.name)
      | where isnotempty(ToolName)
      | summarize ToolNames = make_set(ToolName) by AgentId;
  LatestAgents
  | where isnotempty(Instructions) and Instructions != "N/A"
  | join kind=inner AgentToolNames on AgentId
  | mv-apply ToolName = ToolNames to typeof(string) on (
      where Instructions !has ToolName
      | summarize MissingTools = make_list(ToolName))
  | where array_length(MissingTools) > 0
  | extend OwnerId = tostring(Owners[0])
  | join kind=leftouter IdentityIdtoUPN on $left.OwnerId == $right.AccountObjectId
  | project-rename OwnerUpn = AccountUpn
  | project-away AgentId1, ToolNames, OwnerId, AccountObjectId
  | project-reorder CreatedDateTime, AgentId, Name, Platform, Instructions, MissingTools, OwnerUpn
entityMappings:
  - entityType: Account
    fieldMappings:
      - identifier: FullName
        columnName: OwnerUpn
  - entityType: Host
    fieldMappings:
      - identifier: HostName
        columnName: Name
version: 1.0.0

Required Data Sources

MITRE ATT&CK Context

• Tactic: Impact
• Tactic: DefenseEvasion
• Technique: T1499 — T1499
• Technique: T1562 — T1562

References

• [Source Query](https://github.com/Azure/Azure-Sentinel/blob/main/Hunting Queries/AI Agents/AgentsInfoMissingToolsInInstructions.yaml)

False Positive Guidance

Here are 5 specific false positive scenarios for the AI Agents - Missing Tools in Instructions detection rule, tailored for an enterprise environment:

• Scheduled Data Backup Agent with Pre-Configured Storage Connectors

  • Scenario: An AI agent named NightlyDBBackup is deployed to run every Sunday at 02:00 UTC. It utilizes the AWS S3 Connector and Azure Blob Storage Tool to offload logs. However, its instruction prompt focuses solely on “executing backup routines” without explicitly naming these cloud storage tools in the text description, as the tool configuration is inherited from a global template rather than defined per-agent.
  • Filter/Exclusion: Exclude agents where agent_type = 'scheduled_job' AND tool_source = 'global_template'. Additionally, filter for instructions containing keywords like “standard backup protocol” or “default storage path.”

• HR Onboarding Bot with Integrated Identity Management Tools

  • Scenario: The NewHireOnboarding agent is triggered manually by HR admins to create accounts. It automatically invokes the Okta User Provisioning Tool and Workday API Connector. The instructions provided in the prompt describe the “end-to-end onboarding workflow” but omit specific references to Okta or Workday, assuming these integrations are implicit requirements of the role rather than explicit instruction parameters.
  • Filter/Exclusion: Create a rule to whitelist agents with owner_group = 'HR-Admins' that utilize tools from the list [Okta, Workday, AD_Sync]. Alternatively, exclude cases where the tool usage count exceeds 50% of total agent interactions without instruction updates.

• IT Helpdesk Triage Agent Utilizing Knowledge Base Search

  • Scenario: The ServiceDeskTriage agent routes incoming tickets and uses the `Con