Admin Submissions by DetectionMethod (Phish FP)

Hunt Hypothesis

Admins submitting emails flagged as phish false positives may indicate adversarial attempts to bypass email filtering and manipulate internal users, suggesting potential phishing campaigns or insider threats. SOC teams should proactively hunt for this behavior in Azure Sentinel to identify and mitigate early-stage phishing attacks that evade standard detection mechanisms.

KQL Query

CloudAppEvents
| where ActionType == "AdminSubmissionSubmitted"
| extend SubmissionType = tostring((parse_json(RawEventData)).SubmissionType),SubmissionContentType=tostring((parse_json(RawEventData)).SubmissionContentType)
| extend Admin_SubmissionType=
iff(SubmissionType == "3" and ActionType == "AdminSubmissionSubmitted" and SubmissionContentType=="Mail" ,"Admin_Email_FP",
"Other"),DetectionMethod=tostring((parse_json(RawEventData)).DeliveryMessageInfo.FinalFilterVerdictControlSource),
P2SenderDomain=tostring((parse_json(RawEventData)).P2SenderDomain),NetworkMessageId=tostring((parse_json(RawEventData).ObjectId)),DetectionVerdict=tostring((parse_json(RawEventData)).DeliveryMessageInfo.FinalFilterVerdict),PolicyOverride=tostring((parse_json(RawEventData)).DeliveryMessageInfo.PolicyOverride),PolicyPolicyOverrideType=tostring((parse_json(RawEventData)).DeliveryMessageInfo.PolicySource)
| where SubmissionContentType == "Mail" and SubmissionType == "3" and DetectionVerdict =="Phish"
| summarize count() by DetectionMethod
| project DetectionMethod,Emails = count_
// | render piechart // Uncomment this line to render as a graph

Analytic Rule Definition

id: 665e50c0-f808-4cc3-a309-45bbbcc59bd0
name: Admin Submissions by DetectionMethod (Phish FP)
description: |
  This query visualises the original detection technology of emails submitted as phish false positive by admins
description-detailed: |
  This query visualises the original detection technology of emails submitted as phish false positive by admins
  Query is also included as part of the Defender for Office 365 solution in Sentinel: https://techcommunity.microsoft.com/blog/microsoftdefenderforoffice365blog/part-2-build-custom-email-security-reports-and-dashboards-with-workbooks-in-micr/4411303
requiredDataConnectors:
- connectorId: MicrosoftThreatProtection
  dataTypes:
  - CloudAppEvents
tactics:
  - InitialAccess
relevantTechniques:
  - T1566
query: |
  CloudAppEvents
  | where ActionType == "AdminSubmissionSubmitted"
  | extend SubmissionType = tostring((parse_json(RawEventData)).SubmissionType),SubmissionContentType=tostring((parse_json(RawEventData)).SubmissionContentType)
  | extend Admin_SubmissionType=
  iff(SubmissionType == "3" and ActionType == "AdminSubmissionSubmitted" and SubmissionContentType=="Mail" ,"Admin_Email_FP",
  "Other"),DetectionMethod=tostring((parse_json(RawEventData)).DeliveryMessageInfo.FinalFilterVerdictControlSource),
  P2SenderDomain=tostring((parse_json(RawEventData)).P2SenderDomain),NetworkMessageId=tostring((parse_json(RawEventData).ObjectId)),DetectionVerdict=tostring((parse_json(RawEventData)).DeliveryMessageInfo.FinalFilterVerdict),PolicyOverride=tostring((parse_json(RawEventData)).DeliveryMessageInfo.PolicyOverride),PolicyPolicyOverrideType=tostring((parse_json(RawEventData)).DeliveryMessageInfo.PolicySource)
  | where SubmissionContentType == "Mail" and SubmissionType == "3" and DetectionVerdict =="Phish"
  | summarize count() by DetectionMethod
  | project DetectionMethod,Emails = count_
  // | render piechart // Uncomment this line to render as a graph
version: 1.0.0

Required Data Sources

MITRE ATT&CK Context

• Tactic: InitialAccess
• Technique: T1566 — T1566

References

• [Source Query](https://github.com/Azure/Azure-Sentinel/blob/main/Hunting Queries/Microsoft 365 Defender/Email and Collaboration Queries/Submissions/Admin Submissions by Detection Method - Phish FP.yaml)

False Positive Guidance

• Scenario: Admin submits a phishing test email via Cisco Email Security Appliance (ESA) for internal training purposes.
  Filter/Exclusion: email_subject:*Phishing Test* OR sender_email:*[email protected]*

• Scenario: Microsoft Defender for Office 365 automatically submits a benign email as a false positive during its learning process.
  Filter/Exclusion: detection_method:*Microsoft Defender for Office 365* OR submission_type:*automated learning*

• Scenario: An admin manually submits a suspicious email via Microsoft Exchange Online Protection (EOP) for investigation.
  Filter/Exclusion: submission_source:*Exchange Online Protection* OR admin_email:*[email protected]*

• Scenario: A scheduled job runs daily to submit test emails for security testing using Mailgun or SendGrid.
  Filter/Exclusion: job_name:*security_test_job* OR email_sender:*[email protected]*

• Scenario: An admin uses Microsoft Outlook with Exchange Online to submit a legitimate email as a false positive during a manual review.
  Filter/Exclusion: email_client:*Outlook* OR admin_email:*[email protected]*