User Email Submissions (FN) by Submission Type

Hunt Hypothesis

Adversaries may exploit user email submission data to refine phishing simulations and evade detection by identifying false negatives. SOC teams should proactively hunt for this behavior to uncover potential adversary adaptation and improve phishing detection effectiveness in Azure Sentinel.

KQL Query

CloudAppEvents
| where ActionType contains "UserSubmission"
| extend SubmissionType = tostring((parse_json(RawEventData)).SubmissionType),SubmissionContentType=tostring((parse_json(RawEventData)).SubmissionContentType)
| extend User_SubmissionType=
iff(SubmissionType == "1" and ActionType == "UserSubmission" and SubmissionContentType=="Mail","User_Phish_FN",
iff(SubmissionType == "0" and ActionType == "UserSubmission" and SubmissionContentType=="Mail","User_Spam_FN",
iff(ActionType == "AttackSimUserSubmission" and SubmissionContentType=="Mail","User_AttackSim_Submission",
"Other")))
| where User_SubmissionType!="Other"
| summarize count() by User_SubmissionType
| render piechart

Analytic Rule Definition

id: 0b2cbdf4-12e4-46e9-a8f6-99e559583cd7
name: User Email Submissions (FN) by Submission Type
description: |
  This query visualises the total ammount of user false negative submission by submission type, including the phish simulations reported emails
description-detailed: |
  This query visualises the total ammount of user false negative submission by submission type, including the phish simulations reported emails
  Query is also included as part of the Defender for Office 365 solution in Sentinel: https://techcommunity.microsoft.com/blog/microsoftdefenderforoffice365blog/part-2-build-custom-email-security-reports-and-dashboards-with-workbooks-in-micr/4411303
requiredDataConnectors:
- connectorId: MicrosoftThreatProtection
  dataTypes:
  - CloudAppEvents
tactics:
  - InitialAccess
relevantTechniques:
  - T1566
query: |
  CloudAppEvents
  | where ActionType contains "UserSubmission"
  | extend SubmissionType = tostring((parse_json(RawEventData)).SubmissionType),SubmissionContentType=tostring((parse_json(RawEventData)).SubmissionContentType)
  | extend User_SubmissionType=
  iff(SubmissionType == "1" and ActionType == "UserSubmission" and SubmissionContentType=="Mail","User_Phish_FN",
  iff(SubmissionType == "0" and ActionType == "UserSubmission" and SubmissionContentType=="Mail","User_Spam_FN",
  iff(ActionType == "AttackSimUserSubmission" and SubmissionContentType=="Mail","User_AttackSim_Submission",
  "Other")))
  | where User_SubmissionType!="Other"
  | summarize count() by User_SubmissionType
  | render piechart
version: 1.0.0

Required Data Sources

MITRE ATT&CK Context

• Tactic: InitialAccess
• Technique: T1566 — T1566

References

• [Source Query](https://github.com/Azure/Azure-Sentinel/blob/main/Hunting Queries/Microsoft 365 Defender/Email and Collaboration Queries/Submissions/User Submissions by Submission Type.yaml)

False Positive Guidance

• Scenario: Phishing Simulation Email Reports
  Description: When users complete phishing simulations, their submitted email addresses are often reported as false negatives by the security tool.
  Filter/Exclusion: Exclude emails from known phishing simulation tools like KnowBe4 or PhishMe by checking the email_subject or submission_type field for keywords like “simulation” or “training”.

• Scenario: Scheduled System Job Email Submissions
  Description: Automated system jobs (e.g., Microsoft Exchange Online or AWS SES) may submit emails as part of routine operations, which could be flagged as false negatives.
  Filter/Exclusion: Filter out submissions from system accounts or services by checking the sender_email field against known service accounts or using the source_ip to identify internal infrastructure.

• Scenario: Admin Task Email Confirmations
  Description: Administrators may submit emails as part of routine tasks, such as confirming user account changes or password resets, which could be misclassified as false negatives.
  Filter/Exclusion: Exclude emails from admin roles by checking the user_role or user_group field for roles like “Admin”, “IT”, or “Security”.

• Scenario: Email Submission from Third-Party Tools
  Description: Third-party tools like Cisco Secure Email Gateway or Microsoft Defender for Office 365 may submit emails for analysis, which could be flagged as false negatives.
  Filter/Exclusion: Exclude submissions from known third-party tools by checking the tool_name or submission_source field for specific tool identifiers.

• Scenario: False Negative Reporting from Security Tools
  Description: Some security tools may report false negatives as part of their own internal reporting mechanisms, leading to redundant or misleading data.
  Filter/Exclusion: Filter out internal reporting emails by checking the