Sentinel Saturday - Are your analytics rules actually doing what you think?

Microsoft Sentinel analytics rules are often treated as a finished product. Once enabled they are assumed to be working quietly in the background, protecting the environment. In reality many rules slowly drift out of relevance, accuracy, or value without anyone noticing.

At an intermediate maturity level the question is no longer whether analytics rules exist. The real question is whether they still represent your environment, your risks, and your data reality.

Understanding run behaviour over time

A scheduled analytics rule is only as good as three things: the data it relies on, the logic it applies, and the way it is tuned. All three change over time. A simple example is a rule detecting multiple failed sign ins followed by a successful one. The logic may be sound, but changes in authentication flows, conditional access policies, or new applications can dramatically increase false positives.

A useful first step is reviewing rule output frequency. This KQL can be run in the Logs blade to understand which rules are producing incidents and how often.

SecurityIncident
| summarize IncidentCount = count() by Title
| sort by IncidentCount desc

If a rule produces dozens of incidents every week and none lead to action, it is not a detection. It is noise.

Review the query - Not just the alert

Many teams tune thresholds while ignoring the query logic itself. That is a missed opportunity and can even sometimes lead to gaps or fatigue in amongst your SOC team.

Take a common pattern such as detecting impossible travel. Instead of relying entirely on built in logic, consider validating assumptions in raw data.

SigninLogs
| where ResultType == 0
| summarize Locations = dcount(Location) by UserPrincipalName, bin(TimeGenerated, 1h)
| where Locations > 1

Ask yourself what this query assumes. Does your workforce travel? Do you use cloud hosted desktops? Do your logs reliably populate location fields?

Entity mapping - A silent angel

Entity mapping matters far more than you might think. Poor entity mapping reduces investigation speed and automation effectiveness. Many rules ship with minimal mappings that can be improved.

For example, if your rule detects suspicious activity tied to an IP address, map it explicitly. If it involves a user, ensure the correct identifier is used consistently across rules. This becomes critical when incidents are grouped. Incorrect entity mapping can cause unrelated alerts to merge or related alerts to stay separate.

Closing

When was the last time you disabled one of your analytics rules on purpose? Not because it was broken, but because it no longer served a clear detection goal. Sentinel rewards teams who treat detections as living content rather than static controls.