A quick note on Logic App and Detection Rule validation in Microsoft Sentinel

A lot of the time when I’ve been building logic apps it becomes difficult to test any and all conditions in the logic application’s flow when you’re relying on Sentinel’s Incident / Alert trigger to give it all the data it needs to trigger all possible scenarios.

Rather than sit around and wait for the stars to align (if ever), I found a technique to falsify the data and validate my logic apps functionality by replacing the Sentinel trigger with an ‘Azure Monitor’ KQL query.

My technique is to use a KQL query with the datatable operator and self-made custom logs to mimic what the analytic rule is looking for in the wild.

The analytic rule in Sentinel will look something like this:

WindowsLogs_CL
| where Event_ID = "4740" // Lockout Event ID

For this hypothetical scenario, when this analytic rule catches a 4740 event, I have an automation rule that triggers my logic app.

My logic app wants to sends emails to 2 different teams - one to the team who deals with Service Account lockouts and the other, to the team who want regular User Account lockout notifications.

So my Logic App has a condition check in it that determines whether or not the account is in fact a service account or not — If UserName contains “Service”, then email team 1, if not email team 2. (Forgetting for a moment that ‘contains’ is inefficient).

Now typically if I wanted to test this logic app, I would publish it then sit back and wait for an account lockout to occur and wait for the error or success message. Or if you’re so inclined, proactively lock yourself and another service account out to test the logic app’s functionality. Either or, these options are annoying, time consuming and in some cases impossible.

For those like me who don’t want to wait around or self-generate lockouts— to test the logic app’s functionality, replace the ‘Microsoft Incident / Alert Trigger’ with an ‘Azure Monitor’ KQL query that looks something like:

 let dt = 
datatable(Date:datetime, Event_ID:string, UserName:string, HostName:string)
[
    datetime(2024-01-11), "4740", "JacobLummus", "ABC"),
    datetime(2024-01-11), "4740", "SQL_ServiceAccount", "XYZ")
]; // dt is equivalent to my hypopthetical 'WindowsLogs_CL" table in Sentinel
dt 
| where Event_ID == "4740"

Now when I want to test the logic app by running it manually, I’m actually running it off my custom ‘dt’ table rather than waiting for the above analytic rule to fire off. My custom ‘dt’ table is designed to be a replica of the WindowsLogs_CL table and I’ve made 2 custom logs for it. Followed by a copy of the analytic rule below to read both custom logs. Provided the logic app is designed to include a for loop, these 2 logs will test both conditional statements in the Logic App.

This is a pretty simple alternative to sitting around and waiting for my Sentinel Analytic Rule to detect both conditions in the wild.

Then putting this Logic App into ‘production’ would be a matter of removing the ‘Azure Monitor’ KQL query at the start of the Logic App and replacing it back with your Sentinel analytic rule trigger with the confidence that everything works as it should.

I then realized I could marry this technique with functions in Sentinel for Detection Rule validation as well.

Problems arise when indicators of compromise (IoCs) are published and you want to detect those IoCs in your environment. You develop a lengthy KQL query and it’s returning no results. Best case scenario is that you don’t have any IoCs, great! Worse case though is that your detection rule logic is wrong and you’ve just generated a false negative.

When you publish a detection rule for a particularly scary scenario you want the confidence to know that it works and it can be very hard to thoroughly test when these events are not something you can generate yourself.

A more self-assuring solution to develop detection rules is to have empty replicas of your Sentinel tables, that you yourself populate with the IoCs. Your methodology then is to design and develop your query off of those fake tables where you know the particular IoCs exist.

That way, if you’ve correctly replicated the relevant table as a function and as well populated it with correctly formatted IoCs (as they would appear in your production tables) you can design and develop effective analytic rules with a lot more confidence off of the replica table.

For example, you would make and save a function with the table name DummyWindowsLogs_CL that is a direct replica of your WindowsLogs_CL table in production:

let DummyWindowsLogs_CL= 
datatable(Date:datetime, Event_ID:string, UserName:string, HostName:string)
[
    datetime(2024-01-11),
"", // Event_ID
"", // UserName
"") // HostName
];
DummyWindowsLogs_CL

Now to make a custom log would be a matter of filling in the empty strings with which ever IoCs you’re looking for and saving it as it’s own function.

Then when it comes to developing your detection rule, you would want to be querying and building off of DummyWindows_CL, where you know there is an IoC and can confidently confirm that your logic is effective in detecting it.

Putting this detection rule into production would simply be a matter of removing ‘Dummy’ from the table name and again, provided you’ve replicated the schema and IoCs correctly, you can have peace of mind that what detection rule you’ve built can confidently detect those IoCs in your environment.

Note: Be careful back testing existing detection rules with this technique. Taking an existing rule, and making a custom log off of that rule to see whether it works will ‘overfit’ the log to the detection rule. When what you’re really looking to do is create the log as it should appear and then build the detection off of that — not the other way around.