Microsoft has finally placed the Azure Monitor Agent (AMA) into general release. The Azure Monitor Agent has some nice features including the ability to filter event logs before they are sent to Microsoft Sentinel. In this post I will discuss the Pro's and Con's of this agent and how to XPATH filters.
Prior Microsoft Sentinel Agent
The prior agent that Microsoft leveraged to ingest data into Azure Sentinel and Log Analytics was the OMS agent. This agent was easy to install and configure but it did have limitations. The biggest limitation was the number of logs it could forward to Microsoft Sentinel and it did not have the ability to filter events.
The old agent was not flexible enough to choose what specific events to collect. For example, these are the only options to collect data from Windows machines with the old agent:
- All events - All Windows security and AppLocker events.
- Common - A standard set of events for auditing purposes. The Common event set may contain some types of events that aren't so common. This is because the main point of the Common set is to reduce the volume of events to a more manageable level, while still maintaining full audit trail capability.
- Minimal - A small set of events that might indicate potential threats. This set does not contain a full audit trail. It covers only events that might indicate a successful breach, and other important events that have very low rates of occurrence.
- None - No security or AppLocker events. (This setting is used to disable the connector.)
The events below are what the old agent collected.
Microsoft AMA Agent
Microsoft is attempting to consolidate the agents they are using to send data to Azure. The Agent they choose is the Microsoft AMA agent.
The Microsoft AMA agent is easy to install and once installed it is updated with Windows update or can be updated from the Azure ARC console.
The pro's of this agent is it allows for the ability to filter event logs before they are send to Microsoft Sentinel. This is done using XPath queries. The AMA agent only supports XPath queries for XPATH version 1.0. I would recommend reading this guide Xpath 1.0 Limitation documentation before attempting to write XPath Queries.
The con's of this agent is it can't replace all of the functions of the OMS agent. You can collect DNS query information with the old agent and the AMA does not have the ability to collect DNS information yet.
If you still want to collect DNS information and also leverage the AMA agent you will need to have both installed and set the OMS agent event log collection to none.
This article is a good guide on how to install the AMA agent.
This article provides a good overview of the Microsoft AMA agent.
XPath?
(This information is from the following Blog Post from Roberto Rodriguez. He does a great job of explaining XPATH and how to create XPATH Queries better than I could. He also has a great project call Azure Sentinel to Go, which is a great way to test Microsoft Sentinel)
XPath stands for XML (Extensible Markup Language) Path language, and it is used to explore and model XML documents as a tree of nodes. Nodes can be represented as elements, attributes, and text.
In the image below, we can see a few node examples in the XML representation of a Windows security event:
XPath Queries?
XPath queries are used to search for patterns in XML documents and leverage path expressions and predicates to find a node or filter specific nodes that contain a specific value. Wildcards such as ‘*’ and ‘@’ are used to select nodes and predicates are always embedded in square brackets “[]”.
Matching any element node with ‘*’
Using our previous Windows Security event XML example, we can process Windows Security events using the wildcard ‘*’ at the `Element` node level.
The example below walks through two ‘Element’ nodes to get to the ‘Text’ node of value ‘4688’.
You can test this basic ‘XPath’ query via PowerShell.
- Open a PowerShell console as ‘Administrator’.
- Use the Get-WinEvent command to pass the XPath query.
- Use the ‘Logname’ parameter to define what event channel to run the query against.
- Use the ‘FilterXPath’ parameter to set the XPath query.
Get-WinEvent -LogName Security -FilterXPath '*[System[EventID=4688]]
Matching any attribute node with ‘@’
As shown before, ‘Element’ nodes can contain ‘Attributes’ and we can use the wildcard ‘@’ to search for ‘Text’ nodes at the ‘Attribute’ node level. The example below extends the previous one and adds a filter to search for a specific ‘Attribute’ node that contains the following text: 'C:\Windows\System32\cmd.exe’.
Once again, you can test the XPath query via PowerShell as Administrator.
$XPathQuery = "*[System[EventID=4688]] and *[EventData[Data[@Name='ParentProcessName']='C:\Windows\System32\cmd.exe']]"
Get-WinEvent -LogName Security -FilterXPath $XPathQuery
Can I Use XPath Queries in Event Viewer?
Every time you add a filter through the Event Viewer UI, you can also get to the XPath query representation of the filter. The XPath query is part of a QueryList node which allows you to define and run multiple queries at once.
We can take our previous example where we searched for a specific attribute and run it through the Event Viewer Filter XML UI.
<QueryList>
<Query Id="0" Path="Security">
<Select Path="Security">*[System[(EventID=4688)]] and *[EventData[Data[@Name='ParentProcessName']='C:\Windows\System32\cmd.exe']]</Select>
</Query>
</QueryList>
Writing XPATH Queries
I am not a programmer and I did not have the need to leverage XPATH queries in the past so I struggled a bit when attempting to filter using XPATH. I will show you some tips and tricks below so hopefully you don't struggle as much as I did. You can get your Google on and attempt to find examples, but many of the examples are based on filtering Windows Event logs and those are not always compatible with the AMA agent.
When first writing XPATH queries a good way to test them to ensure they are working is to test them with the powershell module get-Winevent as discussed above. This article does a great job in helping understand XML and XPATH and how to write and test queries using get-winevent.
This article good guide that provides a good overview on how to format the XPATH queries to be comparable with the Azure AMA agent.
Since we are already running Microsoft Endpoint Protection, I did not want to ingest duplicate logs. I also did not want to capture everything as that gets expensive quickly. What I decided to do was capture the events suggested by the the Mitre Attack Matrix and a few others I use to generate auditing reports. I used the following blog post written by Roberto Rodriguez (@Cyb3rWard0g) from the MSTIC R&D team to generate the XPATH quires needed in Azure Sentinel. Since I already have leveraged the Powershell script to extract the necessary XPATH queries, I will place them in my github repository to make it easy for you use.
One of the events that the Mitre Attack Matrix suggests to capture is 4624, which is account was successfully logged in. If you are like most companies, you more than likely have software that leverages a service account. That service account can be noisy and fill up your logs with unseeded and unnecessary events, it can also cost you money ingesting those logs into Azure Sentinel. I had a difficult time figuring out how to filter accounts leveraging XPATH. Most examples I found discussed how to filter using XML, which will not translate to Azure Sentinel.
I could easily get a single account to be filtered, but I could not get the syntax correct to filter multiple accounts. The syntax below is how you filter multiple accounts.
Security!*[System[(EventID=4624)]] and *[EventData[Data[@Name='TargetUserName']!='account1']] and *[EventData[Data[@Name='TargetUserName']!='account2']]
You can Leverage multiple fields in the EventData to filter out events. I choose TargetUserName leveraging xpath's not equal to !=. If you notice, you will need to use the and statement and call out the full Eventdata path to filter out additional accounts.
Once I had cracked the formula, it was easy to write queries to filter out events.
A tip if you are wanting to find out if you have accounts that are causing noise in is to leverage the following query listed below in Azure Sentinel. It will summarize the top events by Target Account Name
SecurityEvent
| where EventID ==4624
| summarize count() by TargetAccount
Now that you have the syntax, you can leverage this knowledge to filter out almost data the is just noise and you do not want to ingest. You can adjust this query to any event id and summarize by any parameter you would like.
Final Thoughts
I like the flexibility the new AMA provides with the ability to filter out logs. I do not like that Microsoft has not fully baked the agent. They should have included the ability to ingest DNS logs to mirror the old agent.
With the ability to inject logs using Windows Event Forwarding (Which is a future blog Post). I would recommend installing this agent only on systems that would like to collect more granular logs from. Also, if you are ingesting logs from Microsoft Endpoint protection, make sure you are not ingesting the same data twice. Microsoft Endpoint protection already ingest many useful logs so make sure you to that into account when deciding which events to collect.
In my GitHub repository I have added the MITRE Attack Frame XPath XML files, which can be used when creating Windows Event Forwarding subscriptions.
I have also added the MITRE Attack Frame Xpath queries you can use with Azure Sentinel.
I also added a query I use to capture Local Machine and Active Directory group changes. You can use Automation in Azure Sentinel to kick out a daily report on Group changes. The Github repository is located here.
Happy Hunting and I hope you found this blog post useful.