Query Languages
Common Security Events, how to analyze them, and the tools to do so
Triaging and investigating security events are the bread and butter of any SOC analyst. Before we can investigate we must first have two things: A common terminology to define attacks and security related activity, and searches to direct us to that activity.
Event search languages and rules
Weather it is an open source tool like Elastic Stack, or a proprietary tool like Splunk, every data mining or security event generating tool, has its own language it works in. Specifically for SIEMs, EDRs, and Data mining platforms, they require an extensive and flexible language to search for data with a multitude of conditions that a user might have. Writing effective searches is an artform, and it is highly encouraged that you do tons of research and testing before deployment. Blue team blog has a fantastic Use Case writing guide as well as some fantastic use cases available for free.
The syntax and format of all of the available languages is vast and complex. Especially so, when you have to convert search parameters from one tool to another. In comes Sigma.
"Sigma is a generic and open signature format that allows you to describe relevant log events in a straightforward manner. The rule format is very flexible, easy to write and applicable to any type of log file. The main purpose of this project is to provide a structured form in which researchers or analysts can describe their once developed detection methods and make them shareable with others.Sigma is for log files what Snort is for network traffic and YARA is for files." - Sigma Github Notes
Sigma is a fantastic tool that decouples rule logic from vendor terminology. Sigma is stored in easy to ready YAML format and is compatible with the MISP intel tool.
Remember two things: First, Sigma queries may not be perfect, but they should get you 90-95% the way towards what you are looking for. Be prepared to tweak!
Second, online converters might not be able to translate from one platform easily. Example: LogRhythm -> Splunk. Sometimes you need to look at various Github repositories to get the code to convert to Sigma, then on to the platform of your choice. Example: LogRhythm -> Sigma -> Splunk.
To make Sigma even easier to use, there are tools like Uncoder.io that can easily translate the rule syntax from one platform to another, free of charge. As stated above, if you cannot do direct translations from one platform to another, try changing the source search to Sigma first, then on to the platform of your choice.
Apache's search language that is used in many technologies including Elastic Stack and Palo Alto's XSOAR
- KQL: Kibana search language based on Lucene. - https://www.elastic.co/guide/en/kibana/master/kuery-query.html
- Threat Hunting in Elastic Stack: Lucene - pg. 212
Solr - Solr is the popular, blazing-fast, open source enterprise search platform built on Apache Lucene
KQL - Kibana Query Language, the default query language of the Kibana seach and alerting utility within Elastic Stack.
- Threat Hunting in Elastic Stack: KQL - pg. 216
EQL - Elastic Query Language, an advanced query language developed by Elastic for use in thier Security App.
- Threat Hunting in Elastic Stack: EQL - pg. 220
Splunk's SPL: Search Processing Language
- Operator Handbook: Splunk - pg. 277
Graylog
- Graylog's query language is very close to Lucene. Watch for syntax errors.
Windows O365
Windows Defender ATP
- Operator Handbook: Windows Defender ATP - pg. 417
Last modified 1mo ago