Yellow - Logging and Security Architecture

Security logging architecture resources for log collection, endpoint logs, cloud audit logs, network telemetry, retention, and log source evaluation.

Security logging is the foundation for investigations, detections, threat hunting, compliance, and operational troubleshooting. This section focuses on what to collect, how to collect it, where to store it, and how to keep log quality high enough to support real work.

Detection engineering, SIEM rules, Sigma, and alert use cases live under Blue Defense.

Event Detection

Logging Guides

• OWASP Logging Cheat Sheet

• OWASP Logging Vocabulary Cheat Sheet

• Neo23x0 Auditd - Best-practice Linux auditd configuration.

• JSCU-NL Logging Essentials - Windows event logging and collection baseline focused on balancing forensic value and retention.

• Applied Network Security Monitoring: Setting up Network Sensors, pg. 49; Sensor Placement, pg. 61.

Network hardening articles and unwanted-traffic reduction guidance belong with network detection use cases.

General Network Traffic

Logging Data Types

• Packet Capture - Complete packet-level record. It is the richest network evidence, but storage cost is high.

• Session / Flow Data - Summarized connection metadata such as source, destination, ports, protocol, bytes, and timestamps.

• Statistical Data - Metrics and summaries derived from logs or traffic.

• Packet String Data (PSTR) - Selected strings extracted from network traffic, such as cleartext protocol headers or URLs.

• Log Data - Raw events generated by systems, applications, devices, cloud platforms, and services.

• Alert Data - Events produced by detection tools or correlation logic.

Packet capture and NFAT tooling are maintained on the Packet Analysis page.

Packet Analysis

Section Map

How to create a logging strategyLogging - Network ServicesLogging - Endpoint Logs (Windows & Linux)Logging - CloudDevice Discovery and Asset MonitoringLog Source EvaluationLogging - User Behavior Monitoring

Logging System Components

Log Collectors

Collectors gather events from endpoints, servers, network devices, cloud platforms, and applications.

• Log agents - Installed on devices to collect, filter, buffer, parse, encrypt, and forward logs.

  • Splunk Universal Forwarder

  • ArcSight Connectors

  • QRadar WinCollect

  • LogRhythm System Monitor Service

  • Elastic Beats

  • NXLog

• Syslog - Common network and Unix logging protocol. RFC 5424 syslog can use UDP, TCP, and TLS depending on implementation and configuration.

• Windows Event Forwarding - Built-in Windows event forwarding through WinRM. Use Microsoft documentation for the built-in feature and the Palantir repository for a deployment model.

  • Microsoft: Use Windows Event Forwarding to help with intrusion detection

  • Palantir Windows Event Forwarding

• Agentless collection - Centralized pull or device-native push collection, often over WMI, SSH, syslog, API, or file export.

• Scripted collection - Useful for cloud services, SaaS platforms, inventory, and APIs that do not support standard log forwarding.

Sysmon configuration and Sysmon-focused detection resources live on the Sysmon page.

Sysmon

Log Aggregators

Aggregators receive logs, parse them, normalize fields, enrich events, and route output.

• Splunk Heavy Forwarder - Collects, parses, filters, and forwards Splunk data.

• Logstash - Elastic pipeline tool for ingestion, transformation, and routing.

  • Logstash input plugins

  • Logstash filter plugins

  • Logstash output plugins

  • Elastic Grok Debugger

  • HASecuritySolutions Logstash

• Nagios - Infrastructure monitoring and alerting platform used heavily by network and systems teams.

SIEM platforms, enrichment tools, and alerting engines are maintained in Blue Defense.

SIEM and Enrichment

Log Brokers

A log broker buffers events during bursts or downstream outages.

• RabbitMQ

• Apache Kafka

Storage

Log storage should support integrity, retention, and investigation speed.

• Hot storage - Recent, frequently searched data.

• Warm storage - Older data that still needs reasonable search performance.

• Cold storage - Long-term retention for compliance, historical investigations, and rare lookbacks.

• WORM - Write Once, Read Many storage for tamper-resistant retention.

Search and Reporting

• Elasticsearch - Distributed search and analytics engine.

• Kibana - Elastic search, dashboard, and investigation interface.

• Splunk - Commercial data platform and SIEM ecosystem.

For query languages used across SIEM and log platforms, see Blue Defense Query Languages.

Query Languages