Elasticsearch Audit

Overview

Elasticsearch X-Pack security provides audit logging for authentication, authorization, index operations, and administrative actions. KYRA MDR collects Elasticsearch audit events to detect unauthorized access, privilege escalation, and data exfiltration attempts across your search and analytics infrastructure.

Prerequisites

KYRA MDR account (MDR tier or above)
KYRA Collector installed and reachable from the Elasticsearch cluster
Elasticsearch 7.x or 8.x with X-Pack security enabled (included in default distribution since 7.11)
Administrative access to the Elasticsearch cluster configuration

Configuration

Step 1: Enable Audit Logging

Edit elasticsearch.yml on each node in the cluster:

# Enable X-Pack security (enabled by default in 8.x)
xpack.security.enabled: true

# Enable audit logging
xpack.security.audit.enabled: true

# Configure audit output (logfile is default)
xpack.security.audit.logfile.events.include:
  - access_denied
  - access_granted
  - anonymous_access_denied
  - authentication_failed
  - authentication_success
  - connection_denied
  - connection_granted
  - run_as_denied
  - run_as_granted
  - security_config_change
  - system_access_granted
  - tampered_request

# Exclude noisy events (optional)
xpack.security.audit.logfile.events.exclude:
  - system_access_granted

# Include request body in audit entries (for write operations)
xpack.security.audit.logfile.events.emit_request_body: true

Step 2: Configure Event Filtering

Reduce noise by filtering specific users and indices:

# Exclude internal system users from audit
xpack.security.audit.logfile.events.ignore_filters:
  system_filter:
    users: ["_xpack_security", "_xpack", "elastic/fleet-server"]
    realms: ["_service_account"]
  monitoring_filter:
    users: ["beats_system", "apm_system", "remote_monitoring_user"]
    indices: [".monitoring-*", ".security-*"]
  health_check_filter:
    actions: ["cluster:monitor/*"]

Step 3: Restart Elasticsearch

sudo systemctl restart elasticsearch

# Verify audit logging is active
curl -s -u elastic:<PASSWORD> "https://localhost:9200/_cluster/settings?include_defaults&filter_path=defaults.xpack.security.audit" | jq '.'

Step 4: Forward Audit Logs via Filebeat

Install Filebeat on each Elasticsearch node to forward audit logs to the KYRA Collector:

filebeat.inputs:
  - type: log
    enabled: true
    paths:
      - /var/log/elasticsearch/*_audit.json
    json.keys_under_root: true
    json.add_error_key: true
    fields:
      source: elasticsearch-audit

output.syslog:
  host: "<COLLECTOR_IP>:514"
  protocol: tcp

If Filebeat syslog output is not available, use Logstash or direct file forwarding:

# Alternative: output to Logstash which forwards to syslog
output.logstash:
  hosts: ["<LOGSTASH_IP>:5044"]

Step 5: Alternative Syslog via rsyslog

Forward the audit log file directly via rsyslog:

module(load="imfile" PollingInterval="5")

input(type="imfile"
  File="/var/log/elasticsearch/*_audit.json"
  Tag="es-audit"
  Facility="local7"
  Severity="info")

local7.*  @@<COLLECTOR_IP>:514

sudo systemctl restart rsyslog

Step 6: Verify on KYRA Collector

kyra-collector status
kyra-collector logs --source elasticsearch-audit --tail 5

# Verify audit log is being written
sudo tail -5 /var/log/elasticsearch/*_audit.json | jq '.event.action'

Audit Event Types

Event Type	Description	Security Use
`authentication_success`	User successfully authenticated	Login monitoring
`authentication_failed`	Failed authentication attempt	Brute force detection
`access_granted`	Authorized action on resource	Access tracking
`access_denied`	Unauthorized action attempt	Privilege violation detection
`anonymous_access_denied`	Unauthenticated request denied	Attack surface monitoring
`connection_denied`	IP-based connection rejected	Network security
`run_as_denied`	RunAs impersonation denied	Privilege escalation detection
`security_config_change`	Security configuration modified	Change management
`tampered_request`	Request with invalid authentication token	Token manipulation detection

Key Audit Log Fields (JSON)

Field	Description
`@timestamp`	Event timestamp
`event.action`	Audit event type
`user.name`	Authenticated username
`user.realm`	Authentication realm (native, ldap, pki)
`user.run_as.name`	Impersonated username (if RunAs)
`origin.address`	Client IP address
`request.name`	API action (e.g., `indices:data/read/search`)
`url.path`	REST API path
`indices`	Target index name(s)
`event.outcome`	success or failure

Critical Actions to Monitor

Action Pattern	Description
`indices:admin/delete`	Index deletion
`indices:data/write/bulk`	Bulk data write
`cluster:admin/settings/update`	Cluster settings change
`cluster:admin/snapshot/*`	Snapshot operations (backup/restore)
`indices:admin/mapping/put`	Index mapping changes
`cluster:admin/repository/*`	Repository management
`security:*`	Security configuration changes

Troubleshooting

Audit logs not generated: Verify xpack.security.audit.enabled: true in elasticsearch.yml and restart the node. Check the Elasticsearch log for audit-related errors.
Excessive volume: The system_access_granted event is very noisy (internal cluster communication). Always add it to events.exclude or use ignore_filters.
Missing authentication events: Ensure X-Pack security is fully enabled with xpack.security.enabled: true. Check that the security features are not running in trial mode.
Audit log file not rotating: Elasticsearch rotates audit logs daily by default (max 2GB per file). Check disk space on the Elasticsearch data directory.
LDAP/SAML users: If using external authentication, the user.realm field indicates the authentication realm. Ensure the Collector maps these correctly.
8.x breaking changes: Elasticsearch 8.x enables security by default and uses HTTPS. Ensure Filebeat/rsyslog is configured to read from the correct audit log path.

Contact kyra@seekerslab.com for integration support.