Skip to content

Events and Fields

Understanding how Kelora represents log data as structured events and how to access their fields.

What is an Event?

An event is Kelora's internal representation of a single log entry. After parsing, each log line becomes an event (a map/object) with fields that you can access and transform.

// In Rhai scripts, the current event is available as 'e'
e.timestamp  // Access field directly
e.level      // Access another field
e.message    // And so on

Event Structure

Events are maps (key-value pairs) where:

  • Keys are field names (strings)
  • Values can be any JSON-compatible type
{
  "timestamp": "2024-01-15T10:30:00Z",
  "level": "ERROR",
  "service": "api",
  "user": {
    "id": 12345,
    "name": "alice"
  },
  "tags": ["authentication", "security"],
  "duration_ms": 1234
}

After parsing, this becomes an event with fields accessible as:

  • e.timestamp → "2024-01-15T10:30:00Z"
  • e.level → "ERROR"
  • e.service → "api"
  • e.user → Map with id and name fields
  • e.tags → Array ["authentication", "security"]
  • e.duration_ms → 1234

Field Types

Kelora preserves JSON types after parsing:

Type Example Value Rhai Access
String "error" e.level
Integer 404 e.status
Float 1.234 e.duration
Boolean true e.success
Null null e.optional_field → ()
Object/Map {"key": "value"} e.metadata
Array [1, 2, 3] e.scores

Unit Type (): In Rhai, null from JSON becomes the unit type (), representing "no value" or "empty".

Field Access Patterns

Direct Access

For simple field names (alphanumeric, no special characters):

e.timestamp
e.level
e.service
e.user_id

Nested Access

Access nested fields using dot notation:

e.user.name          // Object field
e.user.id            // Another nested field
e.metadata.region    // Deeper nesting

Important: Direct nested access requires the field to exist. If the field might be missing, check first or use get_path():

// Safe: Check before access
if "user" in e && "name" in e.user {
    e.user.name
}

// Safer: Use get_path with default
e.get_path("user.name", "unknown")      // Returns "unknown" if missing
e.get_path("metadata.region", "us-west")  // Fallback value

Array Access

Access array elements by index:

e.tags[0]            // First element
e.tags[1]            // Second element
e.tags[-1]           // Last element (negative indexing)
e.tags[-2]           // Second-to-last element

Bracket Notation

For field names with special characters or dynamic access:

e["content-type"]           // Hyphens in field name
e["@timestamp"]             // @ symbol in name
e["user-agent"]             // Multiple special chars
e.headers["authorization"]  // Nested with special chars

Deep Nested Access

Combine patterns for complex structures:

e.user.addresses[0].city                    // Object → array → object
e.data.items[-1].metadata.tags[0]          // Multiple levels
e.response.headers["content-type"]          // Object → bracket notation

Safe Field Access

Check Field Existence

Before accessing fields, check if they exist:

// Top-level field
if "field" in e {
    e.field
} else {
    "default"
}

// Nested field
if e.has_path("user.role") {
    e.user.role
} else {
    "guest"
}

Array Bounds Checking

Check array length before accessing elements:

if e.scores.len() > 0 {
    e.scores[0]
} else {
    0
}

// Last element safely
if e.items.len() > 0 {
    e.items[-1]
} else {
    #{}
}

Safe Path Access

Use get_path() for safe nested access with defaults:

// Returns default if path doesn't exist
e.user_role = e.get_path("user.role", "guest")
e.first_tag = e.get_path("tags[0]", "untagged")
e.response_code = e.get_path("response.status", 0)

// Complex nested paths
e.city = e.get_path("user.address.city", "unknown")

Type Checking

Check if field has a value (not unit type):

if type_of(e.field) != "()" {
    // Field exists and has a value
    e.field
}

Modifying Events

Add Fields

Assign values to new or existing fields:

e.processed = true
e.category = "error"
e.duration_s = e.duration_ms / 1000

Modify Existing Fields

Transform field values in place:

e.level = e.level.to_upper()
e.message = e.message.trim()
e.tags = sorted(e.tags)

Remove Fields

Assign unit () to remove fields:

e.password = ()          // Remove sensitive field
e.internal_id = ()       // Remove another field

Removed fields won't appear in output.

Remove Entire Event

Clear all fields to filter out the event:

if e.level == "DEBUG" {
    e = ()  // Event becomes empty and is filtered out
}

Empty events are counted as "filtered" in statistics.

Field Name Patterns

Common Field Names

Kelora recognizes these standard field names across formats:

Timestamps:

  • timestamp, ts, time, @timestamp

Log Levels:

  • level, severity, loglevel

Messages:

  • message, msg, text

Identifiers:

  • id, request_id, trace_id, span_id

Metadata:

  • service, host, hostname, source

Format-Specific Fields

Different parsers add format-specific fields:

JSON (-f json):

  • Preserves all original fields
  • Nested structures maintained

Syslog (-f syslog):

  • hostname, appname, procid, msgid
  • facility, severity

Combined/Apache (-f combined):

  • ip, timestamp, request, status, bytes
  • method, path, protocol
  • referer, user_agent
  • request_time (NGINX only)

CSV (-f csv):

  • Column names from header row
  • Or col_0, col_1, col_2, etc. without header

Logfmt (-f logfmt):

  • All key-value pairs as top-level fields

Working with Nested Structures

Objects/Maps

Access nested maps using dot notation:

kelora -f json examples/simple_json.jsonl \
  --filter 'e.service == "api"' \
  --exec 'e.req_method = e.get_path("request.method", "UNKNOWN")' \
  --keys timestamp,service,req_method \
  --take 3

kelora -f json examples/simple_json.jsonl \
  --filter 'e.service == "api"' \
  --exec 'e.req_method = e.get_path("request.method", "UNKNOWN")' \
  --keys timestamp,service,req_method \
  --take 3

timestamp='2024-01-15T10:00:00Z' service='api' req_method='UNKNOWN'
timestamp='2024-01-15T10:00:05Z' service='api' req_method='UNKNOWN'
timestamp='2024-01-15T10:01:00Z' service='api' req_method='UNKNOWN'

Arrays

Process arrays with Rhai array functions:

kelora -f json input.log \
    --exec 'e.tag_count = e.tags.len()' \
    --exec 'e.first_tag = e.get_path("tags[0]", "none")' \
    --exec 'e.unique_tags = unique(e.tags)' \
    --keys timestamp,tag_count,first_tag,unique_tags

Fan-Out Arrays

Convert array elements to individual events:

kelora -f json batch.log \
    --exec 'emit_each(e.items)' \
    --keys item_id,status

Each element in e.items becomes a separate event.

Mixed Structures

Handle complex nested structures:

{
  "user": {
    "id": 12345,
    "scores": [85, 92, 78],
    "metadata": {
      "tags": ["premium", "verified"]
    }
  }
}

Access patterns:

e.user_id = e.user.id                           // Nested object
e.first_score = e.user.scores[0]                // Object → array
e.last_score = e.user.scores[-1]                // Negative index
e.avg_score = e.user.scores.sum() / e.user.scores.len()
e.first_tag = e.user.metadata.tags[0]           // Deep nesting
e.is_premium = "premium" in e.user.metadata.tags // Array membership

Field Naming Conventions

Output Format: Bracket Notation

The default formatter uses bracket notation for arrays:

scores[0]=85 scores[1]=92 scores[2]=78
user.name=alice user.scores[0]=85
items[0].id=1 items[0].status=active

This matches the path syntax used in get_path():

// Access and path syntax are consistent
e.get_path("scores[0]", 0)           // Access first score
e.get_path("items[1].status", "")    // Access nested array element

Field Selection: Top-Level Only

The --keys parameter operates on top-level fields only:

# ✅ Supported: Select top-level fields
kelora -f json input.log --keys user,timestamp,message

# ❌ Not supported: Nested paths in --keys
kelora -f json input.log --keys user.name,scores[0]

To extract nested fields, use --exec to promote them to top-level:

kelora -f json input.log \
    --exec 'e.user_name = e.get_path("user.name", "")' \
    --exec 'e.first_score = e.get_path("scores[0]", 0)' \
    --keys user_name,first_score

Common Patterns

Extract Nested Fields

kelora -f json app.log \
    --exec 'e.user_name = e.get_path("user.name", "unknown")' \
    --exec 'e.user_role = e.get_path("user.role", "guest")' \
    --keys timestamp,user_name,user_role

Flatten Structures

kelora -f json app.log \
    --exec 'e.request_method = e.request.method' \
    --exec 'e.request_path = e.request.path' \
    --exec 'e.request = ()' \
    --keys timestamp,request_method,request_path

Combine Fields

kelora -f json app.log \
    --exec 'e.full_name = e.first_name + " " + e.last_name' \
    --exec 'e.endpoint = e.method + " " + e.path' \
    --keys timestamp,full_name,endpoint

Conditional Field Creation

kelora -f json app.log \
    --exec 'if e.status >= 500 { e.severity = "critical" } else if e.status >= 400 { e.severity = "warning" } else { e.severity = "normal" }' \
    --keys timestamp,status,severity

Array Transformations

kelora -f json app.log \
    --exec 'e.tag_count = e.tags.len()' \
    --exec 'e.sorted_tags = sorted(e.tags)' \
    --exec 'e.unique_tags = unique(e.tags)' \
    --keys timestamp,tag_count,sorted_tags

Safe Deep Access

kelora -f json app.log \
    --exec 'e.city = e.get_path("user.address.city", "N/A")' \
    --exec 'e.zip = e.get_path("user.address.zip", "00000")' \
    --exec 'e.country = e.get_path("user.address.country", "Unknown")' \
    --keys city,zip,country

Type Conversions

String to Number

e.status_code = e.status.to_int()           // String to integer
e.duration = e.duration_str.to_float()      // String to float

With defaults if conversion fails:

e.status_code = to_int_or(e.status, 0)
e.duration = to_float_or(e.duration_str, 0.0)

Number to String

e.status_str = e.status.to_string()
e.duration_str = e.duration.to_string()

Boolean Conversions

e.is_error = to_bool(e.error_flag)          // "true"/"false" to boolean
e.success = to_bool_or(e.status_ok, false)  // With default

Type Checking

e.field_type = type_of(e.field)

// Common checks
if type_of(e.field) == "i64" { /* integer */ }
if type_of(e.field) == "f64" { /* float */ }
if type_of(e.field) == "string" { /* string */ }
if type_of(e.field) == "array" { /* array */ }
if type_of(e.field) == "map" { /* object/map */ }
if type_of(e.field) == "()" { /* null/empty */ }

Field Access Performance

Direct vs Path Access

Direct access is fastest for known fields: 

e.level           // Fast: direct map lookup
e.user.name       // Fast: two map lookups

Use direct access when: - Field names are known at script time - Fields are guaranteed to exist (e.g., parser output) - Performance is critical

Path access provides safety and flexibility: 

e.get_path("level", "INFO")              // Safe with default
e.get_path("user.name", "unknown")       // Handles missing fields

Use get_path() when: - Fields might not exist (optional data) - Working with inconsistent log formats - You need default values for missing fields - Path is dynamic or comes from configuration

Hybrid approach for best results: 

// Check existence, then use direct access for speed
if "user" in e {
    e.user_name = e.user.name              // Fast once verified
} else {
    e.user_name = "unknown"
}

// Or use get_path for one-liners
e.user_name = e.get_path("user.name", "unknown")  // Simpler but slower

Field Existence Checks

Fastest - direct membership: 

"field" in e

Flexible - path checking: 

e.has_path("user.role")
e.has_path("items[0].status")

Array Operations

Array functions create new arrays (not in-place):

// Creates new sorted array
e.sorted_scores = sorted(e.scores)

// Original array unchanged
e.scores  // Still unsorted

For large arrays, avoid unnecessary transformations.

Troubleshooting

Field Not Found

Problem: Accessing non-existent field causes error.

Solution: Use safe access patterns: 

// Check before access
if "field" in e {
    e.field
}

// Use get_path with default
e.get_path("field", "default")

Array Index Out of Bounds

Problem: Accessing array element beyond length.

Solution: Check array length first: 

if e.items.len() > 5 {
    e.items[5]
}

// Or use safe path access
e.get_path("items[5]", #{})

Type Mismatch

Problem: Field has unexpected type.

Solution: Use type conversion with defaults: 

e.status_code = to_int_or(e.status, 0)
e.duration = to_float_or(e.duration_ms, 0.0)

Nested Field Access Errors

Problem: Accessing non-existent nested fields causes errors.

Solution: Check field existence before access or use get_path(): 

// Error if user or name doesn't exist
e.user.name = "alice"

// Safe: Check first
if "user" in e {
    e.user.name = "alice"
}

// Safest: Use get_path for reading
let current = e.get_path("user.name", "default")

Note: Direct nested assignment (e.user.name = "alice") works fine when the parent object exists.

Field Name with Special Characters

Problem: Field names contain hyphens, dots, or other special chars.

Solution: Use bracket notation: 

e["content-type"]
e["user-agent"]
e["@timestamp"]

See Also