Work with Time¶
Time filters, time windows, ordering, and time display all depend on one thing: Kelora knowing which field holds the timestamp and how to read it.
Which timestamp Kelora uses¶
Kelora looks for common field names — ts, timestamp, time, @timestamp,
datetime, created_at, t, and a few more — and recognizes ISO 8601,
RFC 3339, syslog, Apache, many application-log layouts, and Unix epochs in
seconds, milliseconds, or microseconds. Checking the result and fixing the
common problems is part of parsing:
Get the timestamp right. The details are
below.
The parsed value is available to scripts as meta.parsed_ts. The timestamp field itself, whatever its name (e.ts,
e.timestamp, …), keeps the original text.
Formats¶
--ts-format uses chrono format codes:
%Y-%m-%d %H:%M:%S,%3f for Python logging, %d/%b/%Y:%H:%M:%S %z for Apache.
Quote the format so the shell leaves % alone. It applies to a field the parser
produced, so for a plain text file first split the timestamp off with
cols: or regex:.
Time zones¶
A timestamp with an offset (+02:00, Z) is always read as written. A
timestamp without one is naive: Kelora reads it in the zone given by
--input-tz, else the TZ environment variable, else UTC. (In containers TZ
is often set — check it if times look shifted.) When that assumption affects the result — a time
filter, a time window, --normalize-ts — Kelora prints a hint:
ts='2024-01-15T10:00:00+00:00' level='INFO' msg='Application started on :8080'
💡 Timestamps carry no zone offset; assuming UTC, and --normalize-ts writes that offset into the output. Pass --input-tz <zone> if your source is not UTC.
A trailing zone abbreviation (CEST, PST) stops auto-detection, because
abbreviations are ambiguous. Add %Z to --ts-format to skip over it, and set
the zone with --input-tz:
--ts-format '%Y-%m-%d %H:%M:%S %Z' --input-tz Europe/Berlin.
Missing years¶
Syslog-style timestamps (Jan 15 10:00:00) have no year. Kelora picks the date
closest to now that is at most a day in the future — right for recent logs,
wrong for archives; --stats says when it had to guess. Pass
--input-year 2024 for old files.
Filter by time¶
timestamp='2024-01-15T10:15:00Z' level='CRITICAL' message='Disk space critical'
timestamp='2024-01-15T10:16:00Z' level='ERROR' message='Service unavailable'
timestamp='2024-01-15T10:17:00Z' level='WARN' severity='high'
message='Alert sent'
timestamp='2024-01-15T10:20:00Z' level='INFO' message='Disk cleanup initiated'
| Form | Example | Meaning |
|---|---|---|
| absolute | 2024-01-15T10:00:00Z, '2024-01-15 10:00', 10:00 |
that moment (a bare time means today) |
| relative to now | 1h, -30m, 2d, now-15m, yesterday, today |
that far back |
| relative to the other bound | --until since+30m, --since until-1h |
a window of fixed length |
| Unix epoch | 1735566123 |
seconds since 1970 |
A server in Berlin that logs local time without a zone, filtered by Berlin time and printed in UTC:
kelora examples/berlin_local.log -f 'cols:ts(2) *msg' --input-tz Europe/Berlin \
--since '2024-09-05 10:00' --until '2024-09-05 10:30' --normalize-ts
ts='2024-09-05T08:02:00+00:00' msg='disk check ok'
ts='2024-09-05T08:14:00+00:00' msg='backup finished'
ts='2024-09-05T08:21:00+00:00' msg='login admin'
--since is inclusive (at or after), --until too (at or before). A date and
time without a zone is read like the log's timestamps: in --input-tz, else
TZ, else UTC. Relative times count from now, so for an archived file use
absolute times.
- The time range is applied before every other filter and script, so
--freqand other counts always cover exactly the events in the range. - Events without a timestamp can't be placed in the range and are dropped.
Kelora warns with a count. Usually the fix is upstream: point
--ts-fieldat the right field, or join continuation lines with-M. - The range uses the timestamp the parser found. If you build a timestamp in a
script (separate date and time fields, say), filter on it with
--filterinstead:
kelora app.jsonl \
-e 'e.when = to_datetime(e.date + "T" + e.time + "Z")' \
--filter 'e.when >= to_datetime("2024-05-01T00:00:00Z")'
Show timestamps differently¶
| Option | Effect |
|---|---|
-Z / -z |
display timestamps in UTC / local time (default output format only) |
--normalize-ts |
rewrite the timestamp field as RFC 3339 — in every output format, including JSON and CSV |
--mark-gaps 5m |
print a divider wherever consecutive events are 5 minutes or more apart |
Time in scripts¶
meta.parsed_ts is a datetime value, so you can use its methods directly:
kelora examples/simple_json.jsonl \
-e 'e.hour = meta.parsed_ts.hour(); e.day = meta.parsed_ts.format("%a %d %b")' \
-k timestamp,hour,day -n 2
timestamp='2024-01-15T10:00:00Z' hour=10 day='Mon 15 Jan'
timestamp='2024-01-15T10:00:05Z' hour=10 day='Mon 15 Jan'
| Want | Write |
|---|---|
| a part of the time | .year(), .month(), .day(), .hour(), .minute(); weekday: .format("%a") |
| custom text | .format("%Y-%m-%d %H:%M"), .to_iso() |
| another zone | .to_timezone("America/New_York"), .to_utc(), .to_local() |
| round to a bucket | .round_to("5m") (down), .ceil_to("1h") (up) |
| parse another field | to_datetime(e.started_at), with format and zone: to_datetime(s, "%d.%m.%Y %H:%M", "Europe/Berlin") |
| the difference | (end - start).as_seconds(), .as_milliseconds(), .to_string() |
| shift | meta.parsed_ts + to_duration("90m") |
| compare | meta.parsed_ts > to_datetime("2024-05-01T00:00:00Z") |
| now | now() |
| readable duration | humanize_duration(e.duration_ms) → "1m 30s" |
Count events per hour, but only those before 11:00 Berlin time:
kelora examples/web_access_large.log.gz \
--filter 'meta.parsed_ts.to_timezone("Europe/Berlin").hour() < 11' \
-m -e 'track_freq("per_hour", meta.parsed_ts.round_to("1h"))'
round_to and ceil_to compute boundaries in UTC.
For per-window rows with their own metrics, use spans.
Related¶
- Merge files by timestamp when several sorted logs need to become one timeline.
- Time format reference for every format code.