Analyze Web Traffic¶
Parse and analyze Apache/Nginx access logs to find slow requests, errors, and traffic patterns.
Problem¶
You have Apache or Nginx access logs and need to find slow requests, 4xx/5xx errors, traffic patterns, or analyze request distribution.
Solutions¶
Basic Combined Log Parsing¶
Parse Apache/Nginx combined format logs:
timestamp='04/Oct/2025:10:27:22 +0200' ip='52.127.35.227'
request='HEAD /harness/methodologies/unleash/methodologies HTTP/1.0' method='HEAD'
path='/harness/methodologies/unleash/methodologies' protocol='HTTP/1.0' status=403 bytes=79332
referer='https://www.nationalgrow.name/revolutionary/24/365/clicks-and-mortar/cross-media'
user_agent='Mozilla/5.0 (X11; Linux i686) AppleWebKit/5330 (KHTML, like Gecko) Chrome/38.0.848.0 Mobile Safari/5330'
timestamp='04/Oct/2025:10:27:22 +0200' ip='166.86.165.21'
request='PUT /channels/out-of-the-box/implement HTTP/1.0' method='PUT'
path='/channels/out-of-the-box/implement' protocol='HTTP/1.0' status=201 bytes=51969
referer='https://www.regionalmetrics.io/repurpose/technologies/innovative/vertical'
user_agent='Opera/9.23 (Macintosh; U; Intel Mac OS X 10_8_6; en-US) Presto/2.12.264 Version/12.00'
timestamp='04/Oct/2025:10:27:22 +0200' ip='24.83.53.204' request='PATCH /markets HTTP/1.0'
method='PATCH' path='/markets' protocol='HTTP/1.0' status=204 bytes=74618
referer='https://www.futureorchestrate.org/integrated/wireless/seize'
user_agent='Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_6_4) AppleWebKit/5360 (KHTML, like Gecko) Chrome/36.0.807.0 Mobile Safari/5360'
timestamp='04/Oct/2025:10:27:22 +0200' ip='37.144.168.216' user='dickens6646'
request='PATCH /evolve/orchestrate HTTP/1.1' method='PATCH' path='/evolve/orchestrate'
protocol='HTTP/1.1' status=201 bytes=22591
referer='https://www.legacydistributed.com/channels/cultivate'
user_agent='Opera/10.55 (Windows NT 5.1; en-US) Presto/2.12.161 Version/11.00'
timestamp='04/Oct/2025:10:27:22 +0200' ip='24.44.139.136' user='block5105'
request='GET /rich HTTP/1.1' method='GET' path='/rich' protocol='HTTP/1.1' status=400 bytes=21175
referer='http://www.productintuitive.org/sexy/experiences'
user_agent='Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_8_4) AppleWebKit/5361 (KHTML, like Gecko) Chrome/40.0.879.0 Mobile Safari/5361'
The combined format includes: ip, timestamp, request, method, path, protocol, status, bytes, referer, user_agent, and optionally request_time.
Find Server Errors (5xx)¶
kelora -f combined examples/simple_combined.log \
--filter 'e.status >= 500' \
-k ip,timestamp,status,request
ip='152.252.182.35' timestamp='04/Oct/2025:10:27:22 +0200' status=502
request='PATCH /experiences/action-items/best-of-breed HTTP/1.1'
ip='42.111.246.109' timestamp='04/Oct/2025:10:27:22 +0200' status=500
request='PATCH /recontextualize/evolve HTTP/2.0'
ip='166.217.70.101' timestamp='04/Oct/2025:10:27:22 +0200' status=503
request='DELETE /visionary/web-readiness/vertical HTTP/1.0'
ip='230.219.40.103' timestamp='04/Oct/2025:10:27:22 +0200' status=504
request='DELETE /technologies/morph HTTP/1.0'
ip='194.248.104.125' timestamp='04/Oct/2025:10:27:22 +0200' status=501
request='DELETE /proactive/open-source/applications HTTP/2.0'
ip='143.72.103.135' timestamp='04/Oct/2025:10:27:22 +0200' status=503 request='HEAD /scale HTTP/1.0'
Find Client Errors (4xx)¶
kelora -f combined /var/log/nginx/access.log \
--filter 'e.status >= 400 && e.status < 500' \
-k ip,timestamp,status,request
Find Slow Requests¶
For Nginx logs with request_time:
kelora -f combined /var/log/nginx/access.log \
--filter 'e.get_path("request_time", "0").to_float() > 1.0' \
-k ip,request,request_time,status
Traffic by Status Code¶
Count requests by status code:
kelora -f combined examples/simple_combined.log \
-e 'track_count("status_" + e.status)' \
--metrics
timestamp='04/Oct/2025:10:27:22 +0200' ip='52.127.35.227'
request='HEAD /harness/methodologies/unleash/methodologies HTTP/1.0' method='HEAD'
path='/harness/methodologies/unleash/methodologies' protocol='HTTP/1.0' status=403 bytes=79332
referer='https://www.nationalgrow.name/revolutionary/24/365/clicks-and-mortar/cross-media'
user_agent='Mozilla/5.0 (X11; Linux i686) AppleWebKit/5330 (KHTML, like Gecko) Chrome/38.0.848.0 Mobile Safari/5330'
timestamp='04/Oct/2025:10:27:22 +0200' ip='166.86.165.21'
request='PUT /channels/out-of-the-box/implement HTTP/1.0' method='PUT'
path='/channels/out-of-the-box/implement' protocol='HTTP/1.0' status=201 bytes=51969
referer='https://www.regionalmetrics.io/repurpose/technologies/innovative/vertical'
user_agent='Opera/9.23 (Macintosh; U; Intel Mac OS X 10_8_6; en-US) Presto/2.12.264 Version/12.00'
timestamp='04/Oct/2025:10:27:22 +0200' ip='24.83.53.204' request='PATCH /markets HTTP/1.0'
method='PATCH' path='/markets' protocol='HTTP/1.0' status=204 bytes=74618
referer='https://www.futureorchestrate.org/integrated/wireless/seize'
user_agent='Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_6_4) AppleWebKit/5360 (KHTML, like Gecko) Chrome/36.0.807.0 Mobile Safari/5360'
timestamp='04/Oct/2025:10:27:22 +0200' ip='37.144.168.216' user='dickens6646'
request='PATCH /evolve/orchestrate HTTP/1.1' method='PATCH' path='/evolve/orchestrate'
protocol='HTTP/1.1' status=201 bytes=22591
referer='https://www.legacydistributed.com/channels/cultivate'
user_agent='Opera/10.55 (Windows NT 5.1; en-US) Presto/2.12.161 Version/11.00'
timestamp='04/Oct/2025:10:27:22 +0200' ip='24.44.139.136' user='block5105'
request='GET /rich HTTP/1.1' method='GET' path='/rich' protocol='HTTP/1.1' status=400 bytes=21175
referer='http://www.productintuitive.org/sexy/experiences'
user_agent='Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_8_4) AppleWebKit/5361 (KHTML, like Gecko) Chrome/40.0.879.0 Mobile Safari/5361'
timestamp='04/Oct/2025:10:27:22 +0200' ip='67.19.236.47'
request='HEAD /next-generation/drive/turn-key/metrics HTTP/2.0' method='HEAD'
path='/next-generation/drive/turn-key/metrics' protocol='HTTP/2.0' status=404 bytes=55635
referer='http://www.centralarchitectures.info/frictionless/mesh/supply-chains'
user_agent='Mozilla/5.0 (X11; Linux x86_64; rv:8.0) Gecko/2009-24-11 Firefox/37.0'
timestamp='04/Oct/2025:10:27:22 +0200' ip='98.121.108.49' request='DELETE /synergistic HTTP/1.0'
method='DELETE' path='/synergistic' protocol='HTTP/1.0' status=200 bytes=45149
referer='http://www.dynamicsynergize.net/schemas/efficient/dynamic/out-of-the-box'
user_agent='Mozilla/5.0 (Macintosh; PPC Mac OS X 10_8_8) AppleWebKit/5322 (KHTML, like Gecko) Chrome/38.0.881.0 Mobile Safari/5322'
timestamp='04/Oct/2025:10:27:22 +0200' ip='94.224.49.21' request='DELETE /vertical HTTP/1.1'
method='DELETE' path='/vertical' protocol='HTTP/1.1' status=406 bytes=97675
referer='http://www.globalmethodologies.io/24/7/channels/infomediaries/interfaces'
user_agent='Mozilla/5.0 (X11; Linux i686; rv:8.0) Gecko/1955-31-12 Firefox/35.0'
timestamp='04/Oct/2025:10:27:22 +0200' ip='152.252.182.35'
request='PATCH /experiences/action-items/best-of-breed HTTP/1.1' method='PATCH'
path='/experiences/action-items/best-of-breed' protocol='HTTP/1.1' status=502 bytes=85715
referer='https://www.productstreamline.com/synthesize/cross-platform/e-business'
user_agent='Mozilla/5.0 (iPhone; CPU iPhone OS 8_0_2 like Mac OS X; en-US) AppleWebKit/533.45.8 (KHTML, like Gecko) Version/3.0.5 Mobile/8B117 Safari/6533.45.8'
timestamp='04/Oct/2025:10:27:22 +0200' ip='45.128.195.64' user='cartwright5615'
request='GET /content/compelling/roi HTTP/1.0' method='GET' path='/content/compelling/roi'
protocol='HTTP/1.0' status=416 bytes=92521
referer='https://www.forwardapplications.org/cutting-edge/initiatives/open-source/drive'
user_agent='Mozilla/5.0 (iPad; CPU OS 7_1_2 like Mac OS X; en-US) AppleWebKit/536.17.7 (KHTML, like Gecko) Version/5.0.5 Mobile/8B120 Safari/6536.17.7'
timestamp='04/Oct/2025:10:27:22 +0200' ip='80.31.51.183' request='GET /platforms HTTP/1.0'
method='GET' path='/platforms' protocol='HTTP/1.0' status=403 bytes=79686
referer='https://www.senioroptimize.info/cross-platform/granular/orchestrate'
user_agent='Mozilla/5.0 (Macintosh; PPC Mac OS X 10_5_0) AppleWebKit/5342 (KHTML, like Gecko) Chrome/39.0.824.0 Mobile Safari/5342'
timestamp='04/Oct/2025:10:27:22 +0200' ip='9.176.6.23'
request='PATCH /ubiquitous/intuitive/innovate HTTP/1.1' method='PATCH'
path='/ubiquitous/intuitive/innovate' protocol='HTTP/1.1' status=203 bytes=51757
referer='https://www.districte-tailers.org/cross-platform/vortals'
user_agent='Mozilla/5.0 (Windows 95) AppleWebKit/5320 (KHTML, like Gecko) Chrome/37.0.851.0 Mobile Safari/5320'
timestamp='04/Oct/2025:10:27:22 +0200' ip='85.23.199.98' user='will8730'
request='PATCH /action-items/drive/plug-and-play HTTP/2.0' method='PATCH'
path='/action-items/drive/plug-and-play' protocol='HTTP/2.0' status=204 bytes=92119
referer='http://www.humanvisionary.net/compelling/deliverables'
user_agent='Mozilla/5.0 (X11; Linux x86_64; rv:7.0) Gecko/1984-16-10 Firefox/37.0'
timestamp='04/Oct/2025:10:27:22 +0200' ip='193.197.229.98'
request='PATCH /ubiquitous/mission-critical/strategic HTTP/2.0' method='PATCH'
path='/ubiquitous/mission-critical/strategic' protocol='HTTP/2.0' status=203 bytes=17697
referer='http://www.producte-business.net/extend/syndicate/intuitive/global'
user_agent='Mozilla/5.0 (X11; Linux i686) AppleWebKit/5322 (KHTML, like Gecko) Chrome/36.0.813.0 Mobile Safari/5322'
timestamp='04/Oct/2025:10:27:22 +0200' ip='42.111.246.109'
request='PATCH /recontextualize/evolve HTTP/2.0' method='PATCH' path='/recontextualize/evolve'
protocol='HTTP/2.0' status=500 bytes=38979
referer='https://www.directaction-items.net/ubiquitous/cultivate/engineer'
user_agent='Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_2 rv:6.0; en-US) AppleWebKit/535.18.1 (KHTML, like Gecko) Version/4.2 Safari/535.18.1'
timestamp='04/Oct/2025:10:27:22 +0200' ip='4.136.187.120'
request='PATCH /facilitate/enterprise/integrated/vortals HTTP/2.0' method='PATCH'
path='/facilitate/enterprise/integrated/vortals' protocol='HTTP/2.0' status=100 bytes=9617
referer='http://www.internationalback-end.net/reinvent/mission-critical/24/365/integrated'
user_agent='Mozilla/5.0 (Macintosh; PPC Mac OS X 10_5_3) AppleWebKit/5342 (KHTML, like Gecko) Chrome/39.0.846.0 Mobile Safari/5342'
timestamp='04/Oct/2025:10:27:22 +0200' ip='122.169.242.41'
request='PUT /roi/target/collaborative HTTP/1.1' method='PUT' path='/roi/target/collaborative'
protocol='HTTP/1.1' status=301 bytes=58087
referer='https://www.futuree-enable.info/bandwidth/grow/aggregate/killer'
user_agent='Mozilla/5.0 (X11; Linux x86_64; rv:7.0) Gecko/1962-21-05 Firefox/35.0'
timestamp='04/Oct/2025:10:27:22 +0200' ip='115.84.140.75' request='POST /optimize HTTP/1.1'
method='POST' path='/optimize' protocol='HTTP/1.1' status=400 bytes=9256
referer='https://www.customernetworks.com/collaborative/deliverables'
user_agent='Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_7_2 rv:7.0; en-US) AppleWebKit/536.40.6 (KHTML, like Gecko) Version/4.2 Safari/536.40.6'
timestamp='04/Oct/2025:10:27:22 +0200' ip='27.232.247.156' user='haag4547'
request='POST /distributed/cutting-edge HTTP/1.1' method='POST' path='/distributed/cutting-edge'
protocol='HTTP/1.1' status=201 bytes=23618
referer='https://www.chiefimpactful.biz/holistic/real-time/e-business/brand'
user_agent='Opera/8.12 (X11; Linux i686; en-US) Presto/2.11.206 Version/11.00'
timestamp='04/Oct/2025:10:27:22 +0200' ip='59.52.88.145' user='sawayn6527'
request='PATCH /scale/sexy HTTP/1.0' method='PATCH' path='/scale/sexy' protocol='HTTP/1.0'
status=304 bytes=18185
referer='https://www.investorrecontextualize.name/brand/applications/dynamic/content'
user_agent='Mozilla/5.0 (X11; Linux i686) AppleWebKit/5342 (KHTML, like Gecko) Chrome/37.0.832.0 Mobile Safari/5342'
timestamp='04/Oct/2025:10:27:22 +0200' ip='98.104.135.174' user='huels1481'
request='GET /visionary/efficient HTTP/1.1' method='GET' path='/visionary/efficient'
protocol='HTTP/1.1' status=416 bytes=98832
referer='http://www.productextensible.info/24/7/platforms/e-services'
user_agent='Mozilla/5.0 (iPhone; CPU iPhone OS 9_0_2 like Mac OS X; en-US) AppleWebKit/534.31.8 (KHTML, like Gecko) Version/3.0.5 Mobile/8B114 Safari/6534.31.8'
timestamp='04/Oct/2025:10:27:22 +0200' ip='119.158.60.77'
request='POST /intuitive/evolve/cutting-edge HTTP/1.0' method='POST'
path='/intuitive/evolve/cutting-edge' protocol='HTTP/1.0' status=200 bytes=6911
referer='https://www.districtparadigms.com/back-end/empower/e-tailers'
user_agent='Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_9_9 rv:2.0) Gecko/1963-13-10 Firefox/36.0'
timestamp='04/Oct/2025:10:27:22 +0200' ip='193.159.210.56' user='kemmer5342'
request='DELETE /bleeding-edge/architect HTTP/1.1' method='DELETE' path='/bleeding-edge/architect'
protocol='HTTP/1.1' status=200 bytes=12505
referer='http://www.dynamicgenerate.org/synergies/collaborative/orchestrate/strategic'
user_agent='Opera/8.16 (Windows NT 6.1; en-US) Presto/2.13.260 Version/11.00'
timestamp='04/Oct/2025:10:27:22 +0200' ip='56.209.231.163' request='POST /revolutionize HTTP/2.0'
method='POST' path='/revolutionize' protocol='HTTP/2.0' status=403 bytes=47650
referer='https://www.regionalrecontextualize.biz/ubiquitous/24/365/reintermediate/transform'
user_agent='Opera/8.59 (Macintosh; U; Intel Mac OS X 10_5_9; en-US) Presto/2.11.262 Version/12.00'
timestamp='04/Oct/2025:10:27:22 +0200' ip='166.217.70.101'
request='DELETE /visionary/web-readiness/vertical HTTP/1.0' method='DELETE'
path='/visionary/web-readiness/vertical' protocol='HTTP/1.0' status=503 bytes=13249
referer='https://www.globalinteractive.com/e-markets/brand/sexy'
user_agent='Opera/10.20 (Windows NT 6.2; en-US) Presto/2.12.322 Version/12.00'
timestamp='04/Oct/2025:10:27:22 +0200' ip='26.89.117.250'
request='PUT /wireless/cross-platform/e-markets HTTP/2.0' method='PUT'
path='/wireless/cross-platform/e-markets' protocol='HTTP/2.0' status=301 bytes=54105
referer='http://www.internalexperiences.io/compelling/architectures/embrace/bleeding-edge'
user_agent='Opera/9.57 (Windows NT 6.1; en-US) Presto/2.9.297 Version/12.00'
timestamp='04/Oct/2025:10:27:22 +0200' ip='246.133.15.116' request='PATCH /granular HTTP/1.0'
method='PATCH' path='/granular' protocol='HTTP/1.0' status=204 bytes=83575
referer='https://www.districtmatrix.io/facilitate/transform/portals'
user_agent='Mozilla/5.0 (Macintosh; PPC Mac OS X 10_8_1) AppleWebKit/5352 (KHTML, like Gecko) Chrome/40.0.861.0 Mobile Safari/5352'
timestamp='04/Oct/2025:10:27:22 +0200' ip='230.219.40.103' user='mosciski2343'
request='DELETE /technologies/morph HTTP/1.0' method='DELETE' path='/technologies/morph'
protocol='HTTP/1.0' status=504 bytes=39288 referer='http://www.dynamicdeliver.name/whiteboard'
user_agent='Mozilla/5.0 (X11; Linux i686) AppleWebKit/5332 (KHTML, like Gecko) Chrome/40.0.887.0 Mobile Safari/5332'
timestamp='04/Oct/2025:10:27:22 +0200' ip='236.152.161.164' user='marks3733'
request='PUT /evolve/initiatives/turn-key HTTP/2.0' method='PUT'
path='/evolve/initiatives/turn-key' protocol='HTTP/2.0' status=304 bytes=3598
referer='https://www.humansynergistic.com/harness/open-source/e-business'
user_agent='Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_9_5) AppleWebKit/5332 (KHTML, like Gecko) Chrome/37.0.851.0 Mobile Safari/5332'
timestamp='04/Oct/2025:10:27:22 +0200' ip='139.53.169.69' user='kuhlman2864'
request='POST /collaborative/platforms/strategize HTTP/2.0' method='POST'
path='/collaborative/platforms/strategize' protocol='HTTP/2.0' status=301 bytes=13064
referer='https://www.futuremesh.org/scale'
user_agent='Mozilla/5.0 (X11; Linux i686) AppleWebKit/5320 (KHTML, like Gecko) Chrome/40.0.802.0 Mobile Safari/5320'
timestamp='04/Oct/2025:10:27:22 +0200' ip='194.248.104.125'
request='DELETE /proactive/open-source/applications HTTP/2.0' method='DELETE'
path='/proactive/open-source/applications' protocol='HTTP/2.0' status=501 bytes=27834
referer='https://www.internationalcompelling.net/granular/integrate/technologies'
user_agent='Mozilla/5.0 (X11; Linux x86_64; rv:6.0) Gecko/1981-28-05 Firefox/37.0'
timestamp='04/Oct/2025:10:27:22 +0200' ip='213.251.18.226' user='kshlerin7682'
request='PUT /channels/target HTTP/1.0' method='PUT' path='/channels/target' protocol='HTTP/1.0'
status=201 bytes=22564
referer='http://www.nationalout-of-the-box.io/e-tailers/syndicate/holistic/disintermediate'
user_agent='Mozilla/5.0 (X11; Linux i686) AppleWebKit/5331 (KHTML, like Gecko) Chrome/40.0.887.0 Mobile Safari/5331'
timestamp='04/Oct/2025:10:27:22 +0200' ip='143.72.103.135' request='HEAD /scale HTTP/1.0'
method='HEAD' path='/scale' protocol='HTTP/1.0' status=503 bytes=92765
referer='http://www.forwardrecontextualize.org/orchestrate/mesh/convergence/sexy'
user_agent='Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_9_2) AppleWebKit/5341 (KHTML, like Gecko) Chrome/36.0.843.0 Mobile Safari/5341'
timestamp='04/Oct/2025:10:27:22 +0200' ip='211.131.195.126' user='waters4461'
request='PATCH /unleash/experiences/exploit/portals HTTP/1.1' method='PATCH'
path='/unleash/experiences/exploit/portals' protocol='HTTP/1.1' status=400 bytes=74772
referer='https://www.lead24/7.com/communities/magnetic/embrace/technologies'
user_agent='Mozilla/5.0 (X11; Linux i686; rv:6.0) Gecko/2016-01-01 Firefox/35.0'
timestamp='04/Oct/2025:10:27:22 +0200' ip='75.163.203.86' user='crona8101'
request='POST /deploy HTTP/2.0' method='POST' path='/deploy' protocol='HTTP/2.0' status=406
bytes=36523
referer='https://www.internalinfrastructures.biz/e-services/distributed/dot-com/evolve'
user_agent='Opera/10.51 (X11; Linux x86_64; en-US) Presto/2.13.302 Version/11.00'
timestamp='04/Oct/2025:10:27:22 +0200' ip='14.84.146.92' user='dickinson1700'
request='HEAD /seamless/world-class/cutting-edge HTTP/1.1' method='HEAD'
path='/seamless/world-class/cutting-edge' protocol='HTTP/1.1' status=403 bytes=89978
referer='http://www.investorbenchmark.com/technologies/e-business/extend'
user_agent='Mozilla/5.0 (Macintosh; PPC Mac OS X 10_7_6) AppleWebKit/5312 (KHTML, like Gecko) Chrome/39.0.861.0 Mobile Safari/5312'
timestamp='04/Oct/2025:10:27:22 +0200' ip='174.54.227.185' request='PATCH /exploit HTTP/1.1'
method='PATCH' path='/exploit' protocol='HTTP/1.1' status=205 bytes=70039
referer='http://www.globalinnovate.org/whiteboard/wireless/benchmark/models'
user_agent='Opera/8.15 (Windows NT 6.0; en-US) Presto/2.8.336 Version/11.00'
timestamp='04/Oct/2025:10:27:22 +0200' ip='121.39.244.166' user='morar5336'
request='GET /experiences/robust HTTP/2.0' method='GET' path='/experiences/robust'
protocol='HTTP/2.0' status=205 bytes=27953
referer='https://www.districtintegrated.net/mesh/visualize'
user_agent='Mozilla/5.0 (X11; Linux i686) AppleWebKit/5312 (KHTML, like Gecko) Chrome/36.0.867.0 Mobile Safari/5312'
timestamp='04/Oct/2025:10:27:22 +0200' ip='248.128.143.72'
request='POST /vortals/collaborative/partnerships/action-items HTTP/1.0' method='POST'
path='/vortals/collaborative/partnerships/action-items' protocol='HTTP/1.0' status=403 bytes=26871
referer='http://www.dynamicturn-key.org/mission-critical/morph/vortals'
user_agent='Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_8_7 rv:4.0; en-US) AppleWebKit/533.1.8 (KHTML, like Gecko) Version/6.1 Safari/533.1.8'
timestamp='04/Oct/2025:10:27:22 +0200' ip='99.37.61.28'
request='GET /plug-and-play/functionalities/efficient HTTP/1.1' method='GET'
path='/plug-and-play/functionalities/efficient' protocol='HTTP/1.1' status=404 bytes=77884
referer='https://www.legacykiller.org/disintermediate'
user_agent='Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_5_1 rv:5.0; en-US) AppleWebKit/535.40.6 (KHTML, like Gecko) Version/4.1 Safari/535.40.6'
kelora: Tracked metrics:
status_100 = 1
status_200 = 3
status_201 = 4
status_203 = 2
status_204 = 3
status_205 = 2
status_301 = 3
status_304 = 2
status_400 = 3
status_403 = 5
status_404 = 2
status_406 = 2
status_416 = 2
status_500 = 1
status_501 = 1
status_502 = 1
status_503 = 2
status_504 = 1
Top IPs by Request Count¶
Analyze Specific Endpoints¶
kelora -f combined /var/log/nginx/access.log \
--filter 'e.path.contains("/api/")' \
-e 'track_count(e.path)' \
--metrics
Find Suspicious Activity¶
Look for unusual patterns:
# High request rates from single IP
kelora -f combined /var/log/nginx/access.log \
-e 'track_count(e.ip)' \
--metrics
# POST requests to unusual paths
kelora -f combined /var/log/nginx/access.log \
--filter 'e.method == "POST" && !e.path.starts_with("/api/")' \
-k ip,timestamp,method,path
# Large response sizes
kelora -f combined /var/log/nginx/access.log \
--filter 'e.get_path("bytes", "0").to_int() > 10000000' \
-k ip,path,bytes,timestamp
Time-Based Analysis¶
Analyze traffic in specific time windows:
# Last hour's errors
kelora -f combined /var/log/nginx/access.log \
--since "1 hour ago" \
--filter 'e.status >= 400'
# Traffic during specific time range
kelora -f combined /var/log/nginx/access.log \
--since "2024-01-15 09:00:00" \
--until "2024-01-15 17:00:00" \
-e 'track_count(e.status)' \
--metrics
Response Time Percentiles¶
Calculate performance metrics for Nginx logs with request_time:
kelora -f combined /var/log/nginx/access.log \
-e 'track_bucket("latency", floor(e.get_path("request_time", "0").to_float() * 1000 / 100) * 100)' \
--metrics
Real-World Examples¶
Daily Error Report¶
kelora -f combined /var/log/nginx/access.log* \
--filter 'e.status >= 400' \
-e 'e.hour = e.timestamp.extract_re(r"(\d{2}):\d{2}:\d{2}", 1)' \
-e 'track_count(e.hour)' \
-e 'track_count(e.status)' \
--metrics
API Endpoint Performance¶
kelora -f combined /var/log/nginx/access.log \
--filter 'e.path.starts_with("/api/")' \
-e 'e.endpoint = e.path.extract_re(r"(/api/[^/]+)", 1)' \
-e 'track_count(e.endpoint)' \
-e 'track_avg(e.endpoint, e.get_path("request_time", "0").to_float())' \
--metrics
Bot Detection¶
kelora -f combined /var/log/nginx/access.log \
--filter 'e.user_agent.contains("bot") || e.user_agent.contains("crawler")' \
-e 'track_count(e.user_agent)' \
-k ip,user_agent,path \
--metrics
Referer Analysis¶
Find where traffic is coming from:
kelora -f combined /var/log/nginx/access.log \
--filter 'e.referer != "-" && !e.referer.contains("yourdomain.com")' \
-e 'e.domain = e.referer.extract_domain()' \
-e 'track_count(e.domain)' \
--metrics
Failed Authentication Attempts¶
kelora -f combined /var/log/nginx/access.log \
--filter 'e.path.contains("/login") && e.status == 401' \
-e 'track_count(e.ip)' \
-k timestamp,ip,path,status \
--metrics
Response Size Distribution¶
kelora -f combined /var/log/nginx/access.log \
-e 'e.size_kb = floor(e.get_path("bytes", "0").to_int() / 1024)' \
-e 'track_bucket("response_size_kb", e.size_kb)' \
--metrics
Export for Analysis¶
Export to CSV¶
kelora -f combined /var/log/nginx/access.log \
-k ip,timestamp,status,bytes,request \
-F csv > access.csv
Export to JSON¶
Performance Tips¶
Large Files:
# Use parallel processing
kelora -f combined /var/log/nginx/access.log.* \
--parallel \
--filter 'e.status >= 500'
# Limit output
kelora -f combined access.log -n 1000
Gzipped Archives:
# Kelora handles .gz automatically
kelora -f combined /var/log/nginx/access.log.*.gz \
--filter 'e.status >= 500'
Multiple Files:
# Process all access logs
kelora -f combined /var/log/nginx/access.log* \
-e 'track_count(e.status)' \
--metrics
Common Patterns¶
Find top N IPs by error count:
Hourly request distribution:
kelora -f combined access.log \
-e 'e.hour = e.timestamp.extract_re(r"(\d{2}):\d{2}:\d{2}", 1)' \
-e 'track_count(e.hour)' \
--metrics
Method distribution:
Status code summary:
kelora -f combined access.log \
-e 'e.status_class = floor(e.status / 100) + "xx"' \
-e 'track_count(e.status_class)' \
--metrics
Troubleshooting¶
Timestamp parsing issues:
# If auto-detect misses, inspect the stats line:
# Timestamp: auto-detected timestamp — parsed 0 of 100 detected events (0.0%). Hint: Try --ts-field or --ts-format.
# Then supply an explicit format:
kelora -f combined --ts-format "%d/%b/%Y:%H:%M:%S %z" access.log
Missing request_time field:
# Apache combined format doesn't include request_time
# Only Nginx with custom log format includes it
# Use safe access with get_path()
e.get_path("request_time", "0")
Large numbers in bytes field:
See Also¶
- Find Errors in Logs - General error finding techniques
- Monitor Application Health - Application-level monitoring
- Function Reference - All available functions
- Concepts: Pipeline Model - How processing works