Cybersecurity Triage
Read-onlyCount the Most Common User Agents
Traffic changed suddenly and you need a quick view of which client-reported user agents dominate the access log.
Command
awk -F'"' '{print $6}' /var/log/nginx/access.log | sort | uniq -c | sort -nr | headBefore you run this
System impact: Read-only. Low when scoped to the shown target.
When not to use it: Do not trust user agents as identity. They are client-supplied strings and can be spoofed or missing.
Expected output
A descending count of user-agent strings, useful as a lead but not as proof of who made the requests.
System impact
Read-only. Nothing changes. The command extracts the user-agent field and counts repeated values; user-agent strings may include unusual client data.
May require elevated permissions on protected paths or service-owned files.
Recovery / rollback: no state is changed.
When to use it
Use this when investigating scraping, monitoring noise, broken clients, or sudden traffic-composition changes.
When not to use it
Do not trust user agents as identity. They are client-supplied strings and can be spoofed or missing.
next steps
Related commands
Find Common Admin Probe Paths
A site does not need WordPress to receive WordPress-looking probes.
awk '$7 ~ /(admin|login|wp-|phpmyadmin)/ {print $1, $7, $9}' /var/log/nginx/access.log | sort | uniq -c | sort -nr | head
Find the IPs Creating the Most 4xx Noise
One address can turn a normal access log into a wall of failed requests.
awk '$9 ~ /^4/ {count[$1]++} END {for (ip in count) print count[ip], ip}' /var/log/nginx/access.log | sort -nr | head
Spot Unusual HTTP Methods in Access Logs
Most site traffic is boring. The weird methods are worth a look.
awk '$6 !~ /^"(GET|POST|HEAD|OPTIONS)$/ {print $1, $6, $7, $9}' /var/log/nginx/access.log | sort | uniq -c | sort -nr
Spot Request Bursts by Minute
Traffic spikes are easier to read when you bucket them by time.
awk '{minute=substr($4,2,17); count[minute]++} END {for (m in count) print count[m], m}' /var/log/nginx/access.log | sort -nr | head
Find Paths Repeatedly Returning 404
One missing URL is normal. A repeated missing URL is a signal.
awk '$9==404 {count[$7]++} END {for (path in count) if (count[path] >= 3) print count[path], path}' /var/log/nginx/access.log | sort -nr | head
Study mapping
Use this as independent command practice: read the notes, predict the output, then compare it with the example before using a real shell.
Independent study support only. No affiliation, endorsement, exam dumps, or real exam questions.