Hosting Operations
Read-onlyFind Top 404 URLs
You need to see which URLs are producing 404 responses.
Command
awk '$9==404 {print $7}' /var/log/nginx/access.log | sort | uniq -c | sort -nr | headBefore you run this
System impact: Read-only. Low when scoped to the shown target.
When not to use it: Do not assume every 404 is a bug; scanners and bots create noise.
Expected output
A ranked list of 404 paths.
System impact
Read-only. Nothing changes. The command counts missing paths.
May require elevated permissions on protected paths or service-owned files.
Recovery / rollback: no state is changed.
When to use it
Use this after deploys, migrations, or SEO cleanup.
When not to use it
Do not assume every 404 is a bug; scanners and bots create noise.
Watch this command run
Command transcript
This sanitized transcript shows the commands and output shape without exposing host details.
$ awk '{print $9, $7}' /var/log/nginx/access.log
200 /
404 /missing.css
502 /api$ awk '$9==404 {print $7}' /var/log/nginx/access.log | sort | uniq -c | sort -nr | head
1 /missing.css$ awk '$9 ~ /^5/ {print $7}' /var/log/nginx/access.log | sort | uniq -c | sort -nr | head
1 /apiView commands shown
These are the commands shown in the sanitized transcript.
Commands shown
awk '{print $9, $7}' /var/log/nginx/access.logawk '$9==404 {print $7}' /var/log/nginx/access.log | sort | uniq -c | sort -nr | headawk '$9 ~ /^5/ {print $7}' /var/log/nginx/access.log | sort | uniq -c | sort -nr | head
next steps
Related commands
See Top Referrers
LinkedIn traffic was not a guess. The referrer field showed it.
awk -F'"' '{print $4}' /var/log/nginx/access.log | sort | uniq -c | sort -nr | head
Find Unusually Large Web Responses
A few huge responses can explain bandwidth, latency, and suspicious download patterns.
awk '$10 ~ /^[0-9]+$/ && $10 > 1000000 {print $10, $1, $7, $9}' ./fixtures/nginx/access.log | sort -nr | head
Group Server Errors by URL Path
A 500 spike is easier to triage when the broken path is obvious.
awk '$9 ~ /^5/ {count[$7]++} END {for (path in count) print count[path], path}' ./fixtures/nginx/access.log | sort -nr | head
Summarize HTTP Status Codes
Before chasing individual lines, get the shape of the whole log.
awk '{count[$9]++} END {for (code in count) print count[code], code}' ./fixtures/nginx/access.log | sort -nr
Find Common Admin Probe Paths
A site does not need WordPress to receive WordPress-looking probes.
awk '$7 ~ /(admin|login|wp-|phpmyadmin)/ {print $1, $7, $9}' ./fixtures/nginx/access.log | sort | uniq -c | sort -nr | head
Study mapping
Use this as independent command practice: read the notes, predict the output, then compare it with the example before using a real shell.
Useful for
- LPIC-1 style command-line practice
- LFCS style performance tasks
- Linux+ style troubleshooting review
Independent study support only. No affiliation, endorsement, exam dumps, or real exam questions.