Find the Files Eating Your Disk
The disk was full, but guessing at folders was the slow part.
find /var -type f -printf '%s %p\n' | sort -nr | head -20
topic
Linux Survival Basics
The commands to inspect a machine before guessing.
110 checked fixes
Commands in this topic
Watch Logs Without Opening the Whole File
The app was failing now. Opening a giant log file was the wrong move.
tail -n 80 -f /var/log/nginx/error.log
Find Errors Before Reading Every Log Line
The error was in the log. The problem was finding it without reading noise.
grep -iE 'error|failed|denied|timeout' /var/log/nginx/error.log | tail -40
Find the Exact Log Line Before You Scroll
The error was there. The useful part was knowing exactly where it was.
grep -inE 'error|failed|denied|timeout' /var/log/nginx/error.log
Find Which Folder Is Filling the Disk
The disk was full. The fastest clue was the folder, not the file.
du -sh /var/* 2>/dev/null | sort -h
Show Only Recent Errors
The log had old failures too. I only cared about the newest ones.
grep -iE 'error|failed|denied|timeout' /var/log/nginx/error.log | tail -10
Check Owner and Mode in One Line
The file existed. The owner and mode explained why it still failed.
stat -c '%A %U:%G %n' /var/www/example/index.html
Find the Processes Using Memory
The server felt slow. Memory pressure was the first thing to rule out.
ps -eo pid,comm,%mem,%cpu --sort=-%mem | head
Show Big Files in Human Units
Byte counts are precise. Human units are faster under pressure.
find /var -type f -printf '%s %p\n' | sort -nr | head -10 | awk '{printf "%.1f MB %s\n", $1/1024/1024, $2}'
List Contents of a Backup Tarball
You can inspect an archive without extracting it.
tar -tf archives/site-backup.tar | sort | head
Count Source Files by Extension
A quick extension count can show whether expected content made it into the source tree.
find source -type f -printf '%f\n' | sed -n 's/.*\.//p' | sort | uniq -c | sort -nr
Fingerprint a Debian or Ubuntu Host
Before package triage, prove what OS family and release you are actually on.
. /etc/os-release && printf '%s %s %s\n' "$ID" "$VERSION_ID" "$VERSION_CODENAME"
Compare Kernel and Distro Versions
The distro version and kernel version answer different questions.
printf 'kernel=%s arch=%s distro=%s\n' "$(uname -r)" "$(uname -m)" "$(lsb_release -ds)"
List Installed Package Versions
A package inventory beats memory when a server is drifting.
dpkg-query -W -f='${Package}\t${Version}\t${Architecture}\n' | sort
See Which Packages Want Updates
Before you upgrade anything, list what would move.
apt list --upgradable
Check the Installed and Candidate Package Version
apt policy explains where the next version would come from.
apt policy nginx
Check One Installed Package Cleanly
For one package, dpkg-query gives a clean status line.
dpkg-query -W -f='${Status} ${Version}\n' openssl
Find Which Package Owns a File
That binary came from somewhere. dpkg can tell you where.
dpkg-query -S /usr/sbin/nginx
Find Broken or Leftover dpkg States
Not every package row is cleanly installed.
dpkg-query -W -f='${db:Status-Abbrev}\t${Package}\n' | awk '$1 !~ /^ii$/'
Find the Largest Installed Packages
Disk cleanup starts with evidence, not random package removal.
dpkg-query -W -f='${Installed-Size}\t${Package}\n' | sort -nr | head -20
Spot Foreign-Architecture Packages
One unexpected architecture can explain confusing dependency output.
dpkg-query -W -f='${Architecture}\t${Package}\n' | awk '$1 != "amd64" && $1 != "all"'
Find the Largest CI Logs
Huge logs often point to loops, noisy tests, or runaway debug output.
find logs/ -type f -printf '%s %p\n' | sort -nr | head -10
Count Failures by Test File
Turn noisy test logs into a ranked failure list.
grep -RhoE '[A-Za-z0-9_./-]+\.(test|spec)\.(js|ts|py|rb)' logs/ | sort | uniq -c | sort -nr | head
Read TLS Certificate Subject and Issuer
The certificate can be valid but issued for the wrong name.
openssl s_client -connect example.com:443 -servername example.com </dev/null 2>/dev/null | openssl x509 -noout -subject -issuer -dates
Find Nginx SSL Certificate Directives
The wrong certificate is often in the server block.
grep -RInE 'ssl_certificate|ssl_certificate_key|server_name' /etc/nginx/sites-enabled /etc/nginx/conf.d 2>/dev/null
Test Apache Config Without Reloading
Apache can check syntax before you reload a live site.
sudo apachectl configtest
Find Apache DocumentRoot and Directory Rules
Apache 403 often comes from the directory block, not the file.
grep -RInE 'DocumentRoot|<Directory|Require all|Options|AllowOverride' /etc/apache2/sites-enabled /etc/apache2/conf-enabled /etc/httpd/conf.d 2>/dev/null
Read Recent Apache Error Log Lines
The Apache error log usually names the denied directory or rule.
sudo tail -80 /var/log/apache2/error.log 2>/dev/null || sudo tail -80 /var/log/httpd/error_log
List Docker Container Restart Status
Restart loops are visible before rebuilding anything.
docker ps -a --format 'table {{.Names}} {{.Image}} {{.Status}} {{.RestartCount}}'
Read Recent Docker Container Logs
Recent container logs usually show the failing command or dependency.
docker logs --tail 80 container_name
Inspect Docker Restart Policy
Policy and exit code explain whether Docker is restarting by design.
docker inspect -f '{{.HostConfig.RestartPolicy.Name}} {{.State.ExitCode}} {{.State.Error}}' container_name
List systemd Timers and Last Runs
A timer can be inactive, missed, or waiting for the wrong calendar.
systemctl list-timers --all --no-pager
Read One systemd Timer Status
Timer status shows whether the schedule is loaded and active.
systemctl status backup.timer --no-pager
Show systemd Timer Unit File
The calendar expression lives in the timer unit.
systemctl cat backup.timer
Find SSH Too Many Authentication Failures Lines
The auth log proves whether the server refused after too many offered keys.
grep -i 'Too many authentication failures' /var/log/auth.log /var/log/secure 2>/dev/null | tail -20
List SSH Agent Keys
Too many loaded keys can cause the client to offer the wrong identities first.
ssh-add -l
Test SSH with One Identity File
Force one key when the agent offers too many identities.
ssh -o IdentitiesOnly=yes -i ~/.ssh/id_ed25519 user@example.com
Show Served Certificate SANs
SANs decide which hostnames the certificate covers.
openssl s_client -connect example.com:443 -servername example.com </dev/null 2>/dev/null | openssl x509 -noout -ext subjectAltName
List Enabled Apache Sites
Apache may not be using the vhost file you edited.
find /etc/apache2/sites-enabled -maxdepth 1 -type l -printf '%f -> %l
' 2>/dev/null | sort
Read Recent Docker Restart Events
Docker events show restart and die loops over time.
docker events --since 30m --until 0s
Read Recent systemd Timer Logs
Timer logs show whether systemd attempted to trigger the task.
journalctl -u backup.timer --since "24 hours ago" --no-pager
Run SSH Verbose Auth Test
Verbose SSH shows which key was offered.
ssh -vvv -o IdentitiesOnly=yes -i ~/.ssh/id_ed25519 user@example.com
Show HTTPS Certificate with curl
curl can show the certificate path a client actually sees.
curl -Iv https://example.com/ 2>&1 | sed -n "/SSL connection/,/expire date/p"
Show Context Around the First App Error
The first error often explains more than the last one.
awk '{buf[NR%5]=$0} tolower($0) ~ /(error|exception|fatal)/ {for (i=NR-4;i<=NR;i++) if (i>0) print buf[i%5]; exit}' /var/log/app/app.log
Spot OOM Kills in the Kernel Journal
Exit code 137 often means the kernel has something to say.
journalctl -k --since "2 hours ago" --no-pager -o short-iso | grep -Ei 'out of memory|oom|killed process'
Test Nginx Config Without Reloading
A config test is safer than a blind reload.
sudo nginx -t
Trace Nginx Web Path Permissions
A 403 is often in a parent directory, not the file.
namei -l /srv/www/site/index.html
Check Web File Owner and Mode
The file can exist and still be unreadable to Nginx.
stat -c '%A %U:%G %n' /srv/www/site/index.html
Find Nginx root alias and access rules
One deny or alias can explain the whole 403.
grep -RInE 'root|alias|deny|allow' /etc/nginx/sites-enabled /etc/nginx/conf.d 2>/dev/null
Read Recent Nginx Error Log Lines
The error log often names the denied path.
sudo tail -80 /var/log/nginx/error.log
Run Certbot Renewal Dry Run
Certbot has a safe renewal rehearsal.
sudo certbot renew --dry-run
Inspect Certbot Certificate Inventory
Know which names Certbot thinks it manages.
sudo certbot certificates
Read Recent LetsEncrypt Log Lines
Certbot logs usually name the failed challenge.
sudo tail -80 /var/log/letsencrypt/letsencrypt.log
Check Domain A Records
A renewal can fail because DNS points elsewhere.
dig +short example.com A
Show Linux Route Table
No default route means packets have nowhere to leave.
ip route
Show Interface Addresses
A route problem may actually be an address problem.
ip addr
Show NetworkManager Device State
NetworkManager may know why the interface is disconnected.
nmcli device status
Read systemd-resolved Status
DNS state can distract from route failures.
resolvectl status
Debug Logrotate Without Rotating
Use debug mode before forcing rotation.
logrotate -d /etc/logrotate.conf
Check logrotate Timer Status
The timer may be disabled, missed, or failing.
systemctl status logrotate.timer --no-pager
Read Recent Logrotate Journal
The journal can show why rotation skipped.
journalctl -u logrotate --since '7 days ago' --no-pager
Check /boot Disk Usage
A full /boot blocks kernel updates.
df -h /boot
List Installed Debian Kernels
Know installed kernels before cleanup.
dpkg -l 'linux-image*' | awk '/^ii/{print $2,$3}'
List Installed RPM Kernels
RPM systems keep multiple kernels too.
rpm -q kernel
Show Running Kernel Version
Never delete the kernel you are running.
uname -r
Find a Host in known_hosts
Do not edit known_hosts blind.
ssh-keygen -F hostname
Scan a Host SSH Key
Keyscan collects a presented key; it does not verify trust.
ssh-keyscan -t ed25519 hostname
Find known_hosts Lines by Name
See the stored entry before editing.
grep -n 'hostname' ~/.ssh/known_hosts
Trace Every Parent Directory on a Permission Denial
The file mode can look fine while a parent directory blocks the whole path.
namei -l /srv/www/example/current/config/prod.token
Check Memory Pressure with free
Linux memory numbers look scary until you know which column matters.
free -h
Read Load Average Before You React
A high load number is a clue, not a diagnosis.
uptime
Show the Real User Cron Jobs
Cron problems often hide behind comments, blank lines, and copied folklore.
crontab -l | sed -n '/^[[:space:]]*#/d;/^[[:space:]]*$/d;p'
Turn Cron Into a Readable Table
Cron is easier to debug when the schedule and command stop blending together.
crontab -l | awk 'NF && $1 !~ /^#/ {printf "%-16s %s\n", $1" "$2" "$3" "$4" "$5, substr($0,index($0,$6))}'
Map systemd Timers to Services
A timer is only half the scheduled job. The service is the payload.
systemctl list-timers --all --no-pager --plain | awk 'NR==1 || /\.timer/ {print $(NF-1), "->", $NF}'
Find Running Package Manager Processes
A package lock is usually a symptom, not the first thing to delete.
ps -ef | grep -E 'apt|dpkg|dnf|yum|pacman' | grep -v grep
Find the dpkg Lock Owner
Find the process holding the dpkg lock before touching lock files.
sudo lsof /var/lib/dpkg/lock-frontend /var/lib/dpkg/lock 2>/dev/null
Check apt Daily Timers
Automatic apt timers can explain locks that seem mysterious.
systemctl list-timers apt-daily* --no-pager
Show Hidden CRLF in a Script Shebang
The hidden carriage return is often the whole bug.
head -1 script.sh | cat -v
Detect Script Line Endings with file
Confirm CRLF before converting anything.
file script.sh
Find CRLF Lines in a Script
Find exactly which lines still contain carriage returns.
grep -n $'\r' script.sh | head
Verify fstab with findmnt
Bad fstab entries should be found before the next reboot.
findmnt --verify
Show Failed Mount Units
systemd often records mount failures as failed units.
systemctl --failed --no-pager
Read Mount and fstab Warnings
The boot journal often names the mount or dependency that failed.
journalctl -b -p warning --no-pager | grep -iE 'mount|fstab|dependency'
Find OOM Killer Lines in the Kernel Journal
Before restarting a service, prove whether the kernel killed it.
journalctl -k --since '24 hours ago' --no-pager | grep -iE 'out of memory|oom-killer|killed process'
Take a Memory Pressure Snapshot
Measure memory and swap before changing service limits.
free -h
Show Top Memory Processes
Find current memory owners before restarting workloads.
ps -eo pid,comm,%mem,%cpu --sort=-%mem | head
Check a User Identity Record
Start sudo troubleshooting with the actual target account.
id username
Show User Groups
Group membership explains many sudo and access failures.
groups username
List Effective sudo Privileges
sudo -l shows policy; guessing invites bad access changes.
sudo -l
List sudo Group Members
Before granting sudo, see who already has it.
getent group sudo
Count Open Files for One Process
Count handles for the target process before raising limits.
sudo lsof -p 1234 | wc -l
Show a Service LimitNOFILE
A shell ulimit is not the service limit.
systemctl show nginx -p LimitNOFILE --no-pager
Inspect One Process Open Files
Look at one target process, not the whole host, when pressure is scoped.
sudo lsof -p 1234 | head
List Tables in a SQLite Database
Before querying a database file, see what tables are actually inside it.
sqlite3 app.db ".tables"
List URLs from a Sitemap
Before comparing sitemap coverage, print the URLs plainly.
grep -o '<loc>[^<]*</loc>' public/sitemap.xml | sed 's#<loc>##;s#</loc>##'
Show Failed systemd Units
One command tells you which services systemd already knows are broken.
systemctl --failed --no-pager
Inspect One Service Without Pager Traps
Make systemctl status safe for scripts, screenshots, and quick incident notes.
systemctl status nginx --no-pager --lines=30
Read Current-Boot Logs for One Service
Ignore stale logs and inspect only what happened since this boot.
journalctl -u nginx -b --no-pager -n 80
Check systemd Journal Disk Usage
Before deleting random logs, ask journald how much disk it owns.
journalctl --disk-usage
Find Slow Services During Boot
Find which units made your VPS boot slowly.
systemd-analyze blame | head -20
Check Whether a Service Starts at Boot
Running now does not mean it will survive the next reboot.
systemctl is-enabled nginx
Check If a Service Is Active
Get a clean yes-or-no service state without the full status page.
systemctl is-active nginx
Show Recent Server Reboots
Confirm whether the server actually rebooted and when.
last -x reboot | head -5
Check Memory Pressure Quickly
See whether memory is actually tight before restarting services.
free -h
List Upcoming systemd Timers
Cron is not the only scheduler on modern Linux servers.
systemctl list-timers --all --no-pager
Read the Failure Cause in systemctl Status
The status page often tells you the failed startup step before you open every log.
systemctl status app-worker --no-pager --lines=50
Print the Exact systemd Exit Fields
Turn a noisy service failure into four fields you can paste into an incident note.
systemctl show app-worker --property=Result,ExecMainCode,ExecMainStatus,NRestarts --no-pager
Inspect the Unit File and Drop-ins Together
The bug may be in an override file, not the main unit.
systemctl cat app-worker
Reset Failed State After Capturing Evidence
Clear the red failed state only after you have captured the evidence.
systemctl reset-failed app-worker
Compare Failure Output With the Effective Unit
Put the failed step next to the unit config that created it.
systemctl status app-worker --no-pager --lines=50 && systemctl cat app-worker