What to Check in Server Inspections: A Ready-to-Use Checklist (with Commands and Interpretation)

The value of an inspection lies not in “having a look,” but in identifying trends before a failure occurs. The disk usage increasing from 60% to 90% is a process; waiting until it hits 100% to act turns it into an incident.

Below is a ready-to-execute inspection checklist. Each item includes the command + criteria for what constitutes an anomaly. Simply “glancing at CPU” is meaningless without interpretation standards.

Inspection Checklist

Item-by-Item Explanation

Disk and Inode

df -h                 # Space
df -i                 # Inode (exhausted first with many small files)

# Find large directories consuming space
du -sh /var/* 2>/dev/null | sort -rh | head -10

# Find large files
find / -type f -size +500M -exec ls -lh {} ; 2>/dev/null | head

Inode is the most easily overlooked item. Its characteristic is that df -h shows there is still space, but files cannot be written. A large number of small files (sessions, cache fragments, log rotations) can consume all inodes.

Memory and Load

free -m
uptime
# The three load average values represent 1/5/15 minutes
# Criterion: Consistently higher than the number of CPU cores indicates queuing
nproc                 # View core count

Load must be interpreted relative to core count. On a 4-core machine, a load of 8 is overloaded; on a 16-core machine, a load of 8 is normal.

Services and Logs

# Check for failed services
systemctl --failed

# System error logs
journalctl -p err --since '24 hours ago' --no-pager

# Web and PHP error logs
tail -50 /var/log/nginx/error.log
tail -50 /var/log/php-fpm/error.log

Certificate Validity

# Local certificate file
openssl x509 -in /etc/nginx/ssl/example.com/fullchain.pem -noout -dates

# Online check (view the actually effective one)
openssl s_client -connect example.com:443 -servername example.com </dev/null 2>/dev/null 
  | openssl x509 -noout -dates

# Batch check multiple domains
for d in a.com b.com c.com; do
  echo -n "$d  "
  openssl s_client -connect $d:443 -servername $d </dev/null 2>/dev/null 
    | openssl x509 -noout -enddate
done

Certificate expiration is a completely avoidable failure because it is predictable. Adding it to the inspection checklist costs almost nothing.

Backup Availability

# When was the most recent backup?
find /data/backup -maxdepth 1 -type d | sort | tail -3

# Is the size normal?
ls -lh /data/backup/$(date +%F)/ 2>/dev/null

# Integrity (gzip files can be verified)
gzip -t /data/backup/latest_date/db.sql.gz && echo OK

When inspecting backups, do not just check if the file exists. There have been cases where backup scripts failed but produced empty files, going unnoticed for months. Checking size and integrity is what matters.

Automation: Output Inspection Results in One Go

#!/bin/bash
# /root/inspect.sh
echo "===== Inspection $(date +%F) ====="
echo "--- Disk ---";      df -h | grep -vE 'tmpfs|overlay'
echo "--- Inode ---";     df -i | grep -vE 'tmpfs|overlay'
echo "--- Memory ---";      free -m
echo "--- Load ---";      uptime
echo "--- Failed Services ---";  systemctl --failed --no-pager
echo "--- Recent Backups ---";  find /data/backup -maxdepth 1 -type d | sort | tail -3
echo "--- Certificates ---"
for d in example.com; do
  openssl s_client -connect $d:443 -servername $d </dev/null 2>/dev/null 
    | openssl x509 -noout -enddate | sed "s/^/$d  /"
done
echo "===================="

chmod +x /root/inspect.sh
/root/inspect.sh | tee -a /var/log/inspect.log

# Add to cron job to run automatically every day
0 9 * * * /root/inspect.sh >> /var/log/inspect.log 2>&1

Decision Tree

Anomaly Detected During Inspection
├─ Disk > 80%
│  ├─ Use du to find large directories → Which are growing? Logs/Backups/Uploads
│  └─ If logs   → Configure logrotate; do NOT directly rm logs currently being written
├─ Inode nearly full
│  └─ Find directories with high file counts → Sessions/Cache fragments
├─ Sustained High Load
│  ├─ Determine if CPU or IO bound (check wa in top)
│  └─ Judge relative to core count, do not look at absolute values
├─ Failed Services Present
│  └─ Check journalctl for specific reasons; do not blindly restart
├─ Certificate Remaining < 30 Days
│  └─ Verify if auto-renewal is working correctly (manual renewal if not)
└─ Backup Anomaly
   └─ Check backup logs + verify file size/integrity

Scenario Analysis: Inspected for Half a Year, Yet Missed All Real Issues

Phenomenon: An inspection regime was executed for half a year, with complete weekly records.
However, two incidents that occurred during this period showed no warning signs in the inspection records.

1. Compare inspection items with the actual root causes of the two incidents → The root causes were disk growth trends and certificate expiration, while the checklist only contained immediate items like “Is current usage normal?” and “Can the site be accessed?”.
2. Identify the commonality: Inspections only recorded the current state → Snapshot checks can detect things that have already happened,
   but cannot detect accumulating trends.
3. Change key items to focus on trends → For items like disk space, connection counts, and slow request counts, what matters is the rate of change,
   not the current value.
4. Add checks for “Did it succeed?” → Backups and renewals are failures that don’t expose themselves easily;
   you must check execution results rather than just file existence.
5. Add time-dimension items like certificate and domain expiration → These show no symptoms before expiration;
   they can only be detected through lead time.
6. Make inspection results comparable data → Archive each output and calculate the difference from the previous one;
   trends will reveal themselves.

# Trend-based: Take two snapshots to calculate the difference
df -h | awk 'NR>1 {print $6, $5}' | tee disk-$(date +%F).txt
# Compare with last week to see growth speed, not just current percentage

# Success-based: Did the backup produce a valid file in the last day?
find /backup_dir -name '*.sql.gz' -mtime -1 | wc -l

# Time-dimension based: How many days left on the certificate?
echo | openssl s_client -connect example.com:443 -servername example.com 2>/dev/null 
  | openssl x509 -noout -enddate

# Check if restarted secretly (if uptime is very short but no changes were made, be vigilant)
ps -eo pid,etime,cmd --sort=start_time | head -5
last reboot | head -3

Root Cause: The inspection items were all snapshots of the “current state”,
while both incidents were cumulative problems—any instantaneous check performed before the incident would have appeared normal.

Two Key Takeaways:

• Effective inspections must distinguish between three types of items: trend monitoring, success verification, and remaining time tracking.
  Relying solely on “Is it currently normal?” will cause all slowly accumulating issues to be missed.
• Inspection records must be archived and comparable.
  A single snapshot cannot show growth; calculating the difference between two or more snapshots reveals the trend.

8 Common Pitfalls

1. Checking disk space but ignoring inodes: When inodes are exhausted, space may remain, but writing fails.
2. Ignoring core count when checking load: Absolute values are meaningless.
3. Inspecting without recording: Without historical data, trends cannot be identified.
4. Checking backup existence only: Empty files also “exist”; verify size.
5. Directly rm-ing logs currently being written: Space is not released immediately, and applications may error. Use logrotate or truncate.
6. Restarting immediately upon finding a failed service: This masks the cause and leads to recurrence.
7. Excluding certificates from inspections: Expiration is a fully predictable failure; adding it to the checklist prevents it.
8. No one reviews inspection results: Either set up alerts or periodically review logs; otherwise, the effort is wasted.

Real Customer Feedback (From Cloud Market Reviews and Consultations for Similar Products)

The following excerpts are taken from real buyer reviews and consultations for similar operations services on Alibaba Cloud / Tencent Cloud / Huawei Cloud Cloud Markets (service providers include our store and peers, provided here as reference for genuine feedback, not fictional tickets from this site):

1****1 Rating: 5.0 2020-06-18 11:51 Many thanks! He**** She Rating: 5.0 2020-04-21 15:04 Boss is great! Configuration was fast! No pauses in between! Very good!

b****4 Rating: 5.0 2025-06-23 16:57 b****4 Rating: 5.0 2025-06-23 16:56 Jian**** Wei Rating: 5.0 2021-01-20 15:03 Technology is awesome, speed is fast! It turned out to be a trojan causing Baidu global hijacking!!!!1 f****u Rating: 5.0 2021-01-02 20:43 Thanks for assisting in maintaining Yuncai’s normal operation. 1****1 Rating: 5.0 2020-11-20 10:03 As always, great service. 1****1 Rating: 5.0 2020-11-17 09:13 Thanks for the consistent great service. 1****1 Rating: 5.0 2020-09-25 15:42 Many thanks. 1****1 Rating: 5.0 2020-08-24 15:34 Many thanks to your company’s technical staff. 1****1 Rating: 5.0 2020-07-24 09:49 Many thanks! 1

b****4 Rating: 5.0 2024-07-25 10:45 b****4 Rating: 5.0 2024-06-28 10:40 b****4 Rating: 5.0 2024-06-28 10:39 b****4 Rating: 5.0 2024-05-30 13:17 w****1 Rating: 5.0 2023-03-22 13:53 a****7 Rating: 5.0 2023-02-15 09:12 h****4 Rating: 5.0 2022-02-27 20:26 Jian**** Wei Rating: 5.0 2021-01-20 15:03 h****4 Rating: 5.0 2020-08-04 09:27