Identifying Anomalous Access: High-Frequency IPs, Scanning Behavior, and Slow Requests

Access logs are the most easily overlooked troubleshooting resource—they record every request, and anomalies have a distinct shape in logs: high-frequency IPs, concentrated URLs, abnormal status codes, and suddenly slow response times.

This assumes the logs contain sufficient information. The default log format is usually adequate, but adding response time makes it significantly more useful.

First, Make Your Logs More Useful

# Define a format with response time in the http block
log_format main_ext '$remote_addr - $remote_user [$time_local] "$request" '
                    '$status $body_bytes_sent "$http_referer" '
                    '"$http_user_agent" rt=$request_time urt=$upstream_response_time';

access_log /var/log/nginx/access.log main_ext;

nginx -t && nginx -s reload

The fields $request_time (total time) and $upstream_response_time (backend time) allow you to identify slow requests directly from the logs without needing to reproduce them.

Common Analysis Commands

① High-Frequency IPs

awk '{print $1}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head -20

# View only the recent segment (take the tail first, then count)
tail -5000 /var/log/nginx/access.log | awk '{print $1}' | sort | uniq -c | sort -rn | head

② High-Frequency URLs

awk '{print $7}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head -20

# Remove parameters to view only paths
awk '{print $7}' /var/log/nginx/access.log | cut -d'?' -f1 | sort | uniq -c | sort -rn | head

③ Status Code Distribution

awk '{print $9}' /var/log/nginx/access.log | sort | uniq -c | sort -rn

# View only errors
awk '$9 >= 400 {print $9, $7}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head -20

# Find all 404s (could be broken links or scanning)
awk '$9 == 404 {print $7}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head -20

④ Slow Requests

# Requests with response time exceeding 1 second (requires 'rt' field in logs)
awk '$NF > 1' /var/log/nginx/access.log | head -20

# Count slow requests by URL
awk '$NF > 1 {print $7}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head

# Find the 10 slowest requests
awk '{print $NF, $7}' /var/log/nginx/access.log | sort -rn | head -10

⑤ Scanning Behavior (A Highly Valuable Signal)

# Find requests probing sensitive paths
grep -iE '.env|.git|wp-login|phpmyadmin|admin|xmlrpc|.sql|.bak' 
  /var/log/nginx/access.log | awk '{print $1, $7}' | sort | uniq -c | sort -rn | head -20

# A single IP generating大量 404s in a short time → likely scanning
awk '$9 == 404 {print $1}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head -10

“A single IP generating many 404s” is a typical characteristic of scanning. Normal users do not continuously access dozens of non-existent paths. When you see this pattern, consider blocking or adding protection.

⑥ Abnormal User Agents (UA)

awk -F'"' '{print $6}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head -15

# Empty UA or obvious tool UAs
awk -F'"' '$6 == "" || $6 ~ /curl|python|scan|bot/ {print $1, $6}' 
  /var/log/nginx/access.log | sort | uniq -c | sort -rn | head

⑦ Request Volume by Time Period (Detecting Spikes)

# Count by minute (check for sudden spikes)
awk '{print substr($4, 2, 17)}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head -20

# Count by hour
awk '{print substr($4, 2, 14)}' /var/log/nginx/access.log | sort | uniq -c

Remember to Configure Log Rotation

# Confirm logrotate configuration exists
cat /etc/logrotate.d/nginx

# Large logs affect analysis speed and consume disk space
ls -lh /var/log/nginx/

If you don’t configure rotation, logs will eventually fill up your disk, and single files becoming too large will make analysis commands very slow.

Decision Tree

Analyze access log
├─ Want to check for abnormal traffic
│  ├─ High-frequency IP statistics           → Pay attention if concentrated on few IPs
│  └─ Same IP with many 404s         → Scanning behavior
├─ Want to find performance issues
│  ├─ Count slow requests by URL       → Pinpoint specific pages
│  └─ Compare rt vs upstream rt → Distinguish between nginx slowness and backend slowness
├─ Want to find errors
│  ├─ Status code distribution             → Ratio of 5xx
│  └─ Paths with concentrated 404s         → Broken links or scanning?
└─ Want to detect spikes
   └─ Count requests by minute/hour   → Identify peak times for detailed investigation

Scenario Walkthrough: Site Slows Down, Half the Requests Are Not From Real Users

Symptom: A site’s response slows down, and resource usage is high.
The traffic volume doesn’t appear large, but actual load is disproportionate to the traffic.

1. Look at the composition of requests first, not resources → Count by source address and path.
   You quickly discover that a large number of requests are concentrated on a few sources,
   and the accessed paths show obvious probing patterns.
2. Determine if these are normal accesses → Many return codes are 404,
   and request paths are common management entry points and known vulnerable paths—this is automated scanning.
3. Check time distribution to confirm nature → Requests are highly uniform and dense over time,
   which is typical of scripts, distinctly different from the sparse distribution of human access.
4. Identify repeatedly targeted objects → Count by path to see which entry points are most focused on;
   these are the targets requiring key protection.
5. Targeted handling → Rate limit or restrict by address for high-frequency sources;
   restrict source or rename management entry points.
6. Preserve evidence and archive → Archive the statistical results at that time for future rule threshold adjustments.

# Statistics by source
awk '{print $1}' access.log | sort | uniq -c | sort -rn | head -20

# Statistics by path (see which entry points they are targeting)
awk '{print $7}' access.log | sort | uniq -c | sort -rn | head -20

# Return code distribution (high 404 ratio is typical of scanning)
awk '{print $9}' access.log | sort | uniq -c | sort -rn | head

# Time distribution: Uniform and dense = script; Sparse with intervals = human
grep 'someIP' access.log | awk '{print $4}' | cut -d: -f1-3 | uniq -c | head

Root Cause: A significant portion of requests comes from automated scanning, not real access.
They are numerous and targeted, consuming resources and slowing down responses for normal users.

Two Key Takeaways:

• When “traffic is low but load is high,” the first reaction should be to count request composition.
  Count by source and path separately; abnormal access will usually reveal itself immediately.
• To judge if it’s a script, look at time distribution rather than total volume.
  Human access naturally has intervals, while scripts are uniform and dense—this rule rarely has exceptions.

Characteristics and Threshold Reference for Various Anomalies

Thresholds are not standards; they are the starting point for “worth a look”. Only you know what normal request distribution looks like.
Therefore, the prerequisite for judging anomalies is to observe data during normal periods for a few minutes to establish a baseline.
A 20% ratio might be anomalous for a personal blog but normal for an API service serving specific clients.

# Top 20 IPs by request count
awk '{print $1}' access.log | sort | uniq -c | sort -rn | head -20

# Status code distribution (check 404 rate)
awk '{print $9}' access.log | sort | uniq -c | sort -rn | head

# Find the 20 slowest requests (requires log format with $request_time)
awk '{print $NF, $7}' access.log | sort -rn | head -20

# Time distribution for a specific IP (judge if it's a script)
grep '1.2.3.4' access.log | awk '{print $4}' | cut -d: -f1-3 | uniq -c | head

The last command is crucial: Human access is sparse and spaced out in time, while scripts are dense and uniform.
Checking if an IP’s requests have a “once per second” rhythm is more effective for determining nature than looking at total volume.

8 Common Pitfalls

1. Logs don’t record response time: Cannot locate slow requests from logs; must reproduce them.
2. No log rotation configured: Disk fills up, and analysis becomes slow.
3. Counting without comparing: Without a baseline, you can’t determine what counts as anomalous.
4. Assuming many 404s are just broken links: They could also be scanning.
5. Ignoring UA analysis: Scripts and crawlers have distinct UA characteristics.
6. Misjudging source IPs when using CDN: Logs show CDN nodes; blocking them causes collateral damage.
7. Looking only at single-day data: Trends require multi-day comparisons.
8. Finding scanning but not checking success: Check if those requests returned 200.

Real Customer Feedback (From Cloud Market Reviews and Inquiries for Similar Products)

The following excerpts are taken from real buyer reviews and inquiries for similar operations services on Alibaba Cloud / Tencent Cloud / Huawei Cloud Cloud Markets (service providers include our store and peers, provided solely for reference of real feedback, not fictional tickets from this site):

10***77 2022-09-19 5.0 Affordable price | Skilled technology | Patient and professional Received filing rectification notice, found this service just in time, handled quickly 10***49 2022-04-12 5.0 On-time delivery | Affordable price | Skilled technology Needed rectification, fast configuration, good

b****4 Rating: 5.0 Points 2025-06-23 16:57 b****4 Rating: 5.0 Points 2025-06-23 16:56 Jian****Wei Rating: 5.0 Points 2021-01-20 15:03 Technology is awesome! Speed is very fast! It turned out to be a Trojan causing Baidu global hijacking!!!!1 f****u Rating: 5.0 Points 2021-01-02 20:43 Thanks for assisting in maintaining Yuncai’s normal operation. 1****1 Rating: 5.0 Points 2020-11-20 10:03 Consistently good service 1****1 Rating: 5.0 Points 2020-11-17 09:13 Thanks for the consistently good service. 1****1 Rating: 5.0 Points 2020-09-25 15:42 Very grateful. 1****1 Rating: 5.0 Points 2020-08-24 15:34 Very grateful to your company’s technical staff 1****1 Rating: 5.0 Points 2020-07-24 09:49 Very grateful!