Where is the Bottleneck When Concurrency Can’t Scale: Web, Database, System, and External Dependencies

Issues like “lag spikes as traffic increases” are often not solved by simply adding resources, because the bottleneck may not be where you think it is. Adding memory won’t help with database lock waits, and increasing php-fpm processes won’t fix slow queries.

The correct approach is to first identify the bottleneck layer by layer, then optimize that specific layer.

Four Layers of Bottlenecks

Quantify First: What is Your Current Concurrency Capacity?

# Simple load test (50 concurrent users, 500 total requests)
ab -n 500 -c 50 https://example.com/

# More modern load testing tools
wrk -t4 -c100 -d30s https://example.com/

# Focus on these metrics:
# Requests per second (RPS)
# Time per request (Average latency)
# Failed requests (Crucial!)

# Monitor server resources while running load tests
top                              # CPU usage
free -m                          # Memory usage
mysqladmin processlist           # What the database is doing
ss -s                            # Total connections

You must monitor the server during load tests; otherwise, you only know it’s “slow,” but not “why.”

Web Layer: Are There Enough Processes?

# Check current process count
ps aux | grep -c '[p]hp-fpm'

# Check if they are maxed out
grep -i 'max_children' /var/log/php-fpm/error.log | tail -5
# If you see "server reached pm.max_children" → Not enough processes

; /etc/php-fpm.d/www.conf
pm.max_children = 50

How Many Processes Can You Run? It Depends on Memory

# Check how much memory a single php-fpm process consumes
ps --no-headers -o rss -C php-fpm | awk '{s+=$1} END {print "Total Memory (MB):", s/1024}'
ps --no-headers -o rss -C php-fpm | awk '{s+=$1; n++} END {print "Average (MB):", s/n/1024}'

# Available Memory ÷ Single Process Memory ≈ Max Processes (Leave headroom for OS and other services)
free -m

Increasing process counts without checking memory is dangerous: it can trigger OOM (Out of Memory) errors, turning a “502 error” issue into overall system instability.

Database Layer

# Check if connection count is near the limit
mysql -e "SHOW STATUS LIKE 'Threads_connected'; SHOW VARIABLES LIKE 'max_connections';"
mysql -e "SHOW STATUS LIKE 'Max_used_connections';"

# Check for lock waits
mysql -e "SELECT * FROM information_schema.processlist
          WHERE command != 'Sleep' ORDER BY time DESC LIMIT 10G"

# Check slow queries
mysqldumpslow -s t -t 10 /var/log/mysql/slow.log

Increasing Web layer processes amplifies pressure on database connections. Therefore, before increasing php-fpm, ensure the database can handle the corresponding number of connections; otherwise, you’re just moving the bottleneck from one layer to another.

System Layer Limits

# File descriptor limits (often a bottleneck under high concurrency)
ulimit -n
cat /proc/$(pgrep -o nginx)/limits | grep -i 'open files'

# Current usage
ls /proc/$(pgrep -o nginx)/fd | wc -l

# Ports and connections
ss -s
cat /proc/net/sockstat

# Increase limits (systemd service)
# /etc/systemd/system/nginx.service.d/override.conf
[Service]
LimitNOFILE=65535

systemctl daemon-reload && systemctl restart nginx

External Dependencies (Most Often Overlooked)

# Find external calls in code without timeout settings
grep -rniE 'curl_setopt|file_get_contents|wp_remote_get' 
  /var/www/html --include='*.php' | grep -v timeout | head

When external APIs lack timeout settings, a slow response will hold up php-fpm processes, cascading to bring down the entire site. This is the most common and easiest-to-fix cause of “inability to scale concurrency.”

// Must set timeouts
$args = [
    'timeout' => 5,        // Critical
    'blocking' => true,
];
wp_remote_get($url, $args);

Decision Tree

Concurrency cannot scale
├─ First, run load tests to quantify and simultaneously monitor server resources
├─ Identify which layer has the bottleneck
│  ├─ php-fpm processes maxed out
│  │  ├─ Is there enough memory? Yes → Increase max_children
│  │  └─ No → Optimize scripts or add memory (don't force increase)
│  ├─ Database connections full / Lock waits present
│  │  └─ Optimize slow queries or use connection pooling
│  ├─ File descriptors / System limits hit
│  │  └─ Increase ulimit and LimitNOFILE
│  └─ External APIs are slow
│     └─ Add timeouts and fallback mechanisms (Easiest fix, highest ROI)
└─ After adjustments, re-run load tests to confirm effectiveness

Scenario Analysis: Why Concurrency Still Won’t Scale After Adding Multiple Servers

Symptom: An interface fails under peak business load.
After scaling up by several times based on the assumption of “insufficient resources,” concurrency capacity remains virtually unchanged.

1. First, confirm if scaling actually took effect → Ensure new instances are running and receiving traffic,
   ruling out failed scaling deployment.
2. Check if the bottleneck isn’t at this layer → The interface itself processes quickly,
   but time is spent waiting for external dependencies. Adding servers just parallelizes the “waiting.”

3. Perform timing breakdowns for each step → Split a request into ingestion, application, database, and external calls,
   to see where the time is spent.
4. Discover the database has long been the bottleneck → Connections are fully occupied,
   so new Web instances only worsen the queue.
5. Confirm multiple constraints exist simultaneously →
   External calls lack timeouts, database lacks indexes, and connection pool config is too small; all three compound.
6. Fix bottlenecks in order and re-test → First resolve external call waiting issues,
   then address the database, and finally consider further scaling.

# Timing breakdown: Confirm where time is spent
curl -sS -o /dev/null -w 'connect=%{time_connect} ttfb=%{time_starttransfer} total=%{time_total}n' https://example.com/api

# Check if database connections are fully occupied
mysql -e "SHOW STATUS LIKE 'Threads_connected'; SHOW VARIABLES LIKE 'max_connections';"
mysql -e "SELECT COUNT(*) FROM information_schema.processlist WHERE command != 'Sleep';"

# Slow queries: A common reason why adding servers doesn't help
mysqldumpslow -s t -t 10 /path/to/slow-log 2>/dev/null | head -20

# Check if external calls have timeout limits (missing limits exhaust worker slots)
grep -rIn 'timeout' /app-config-dir | head

Root Cause: The bottleneck is not at the layer being scaled.
Each request occupies backend resources for a long time waiting for external dependencies; adding instances only lengthens the queue.

Two Key Takeaways:

• Before scaling, confirm the bottleneck is at that layer.
  If not, adding more machines will yield no change and only increase pressure downstream.
• Set timeouts for external calls before discussing scaling.
  Unbounded external waits occupy limited resources for extended periods; this is the most common root cause of such issues.

8 Common Pitfalls

1. Adding resources based on intuition: Upgrading without locating the bottleneck.
2. Increasing process counts without checking memory: Triggers OOM.
3. Only looking at CPU and memory: Both may appear low during lock waits.
4. Ignoring external API timeouts: A concurrency killer.
5. Not monitoring the server during load tests: Knowing it’s slow but not why.
6. Failing to re-test after changes: Cannot verify effectiveness.
7. Not establishing a load test baseline: Without pre-optimization numbers, comparison is impossible.
8. Running load tests during production peaks: Impacts real users.

Real Customer Feedback (From Cloud Market Reviews and Inquiries for Similar Products)

The following excerpts are taken from real buyer reviews and inquiries for similar operations services on Alibaba Cloud, Tencent Cloud, and Huawei Cloud Cloud Markets (service providers include our store and peers, provided here for reference only, not fictional tickets from our site):

b****4 Rating: 5.0 Jun 23, 2025 16:57 b****4 Rating: 5.0 Jun 23, 2025 16:56 Jian****Wei Rating: 5.0 Jan 20, 2021 15:03 Excellent technical skills, very fast! It turned out to be a trojan causing global Baidu hijacking!!!!!! f****u Rating: 5.0 Jan 02, 2021 20:43 Thanks for assisting in maintaining normal cloud operations. 1****1 Rating: 5.0 Nov 20, 2020 10:03 Consistently good service 1****1 Rating: 5.0 Nov 17, 2020 09:13 Thanks for the consistently good service. 1****1 Rating: 5.0 Sep 25, 2020 15:42 Many thanks. 1****1 Rating: 5.0 Aug 24, 2020 15:34 Many thanks to your company’s technical staff 1****1 Rating: 5.0 Jul 24, 2020 09:49 Many thanks! 1

d****9 Rating: 5.0 Feb 15, 2023 09:11 Yong****Ying Rating: 5.0 Mar 29, 2021 11:04

1****4 Rating: 5.0 May 08, 2023 11:53 1****3 Rating: 5.0 Sep 22, 2022 22:01 r****u Rating: 5.0 Jul 07, 2020 15:05