Keywords: Apache Configuration | MPM Modules | MaxRequestWorkers
Abstract: This article provides an in-depth analysis of the common AH00161 error in Apache servers, which indicates that the server has reached the MaxRequestWorkers setting limit. Through a real-world case study, the article reveals the root cause of MPM module mismatch in configuration files. The case involves a server running Ubuntu 14.04 handling a WordPress site with approximately 60,000 daily visits. Despite sufficient resources, the server frequently encountered errors. The article explains the differences between mpm_prefork and mpm_worker modules, provides correct configuration modification methods, and emphasizes the importance of using the apachectl -M command to verify currently loaded modules. Technical discussions cover Apache Multi-Processing Module working principles, configuration inheritance mechanisms, and best practices to avoid common configuration pitfalls.
In Apache server administration practice, configuration errors are common causes of performance issues and system interruptions. This article explores key issues in Apache Multi-Processing Module (MPM) configuration through a specific case analysis.
Problem Background and Error Symptoms
The case involves a server running Ubuntu 14.04.4 LTS hosting a WordPress-based website with approximately 60,000 daily visits. The server is configured with multiple caching optimizations including W3TotalCache APC, Varnish, and Cloudflare. Despite system load averages remaining between 0.5-1 and low memory usage, Apache logs frequently showed the following error:
[Thu Apr 28 14:14:42.938075 2016] [mpm_prefork:error] [pid 19137] AH00161: server reached MaxRequestWorkers setting, consider raising the MaxRequestWorkers setting
Accompanied by multiple child process segmentation fault exits:
[Thu Apr 28 15:43:25.594147 2016] [core:notice] [pid 19137] AH00051: child pid 19866 exit signal Segmentation fault (11), possible coredump in /etc/apache2
Initial Diagnosis and Incorrect Attempts
The administrator initially attributed the problem to insufficient MaxRequestWorkers settings and attempted to gradually increase the value: from 100 to 500, eventually setting it to 1024. However, errors persisted. The Apache configuration at that time was:
<IfModule mpm_worker_module>
StartServers 256
MinSpareThreads 256
MaxSpareThreads 256
MaxClients 256
ServerLimit 256
ThreadLimit 256
ThreadsPerChild 256
MaxRequestWorkers 1024
MaxConnectionsPerChild 0
MaxRequestPerChild 1000
</IfModule>
System monitoring data showed sufficient server resources:
top - 16:07:47 up 3 days, 18:18, 2 users, load average: 0.57, 0.46, 0.55
Tasks: 113 total, 1 running, 112 sleeping, 0 stopped, 0 zombie
%Cpu(s): 10.5 us, 2.0 sy, 0.0 ni, 87.4 id, 0.1 wa, 0.0 hi, 0.0 si, 0.0 st
KiB Mem: 8176816 total, 2372560 used, 5804256 free, 189684 buffers
Root Cause Analysis
The error log clearly indicates the problem occurs in mpm_prefork_module, but the configuration modifies settings for mpm_worker_module. This is a typical module configuration mismatch issue. Apache's Multi-Processing Module system requires configurations to exactly match the currently loaded MPM module.
In Ubuntu systems, Apache configuration adopts a modular structure, with main configuration files located in the /etc/apache2/ directory. Key configuration files include:
apache2.conf: Main configuration filemods-available/: Available module configurationsmods-enabled/: Enabled module configurations
Correct Solution
The correct solution is to modify the configuration file corresponding to the currently loaded module. According to best practices:
- First determine the currently loaded MPM module:
$ apache2ctl -M | grep mpm
Output may show mpm_prefork_module, mpm_worker_module, or mpm_event_module.
mpm_prefork_module, edit:/etc/apache2/mods-available/mpm_prefork.conf
Correct configuration example:
<IfModule mpm_prefork_module>
StartServers 10
MinSpareServers 10
MaxSpareServers 20
ServerLimit 2000
MaxRequestWorkers 1500
MaxConnectionsPerChild 10000
</IfModule>
Understanding Configuration Inheritance Mechanism
Apache's configuration system employs inheritance and override mechanisms. When multiple configuration files contain the same directives, later-loaded configurations override previous settings. In Ubuntu's default configuration:
# /etc/apache2/apache2.conf includes
IncludeOptional mods-enabled/*.load
IncludeOptional mods-enabled/*.conf
This means configurations in the mods-enabled/ directory override settings in the main configuration file. Therefore, modifying MPM configurations only in apache2.conf is usually ineffective because module-specific configuration files override these settings.
MPM Module Technical Details
Understanding different MPM module working mechanisms is crucial for proper configuration:
mpm_prefork_module
The prefork MPM uses multiple child processes, each handling one connection at a time. This is the most compatible MPM, particularly suitable for PHP applications requiring non-thread-safe libraries.
mpm_worker_module
The worker MPM uses a hybrid model of multiple processes and threads. Each child process creates multiple threads, with each thread handling one connection. This provides better performance but requires all modules to be thread-safe.
mpm_event_module
The event MPM is a variant of worker MPM optimized for maintaining numerous connections, using dedicated listener threads to manage idle connections.
Best Practice Recommendations
- Verify Current Module: Always use
apache2ctl -Mto confirm currently loaded modules before modifying any MPM configuration. - Modify Correct Files: Do not modify MPM settings directly in
apache2.conf, but edit corresponding module configuration files. - Understand Configuration Hierarchy: Familiarize yourself with Apache's configuration inheritance mechanism, particularly the loading order of
Includedirectives. - Monitor Resource Usage: When adjusting
MaxRequestWorkers, monitor memory usage as each worker process/thread consumes system resources. - Test Configuration Changes: Use
apache2ctl configtestto verify configuration syntax, then gradually reload configurations.
Troubleshooting Process
When encountering similar AH00161 errors, follow this troubleshooting process:
1. Check error logs to identify specific module
2. Use apache2ctl -M to verify currently loaded modules
3. Check corresponding module configuration files
4. Verify configuration syntax
5. Gradually adjust parameters and monitor effects
6. Consider system resource limits (e.g., ulimit)
Conclusion
Apache server configuration requires precise matching with currently loaded MPM modules. The case problem originated from modifying settings in the mpm_worker_module configuration block while the system actually used mpm_prefork_module. By understanding Apache's modular configuration system and MPM working mechanisms, administrators can avoid such configuration errors and ensure stable server operation. The correct approach is to always verify current modules and modify corresponding configuration files rather than relying on settings in the main configuration file.