Default IPCCommTimeout is too low

konamacphee's picture
Submitted by konamacphee on Tue, 04/05/2016 - 04:51

When a VirtualServer is created or the PHP version for that server is changed (and possibly in other circumstances), an entry for IPCCommTimeout is added to the relevant Apache VirtualHost entry in /etc/httpd/conf/httpd.conf.

The default setting added to the VirtualHost in httpd.conf is:

IPCCommTimeout 31

This value is set programmatically in the following script: /usr/libexec/webmin/virtual-server/php-lib.pl

# For fcgid mode, set IPCCommTimeout to either the configured value
# or the PHP max execution time + 1, so that scripts run via fastCGI
# aren't disconnected

(comment above at lines 352-354)

When using mod_fcgid, this default setting for IPCCommTimeout results in many read data timeout errors of the following kind:

[Mon Apr 04 20:01:26.919877 2016] [fcgid:warn] [pid 23943] [client xxx.xxx.xxx.xxx:33985] mod_fcgid: read data timeout in 31 seconds

It's suggested in a previous report for this issue that IPCCommTimeout refers to clock time (i.e. real-world seconds) while PHP's max_execution_time is in CPU time. (Therefore the above error would be quite expected for longer-running PHP processes if the server is under load.) This is consistent with what I'm seeing, as if I increase IPCCommTimeout to something much bigger (e.g. 1200 rather than 31) the errors immediately disappear.

Suggested fix, as per previous report, is to set default IPCCommTimeout to some suitable multiple of php_execution_time (e.g. 40x is what I'm using successfully).

Status: 
Active

Comments

Submitted by JamieCameron on Tue, 04/05/2016 - 23:44

Is the load average on your system above the number of cores? For wall time to differ substantially from execution time, the system would have to be pretty loaded..

konamacphee's picture
Submitted by konamacphee on Wed, 04/06/2016 - 09:07

This is running on a single-core VPS with low but bursty traffic. 15-min load average does briefly go over 2 a few times a day, and 5-min load average can spike higher.

Submitted by JamieCameron on Wed, 04/06/2016 - 22:29

BTW, you can increase the default timeout for new domains at System Settings -> Virtualmin Configuration -> PHP options -> Maximum fcgid execution time.

konamacphee's picture
Submitted by konamacphee on Thu, 04/07/2016 - 02:36

Thanks Jamie! I can confirm that setting the default as you suggest does control the initial value of IPCCommTimeout in httpd.conf for new virtual servers.

It's not the most obvious naming connection ("Maximum fcgid execution time" -> "IPCCommTimeout") so it might help if that config screen could document the connection? (One might assume otherwise that this setting relates to e.g. FcgidProcessLifeTime.)

I reckon it would still help to change the calculated default in php-lib.pl, for when "Maximum fcgid execution time" is left unspecified (ie by default.) Otherwise anyone running a small server with bursty traffic is likely to see these read data timeout errors.

Submitted by JamieCameron on Thu, 04/07/2016 - 11:23

Good to hear - I will improve the documentation on this feature. I'm not convinced that the diff between the PHP setting and IPCCommTimeout should be increased though, as it would lead to unusual behavior on non-loaded systems.

konamacphee's picture
Submitted by konamacphee on Fri, 04/08/2016 - 02:47

Thanks Jamie. I'm still trying to optimise - or at least sensible-ize! - the fcgid setup. I'll try e.g. a 4x rather than 40x setting for IPCommTimeout and see how that plays on my server.

Would you happen to know if all the time-related settings for mod_fcgid are CPU time rather than wall time, and/or why IPCCommTimeout uses wall time? It's bound to cause trouble when mixing the two together :(

Submitted by JamieCameron on Fri, 04/08/2016 - 11:28

I would hope they are wall time, since CPU time over some wall time could differ between processes.