I have a problem between virtualmin and my new server, and I hope that, maybe, you could help me with it :) I tested all I could, investigated as far as my googling allowed me to, but now I'm stuck. If you know what I could do, where I could investigate, I would be EXTREMELY grateful ! :)
If I open a dynamic page in one of the sites I created on the new server, I have, in the browser, after a long wait :
Service Temporarily Unavailable The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.
In the apache error_log :
[warn] [client 18.104.22.168] mod_fcgid: can't apply process slot for /home/olivertest/fcgi-bin/php5.fcgi
(Or, the same, for every other website that I tested it for)
The context : until last month, I had a relatively average server, on which I had debian squeeze, webmin and virtualmin, and everything worked fine, I could add sites, all was cool. I moved to a new server, a powerful beast (two SSDs for / in raid-1, two SATAS for /home in Raid-1, the CPU is an AMD Opteron 4334, 6 cores, 3.6 Ghz. There is 32 GB of RAM). On this new server, the same tech guy installed the same software with the same settings, debian (this time Wheezy, the new one), webmin, virtualmin, suhosin, with FastCGI (FCGId) as handler. And then... all hell broke loose.
On my server, save two sites that work correctly (maybe because they're sitting alone on their own IP, or because they were the two that were imported first), all the other sites as as good as dead. The problem : static files are OK, dynamic files using PHP aren't OK.
It's the same, whether I import a virtualmin backup of a site or delete it and re-create it from scratch.
Following http://www.megalinux.net/mod_fcgid-cant-apply-process-slot/, I checked that the directory in which Apache kept the socks for fcgid (/var/lib/apache2/fcgid/sock , on my debian wheezy) had the proper 755 permissions. It already had these.
Just in case, I turned off fail2ban and restarted Apache... no change.
Maybe a server limitation problem ? I checked, and my server is VERY FAR from hitting its hardware limits. the Top command shows all is OK, the most extensively used one is the CPU, around 20%. If you want to look at a Top : http://imgur.com/NY0VvPP Munin graphs show : the Apache processes limit is 600, with production peaks at 300. The CPU usage, with a maximum of 600% (6 cores), adding system, user and rare iowait, hardly amounts to 100% at worst, with everything else being Idle. RAM usage is peaceful, with most of the RAM either unused or used for cache (I'd made plenty of restarts lately because of my tests, it would take 3 or 4 hours to have all spare ram becoming cache instead of unused.)
I thought I would check the system configuration files, to see if there was an issue :
My /etc/apache2/apache2.conf file (whose contents I've seen mentioned as httpd.conf in older discussions, I suppose it was before Apache2) mentions :
Timeout 300 KeepAlive On MaxKeepAliveRequests 200 KeepAliveTimeout 3
(Initially, keepalive was at 100, timeout at 2 : no change)
<IfModule mpm_prefork_module> StartServers 5 MinSpareServers 5 MaxSpareServers 10 ServerLimit 600 MaxClients 600 MaxRequestsPerChild 1000 </IfModule> <IfModule mpm_worker_module> StartServers 2 MinSpareThreads 25 MaxSpareThreads 75 ThreadLimit 64 ThreadsPerChild 25 MaxClients 150 MaxRequestsPerChild 0 </IfModule> <IfModule mpm_event_module> StartServers 2 MinSpareThreads 25 MaxSpareThreads 75 ThreadLimit 64 ThreadsPerChild 25 MaxClients 150 MaxRequestsPerChild 0 </IfModule>
My /etc/apache2/mods-available/fcgid.conf mentions :
<IfModule mod_fcgid.c> AddHandler fcgid-script .fcgi FcgidConnectTimeout 60 MaxProcessCount 9 MaxRequestLen 33554432 </IfModule>
(Initially MaxProcessCount was 3, timeout was 30 : increasing it made no difference.)
I uploaded to pastebin the php.ini of a site having the problem : http://pastebin.com/t88GNvN1
Some odd things, also. Coming back from the outside (bringing my kids home from school), I saw a site was finally working, I thought, thanks to the tests I made in its php.ini. Later on, for more testing, I restarted Apache ("service apache2 restart"). Guess what ? The site that used to be working again, had once again ceased to be served. Four hours later, no change is to be seen.
I carefully, gradually raised the limits, sometimes lowering them, to no change. I either imported virtualmin backups from the previous server, or deleted everything, recreated from nothing, and reuploaded files by SFTP. No difference.
In Virtualmin > Virtual Server > Server Configuration > Website options, I tested that the bug was GONE if I chose mod_php or CGI wrapper as php handler. But, according to http://boomshadow.net/tech/php-handlers/ , this is NOT secure outside of Fcgid, sob. Because, switching back to Fcgid, the problem was back again.
Would you know what may have been wrong with virtualmin, webmin, or debian ?
Maybe, for an experienced person, it's all making sense...
Thank you SO MUCH if you can help ! :)
(edit : 5'30 hours after the last Apache restart, I find that suddenly one of the dead sites is working. For the moment. But not the others. And I have no idea why, I didn't restart the server or apache. I only found as php.ini differences that on the broken site zend.enable_gc = On isn't uncommented, max_execution_time = 30 VS 90 on resurrected site, and session.entropy_length = 0 is a live setting while it's commented on the rezzed site. And I still have no clue what those thigns mean, save, at least, the zend reference to the ionclube loader needed by one of the working sites on the server, but that isn't used by anything on the dead sites.) (Edit2 : I copied the settings of the live site to the dead site's php.ini - Beyond Compare does marvels to save time -, and restarted apache. Now, the dead site is live again. And the site that was live is now dead again. I give up for the moment, once again, thank you SO MUCH if you've got a hint !)