PHP7.0-FPM failure after upgrade to 6.07

35 posts / 0 new
Last post
#1 Sat, 07/27/2019 - 05:33
anahata
anahata's picture

PHP7.0-FPM failure after upgrade to 6.07

I've just installed the virtualmin, webmin and usermin upgrades posted today on a Debian 9 system using PHP7.0-FPM for all sites, and they have all stopped working.

I did the upgrade by: sudo aptitude upgrade

and those were the only packages that have changed. (no standard Debian packages to upgrade)

My apache2 error log file contains pairs of entries like this:

[proxy_fcgi:error] [pid 23283:tid 140561914713856] [client 2a02:8010:6315:0:7433:f2a:a4f6:d8c9:36916] AH01079: failed to make connection to backend: localhost
[proxy:error] [pid 26583:tid 140687131494144] (111)Connection refused: AH00957: FCGI: attempt to connect to 127.0.0.1:8063 (*) failed

I'm not hugely experienced with FPM and don't know where to start. Any ideas?

Sat, 07/27/2019 - 07:50
anahata
anahata's picture

Update: panic over!

The listen port numbers in /etc/apache2/modules-available and in /etc/php/7.0/fpm/pool.d didn't match up. Probably because I had edited the FPM pool files and Virtualmin didn't want to touch them, but after a tedious manual editing session all is well now.

Anahata www.treewind.co.uk West Yorkshire, UK

Sat, 07/27/2019 - 08:28
randallflagg

same problem but if you do the virtualmin config check it complains about php-fpm ports misconfig and messes everything up by screwing all the ports in the php pool

I'm on Debian 10 with php7.3

for now I had to hand change all the ports one by one in the /etc/php/7.3/fpm/pool.d/.conf I also tried to set the same ports in the /etc/webmin/virtualserver/domains/ configs without luck

every time the virtualmin re-check config starts the ports are changed in the php pool and nothing works

Wed, 08/28/2019 - 12:37 (Reply to #3)
rrcatto

I'm running virtualmin (LEMP installation) on Ubuntu 18.04.3 LTS and can confirm that running "Re-Check Configuration" messes up my nginx + php-fpm port settings every time. The nginx conf files retain the original port settings, but the numeric conf files in /etc/php/7.2/fpm/pool.d/ are all assigned new numbers which do not correspond with their associated nginx conf files. This results in a 502 server error when accessing the web sites. The solution is to manually edit all the numeric conf sites so that their listen directives have the correct port numbers.

The conf files have this naming convention: [domain-id].conf

It would have been much easier if they had adopted [domain-name].conf instead.

If you're hosting a lot of domains, you would have to keep an index file of domain-id, domain-name so you can quickly find the correct conf file to edit for a given domain name. Manually editing each conf file to correct the ports that re-check bug creates would become very tedious very soon when dealing with a lot of domains.

This is definitely a high priority fix, because right now, using re-check configuration is BROKEN.

Sat, 07/27/2019 - 13:13
anahata
anahata's picture
I had to hand change all the ports one by one in the /etc/php/7.3/fpm/pool.d/.conf

Yes, That's exactly what I did.

I hadn't got as far as looking at /etc/webmin/virtualserver/domains/ but it's disappointing that you can't get predictable behaviour that way either.

I'm on Debian 10 with php7.3

As a matter of interest, did you have problems or need to change anything to make current Virtualmin work on Debian 10 (compared with Debian 9) ?
I have a customer running Joomla! which is complaining that PHP7.0 is out of date. Technically true but with all Debian's security fixes it's actually safe, but even so it's disconcerting to have to tell him to ignore Joomla's warning.

Anahata www.treewind.co.uk West Yorkshire, UK

Sun, 07/28/2019 - 03:30
randallflagg

I am running without problems Apache, Dovecot, Postfix and various other services without problem. As a matter of fact even php didn't had problems until I re-checked virtualmin config. The only thing that I had to do is to rename the /etc/php/7.3 into /etc/php/7.0 to made it work. You might also want to delete all the php7.0* packages.

But since you need to re-check config in order to make virtualmin recognize 7.3 I won't suggest upgrading until everything is fixed. BTW I was on webmin 1.920 and virtualmin 6.06 when I made the switch.

Sun, 07/28/2019 - 10:32
strejc.david

I had same problem. I've rollback to 6.05 which is working and I got etckeeper which helped me to solve the issue.

Sun, 07/28/2019 - 18:14
jimdunn

Yep, I had same issue last night... I think I'm going back to FCGID and will let FPM mature a while longer.

The solution (workaround) is to change all domains to FCGID and back to FPM:

Step 1:

#!/bin/sh
mkdir /var/run/php
chown -R www-data:www-data /var/run/php
/usr/sbin/virtualmin modify-web --all-domains --mode fcgid
/etc/init.d/apache2 restart
/etc/init.d/php7.0-fpm restart

Step 2:

#!/bin/sh
mkdir /var/run/php
chown -R www-data:www-data /var/run/php
/usr/sbin/virtualmin modify-web --all-domains --mode fpm
/etc/init.d/apache2 restart
/etc/init.d/php7.0-fpm restart

(you might need to adjust the init.d lines, per your configuration)

Tue, 07/30/2019 - 07:33
randallflagg

I've found out that iptables dives a warning because of iptables-legacy Does not broke anything and can be fixed easily (I think)

Tue, 07/30/2019 - 09:17
anahata
anahata's picture

Ah yes, nftables is the default in Debian 10. I'd quite like to be using that; I wonder if Virtualmin will support it.

Anahata www.treewind.co.uk West Yorkshire, UK

Wed, 07/31/2019 - 04:42
virtualroad

In my case webmin is changing in all /etc/php5/fpm/pool.d/*.conf files the value of listen directive from socket to port. Tested with "dpk-reconfigure webmin" and using "System settings -> re-check configuration" from virtualmin. The change is made without asking for confirmation

The following PHP-FPM versions are available on this system : 5.6.33 5.6.33 5.6.33
Fixing port clash for PHP-FPM version 5.6.33
Fixing port clash for PHP-FPM version 5.6.33
Restarting PHP-FPM server ..
.. done
Fixing port clash for PHP-FPM version 5.6.33
Fixing port clash for PHP-FPM version 5.6.33
Restarting PHP-FPM server ..
.. done
Wed, 07/31/2019 - 07:46
Jfro

i only knows if using ports before there was a bug and you have to do some manual changes to have it work.

Didn't update while waiting for reply's about this from support?

those 3: https://www.virtualmin.com/node/66717

https://www.virtualmin.com/node/66727

https://www.virtualmin.com/comment/815098#comment-815098

Sun, 08/04/2019 - 09:34
sz00gun

I have the same problem with it. Any fix for that?

No doubt, that's the bug.

Changing all websites to fastcgi and back to fpm can fix the problem, but won't fix the bug.

That helps as a charm:

#!/bin/sh
mkdir /var/run/php
chown -R www-data:www-data /var/run/php
/usr/sbin/virtualmin modify-web --all-domains --mode fcgid
/etc/init.d/apache2 restart
/etc/init.d/php7.2-fpm restart

#!/bin/sh
mkdir /var/run/php
chown -R www-data:www-data /var/run/php
/usr/sbin/virtualmin modify-web --all-domains --mode fpm
/etc/init.d/apache2 restart
/etc/init.d/php7.2-fpm restart
Mon, 08/12/2019 - 04:00
sz00gun

Once a week, Saturday, 24.00 all websites go down. Something in Cron, but I cannot recognise what. This problem is after the last upgrade. Only above coffee can help...

Had anyone got the same problem?

Mon, 08/12/2019 - 07:08 (Reply to #14)
Jfro

Check if it is with / on graceful restart, of yes try to change to normal restart apache.as workarround for the time beeing

Mon, 08/12/2019 - 07:12
sz00gun

Apache restart don't send all websites down, it must be something else...

Mon, 08/12/2019 - 07:15
sz00gun

I recon it's a check configuration script. How can I check the Cron for that?

Btw. If I press check configuration manually in latest Virtualmin version, all websites go down...

Mon, 08/12/2019 - 08:33
Jfro

Check not the restart but graceful we had that with a problem bug in apache part once then the cron changed from graceful to restart solved.

long long ago don't know where to look. some info about http://csuca.org/manual/en/programs/apachectl.html

And some has solved to build a "wait" in script somewhere

Tue, 08/20/2019 - 12:29
eddieb

same problem. subscribing

Mon, 08/26/2019 - 08:22
geissweb

Just updated webmin+usermin and now FPM is fu**ed up again. Since several versions and months, the autors of this so far beautiful software can't seem to fix it or they just don't care.

https://www.virtualmin.com/node/55788 https://www.virtualmin.com/node/59503 https://www.virtualmin.com/node/66439 https://www.virtualmin.com/node/66795

;(

Mon, 08/26/2019 - 08:40
anahata
anahata's picture

I've just done the update too, with the same result. One again I edited files to make the port numbers match in /etc/apache2/sites-available/* and /etc/php/7.0/fpm/pool.d/* and all sites are working now.

I thought I saw a promise that this would be fixed soon, but the recent security upgrade is arguably more urgent, so maybe that had to be done first.

Anahata www.treewind.co.uk West Yorkshire, UK

Mon, 08/26/2019 - 08:58
geissweb

Thanks @anahata for the info to edit apache2/sites-available/*.

Mon, 08/26/2019 - 09:04
anahata
anahata's picture

In case it's not obvious, you change the SetHandler line (both of them if you it's an SSL site) to have a port number that matches the "listen =" line in the PHP conf file. You then have to restart both services apache2 and php7.0-fpm.

Anahata www.treewind.co.uk West Yorkshire, UK

Wed, 08/28/2019 - 10:45
eddieb

is there an ETA for an official fix?

Sat, 08/31/2019 - 04:44 (Reply to #24)
Jfro

Did someone test / noticed / noted what all changed exactly when not working anymore?

Sat, 08/31/2019 - 11:08 (Reply to #25)
eddieb

I do every single update and I first noticed the problem on the update prior to the last one (the one to fix the password reset vuln)

Sat, 09/07/2019 - 14:24
kovinet

Same thing happened to me. Took me a while to figure out the cause, so all sites were offline for like half an hour. :/

Sat, 09/07/2019 - 17:40
sz00gun

Welcome in the club...

Sun, 09/08/2019 - 03:47
kovinet

So yesterday I have changed ports for all nginx virtual hosts to new fpm ports, so I brought all sites back up. Today I have changed some unrelated configurations in webmin / virtualmin which somehow caused port change again, meaning that virtualmin regenerated fpm pool files for all installed PHP versions in:

/etc/php/7.0/fpm/pool.d/ /etc/php/7.1/fpm/pool.d/ /etc/php/7.2/fpm/pool.d/ /etc/php/7.3/fpm/pool.d/

making new port numbers for existing pools bringing all sites offline again.

This is critical bug, which should be addressed as soon as possible.

Meanwhile, what can I do to prevent these port changes in future, is there something I can do to prevent it?

Sun, 09/08/2019 - 05:42
adamjedgar

Guys can anyone confirm that in Webmin>Bootup options, php-fpm status is set to "start on boot" (this was the solution on my system for this error... php-fpm somehow was no longer set to start on boot)

AJECreative is the home of $5 webhosting, $15/month VPS servers (1cpu,1gb RAM, 25GB storage)
Centos7, Debian9, or Ubuntu18LTS
Available Control Panels = Centos-Webpanel, Cyberpanel, or Virtualmin

https://ajecreative.com.au

Sun, 09/08/2019 - 05:51 (Reply to #31)
sz00gun

In my case it was: start on boot -> activated

Mon, 09/09/2019 - 11:38
randallflagg

Same, Start on boot is activated

Wed, 10/16/2019 - 04:15
adamjedgar

has this been fixed yet? I am still having to change all domains from php-fpm to fcgid and back again every time i either run an apache update or virtualmin>system>recheck configuration

This is incredibly frustrating when trying to problem solve other things...one runs a recheck config and then domain php driven websites all go offline. such a time waster, not happy!!!!

AJECreative is the home of $5 webhosting, $15/month VPS servers (1cpu,1gb RAM, 25GB storage)
Centos7, Debian9, or Ubuntu18LTS
Available Control Panels = Centos-Webpanel, Cyberpanel, or Virtualmin

https://ajecreative.com.au

Wed, 10/16/2019 - 04:48
Jfro

UH you write here.?

My install of virtualmin has no longer experienced php fpm issues for a while now.

https://www.virtualmin.com/comment/817827#comment-817827

There are at the moment many many to many open .....

Topic locked