instablity after webmin upgrade (1.540)

Submitted by smartvirtualmin on Wed, 04/06/2011 - 10:18

we have an instability issue after webmin upgrade, i think it's virtualmin realted anyway.

It's only on our oldest server all others are running well.

So all the problems are mail related based on procmail and lookup-domain General problem is: Mail delivery message sent back to every mailbox owner but messages delivered normally.

  1. First problem with /etc/webmin/virtual-server/lookup-domain.pl there were a permission problem to open the miniserv.conf file so we patch the pl file a bit and now it works but webmin reset this file back to original when somebody create a new virtual server. modified part: #open(CONF, "/etc/webmin/miniserv.conf") or die $!; #while() { # $root = $1 if (/^root=(.*)/); # } #close(CONF); $root = "/usr/share/webmin";

miniserv rights: -rw------- 1 root bin 1043 2011-04-06 09:55 miniserv.conf

procmail rights: -rwsr-sr-x 1 root root 85576 2007-03-27 20:38 procmail

we had the same problem after virtualmin upgrade but then this issue solve the situation: https://www.virtualmin.com/node/17616

But now it's not help us. procmail log: Permission denied at /etc/webmin/virtual-server/lookup-domain.pl line 9. procmail: Program failure (13) of "/etc/webmin/virtual-server/lookup-domain.pl"

  1. it's a more problematic stuff, lookup-domain daemon stop working or have soem side effect i dont know but restart fix this issue procmail log: procmail: Program failure (1) of "/etc/webmin/virtual-server/lookup-domain.pl" Error: Script was not run with full path (failed to find /usr/share/webmin/virtual-server/lookup-domain.pl under )

Timeout connecting to lookup-domain-daemon.pl Error: Script was not run with full path (failed to find /usr/share/webmin/virtual-server/lookup-domain.pl under )

so we add a crontab as temporary solution: 0 */2 * * * /etc/init.d/lookup-domain restart

now we stop webmin on server and things are nearly working but we need a real solution.

procmailrc: DROPPRIVS=yes LOGFILE=/var/log/procmail.log TRAP=/etc/webmin/virtual-server/procmail-logger.pl :0wi VIRTUALMIN=|/etc/webmin/virtual-server/lookup-domain.pl $LOGNAME EXITCODE=$? :0 * ?/usr/bin/test "$EXITCODE" = "73" /dev/null :0 * ?/usr/bin/test "$VIRTUALMIN" != "" { INCLUDERC=/etc/webmin/virtual-server/procmail/$VIRTUALMIN } ORGMAIL=$HOME/Maildir/ DEFAULT=$HOME/Maildir/

Status: 
Closed (fixed)

Comments

Submitted by JamieCameron on Wed, 04/06/2011 - 14:07

A couple of other people reported this, and it is not really a Webmin issue .. instead the problem is that procmail is being run with the wrong permissions, which causes more problems in Virtualmin 3.84.

To fix it, make sure that in /etc/postfix/main.cf the mailbox_command line is like :

mailbox_command = /usr/bin/procmail-wrapper -o -a $DOMAIN -d $LOGNAME

The permissions on procmail-wrapper and procmail should be :

# ls -l /usr/bin/procmail*
-rwxr-xr-x 1 root mail 72316 2007-03-27 11:35 /usr/bin/procmail
-rwsr-sr-x 1 root root  2916 2006-11-28 01:08 /usr/bin/procmail-wrapper

Submitted by smartvirtualmin on Thu, 04/07/2011 - 03:10

on this server we have no procmail-wrapper unlucky somehow: we change procmail permissions but that's not help: -rwxr-xr-x 1 root mail 85576 2007-03-27 20:38 procmail

mailbox command is: mailbox_command = /usr/bin/procmail -a "$EXTENSION" DEFAULT=$HOME/Maildir/ MAILDIR=$HOME/Maildir

what is your suggestion how to fix it? copy a procmail-wrapper here form an other server and change mailbox_command?

this is a really old server which run years without webmin and we installed it when we start use webmin/virtualin products so this surely cause some problems as i see.

Submitted by smartvirtualmin on Thu, 04/07/2011 - 05:48

we copy a procmail-wrapper there and setup permissions as well: -rwxr-xr-x 1 root mail 85576 2007-03-27 20:38 procmail -rwsr-sr-x 1 root root 2916 2011-04-07 12:36 procmail-wrapper

and change the mailbox command as well: mailbox_command = /usr/bin/procmail-wrapper -o -a $DOMAIN -d $LOGNAME

But we still have: Permission denied at /etc/webmin/virtual-server/lookup-domain.pl line 9.

miniserv.conf file permissions are: -rw------- 1 root bin 1043 2011-04-06 09:55 miniserv.conf

Submitted by smartvirtualmin on Thu, 04/07/2011 - 06:50

postfix/local[17168]: B6174200BCD7: to=, orig_to=, relay=local, delay=0.35, delays=0.35/0/0/0.01, dsn=5.3.0, status=bounced (Command died with status 127: "/usr/bin/procmail-wrapper -o -a $DOMAIN -d $LOGNAME". Command output: sh: /usr/bin/procmail-wrapper: not found ) but procmail-wrapper is there with the permissions above.

sorry about the much posts just it's a mass used server and we have several issues from the customers now.

now we recive that message unlucky :-/

Submitted by smartvirtualmin on Thu, 04/07/2011 - 07:22

sorry silly me it's just some stress side effect :/ anyway procmail-wrapper is successfuly installed and working now:

status=sent (delivered to command: /usr/bin/procmail-wrapper -o -a $DOMAIN -d $LOGNAME)

but problem is still persist unlucky :/

Submitted by smartvirtualmin on Thu, 04/07/2011 - 09:54

so we disable spam filtering on the server and now its working well. patched the lookup-domain.pl in the etc and disable to run the same file in the usr.

generally the main issue is: with high load the lookup-doamin-daemon is start throw errors and this cause a lot problems (bounced mails) so we must disable the spam filtering to fix this issue.

other part with permission is still exists but we do not use that pl file now so now it's working, but when somebody create a domain this file is overwritten by webmin ot the original, so we add +i attribute to the file.

it's not a real solution just a temporary patch i think, i hope it can be managed better. Anyway we no need spamfiltering on that server graylisting is enough so that can be the way but withou t pl files patch it can be great.

i can't disable the plugin on the virtualmin admin area beacause it's enabled on ~850 domains and virtualmin said it can't be disabled now globally.

Submitted by JamieCameron on Thu, 04/07/2011 - 12:11

So is it still failing all the time, or just when the system load is high?

When a failure happens, what gets logged to /var/log/procmail.log ?

If you like, I could login to your system and take a look .. but I'd need root SSH access.

Submitted by smartvirtualmin on Thu, 04/07/2011 - 14:01

As i see only when the system load is high.

procmail error:

Timeout connecting to lookup-domain-daemon.pl Error: Script was not run with full path (failed to find /usr/share/webmin/virtual-server/lookup-domain.pl under )

Script was not run with full path (failed to find /usr/share/webmin/virtual-server/lookup-domain.pl under )

procmail: Program failure (1) of "/etc/webmin/virtual-server/lookup-domain.pl"

After we remove this line form lookup-doamin.pl:

exec("$root/virtual-server/lookup-domain.pl", @ARGV) || die "Failed to run $root/virtual-server/lookup-domain.pl : $!";

the webserver is muchmuch faster and load decreased dramatically.

  1. load with spamassasin run up to 50
  2. load without spamassasin but with lookupdoamin 10
  3. load without spamassaans and lookupdomain : 1-2

we will see mor result tomorrow, i will try arrange you a ssh user soon. i think we can move this topic private then.

Other things just we notice: This new webmin version makes webmin ui much much faster thanks great :)

Submitted by JamieCameron on Thu, 04/07/2011 - 14:04

Do you have the lookup-domain-daemon.pl server process running on your system?

Submitted by smartvirtualmin on Fri, 04/08/2011 - 03:40

absolutely, just now do not do anything.

root 30284 0.0 0.4 77280 47208 ? SNs Apr07 0:00 /usr/share/webmin/virtual-server/lookup-domain-daemon.p

Submitted by JamieCameron on Fri, 04/08/2011 - 12:01

So are you now only seeing failures when the system load is high?

If possible, I'd like to login to your system myself to debug this further ..

Submitted by smartvirtualmin on Mon, 05/16/2011 - 12:30

We decidde to reinstall the system on 10.04 LTS i'm sure it will solve the problem. Have you any guide for reinstall steps or can you give us some suggestions what is the vulnerability of a reinstall? (I poke around here but can't find anything useful) Can we copy the etc/webmin area simply and everything will work fine? What about our serial number it ill be accepted on the new system?

Submitted by andreychek on Mon, 05/16/2011 - 12:54

The Ubuntu version you're on hits it's end of life next month -- so moving that to 10.04 wouldn't be a bad idea.

My suggestion for handling that would be to use a different server, and then:

  • Perform the 10.04 installation on that new server

  • Install Virtualmin Pro on it (you can use your same license, that's no problem)

  • Generate a set of full backups from your old server, copy them to your new server, and restore them all. That restore process should include the virtualmin.tar.gz file, which will include all your Virtualmin settings such as server templates, account plans, backup schedules, and such.

After all that is done -- you can do some testing to make sure everything is working as expected.

Once it's all working, you can then make that server live.

It's a big jump moving all your websites from an Ubuntu 6.06 server to one running 10.04, so you'll definitely want to allow for time to test all your sites and make any tweaks to get things working.

Submitted by smartvirtualmin on Mon, 06/27/2011 - 08:14

upgrade was successful thanks for the help. Everything looks like ok now.

Lawrence

Submitted by danny on Wed, 04/08/2020 - 04:09

Hello, we had the same problem. Since we have all users in chroot jails, we run procmail with user rights and not with root rights. The webmin update to version 1.941 removed the file rights of the miniserv.conf file for the groups and other users, presumably in line 780 in the setup.sh file with the command "chmod -R og-rw $config_dir/$f". I changed the file permissions to 644 and the bug is fixed. But for further updates it would be nice if this is checked and in this case not changed, thanks.