Re-Check Configuration messes procmail config + others

Submitted by ITCiao on Wed, 03/06/2013 - 20:02

Hi,

I just got a new Virtualmin Pro licence and made a clean install.

Noticed the fallowing bugs:

No 1. When using Re-Check Configuration, procmail

Set variable ORGMAIL to $HOME/Maildir/       
Set variable DEFAULT to $HOME/Maildir/

Reset to /var/mail/ ... making the mailclients like Roundcube or Squirrelmail uesless. You can see send mails but not inboxes, becouse they go into /var/mail/...

It's this related to main.cf -> mailbox_command = /usr/bin/procmail-wrapper -o -a $DOMAIN -d $LOGNAME ?

No 2. Roundcube does not display mail dates. I really need roundcube working!

No 3. Trash folders does not appear in Roundcube or Squirrelmail by default. Hat to go to Webmin -> Webmin -> Usermin Configuration -> Usermin Module Configuration -> Read Mail -> Mail storage format for Inbox -> Qmail style directory (Maildir).

No 4. Mail autoconfig dosen't work properly. When using user@domain.tld this patch need to be applyed.

--- autoconfig.cgi      2013-02-05 15:47:17.000000000 +1100
+++ autoconfig2.cgi     2013-02-05 15:47:41.000000000 +1100
@@ -59,7 +59,7 @@
        $login = $PREFIX."_".$name;
        }
 elsif ($STYLE == 6) {
-       $login = $emailaddress;
+       $login = $email;
        }
 elsif ($STYLE == 7) {
        $login = $name."\%".$PREFIX;

No 5. Thunderbird does not download subfolders. Is a particular feature i need to enable?

I need help with No. 2 and No. 5, the rest are just various fixes you guys should consider.

Thank you.

Status: 
Active

Comments

Submitted by andreychek on Wed, 03/06/2013 - 20:56

Howdy --

No 1. When using Re-Check Configuration, procmail

You're right, that is a rather big problem, and would break email delivery for anyone that happened to.

You're the only one to report that though, and I unfortunately don't seem to be able to reproduce it.

Is there any chance you could post the contents of your /etc/procmailrc, as well as the output of the commands "postconf -n" and "dovecont -n"?

No 2. Roundcube does not display mail dates. I really need roundcube working!

Well, Virtualmin doesn't do anything to the email messages that come in for delivery -- those are being delivered by Postfix and procmail... Virtualmin doesn't control how the emails look within RoundCube.

That said, I hadn't run into that issue before, the dates look okay for when I log into RoundCube on my system. Is there any chance you could post a screenshot of what you're seeing?

No 5. Thunderbird does not download subfolders. Is a particular feature i need to enable?

There shouldn't be a feature you need to enable for that, you should be able to download all the folders by default.

How did you initially create the sub-folders -- what email client did you create them in?

And out of curiosity, are you able to download the sub-folders in another email client?

I believe issues #2 and #3 were something Jamie was working on recently, but he'll be able to comment on those more thoroughly.

Submitted by ITCiao on Wed, 03/06/2013 - 21:23

Hello and thank you for your reply.

No 1. it's easy to reproduce, just * get a fresh CentOS 6.3 minimal installed. * Run yum update * Run yum install wget mlocate perl * Install virtualmin and hit recheck config. Might need to create a virtualmin first.

This is what i have in /etc/procmailrc. I modifyed the ORGMAIL and DEFAULT manually via Webmin.

 cat /etc/procmailrc
LOGFILE=/var/log/procmail.log
TRAP=/etc/webmin/virtual-server/procmail-logger.pl
:0wi
VIRTUALMIN=|/etc/webmin/virtual-server/lookup-domain.pl $LOGNAME
EXITCODE=$?
:0
* ?/usr/bin/test "$EXITCODE" = "73"
/dev/null
EXITCODE=0
:0
* ?/usr/bin/test "$VIRTUALMIN" != ""
{
INCLUDERC=/etc/webmin/virtual-server/procmail/$VIRTUALMIN
}
ORGMAIL=$HOME/Maildir/
DEFAULT=$HOME/Maildir/
DROPPRIVS=yes
:0
$DEFAULT

postconf -n

alias_database = hash:/etc/aliases
alias_maps = hash:/etc/aliases
command_directory = /usr/sbin
config_directory = /etc/postfix
daemon_directory = /usr/libexec/postfix
data_directory = /var/lib/postfix
debug_peer_level = 2
html_directory = no
inet_interfaces = all
inet_protocols = ipv4
mail_owner = postfix
mailbox_command = /usr/bin/procmail-wrapper -o -a $DOMAIN -d $LOGNAME
mailq_path = /usr/bin/mailq.postfix
manpage_directory = /usr/share/man
milter_default_action = accept
milter_protocol = 2
mydestination = $myhostname, localhost.$mydomain, localhost
newaliases_path = /usr/bin/newaliases.postfix
non_smtpd_milters = inet:localhost:8891
queue_directory = /var/spool/postfix
readme_directory = /usr/share/doc/postfix-2.6.6/README_FILES
sample_directory = /usr/share/doc/postfix-2.6.6/samples
sendmail_path = /usr/sbin/sendmail.postfix
setgid_group = postdrop
smtpd_milters = inet:localhost:8891
smtpd_recipient_restrictions = permit_mynetworks permit_sasl_authenticated reject_unauth_destination
smtpd_sasl_auth_enable = yes
smtpd_tls_cert_file = /etc/postfix/postfix.cert.pem
smtpd_tls_key_file = /etc/postfix/postfix.key.pem
smtpd_tls_mandatory_ciphers = high
smtpd_tls_security_level = may
unknown_local_recipient_reject_code = 550
virtual_alias_maps = hash:/etc/postfix/virtual

dovecot -n

# 2.0.9: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.32-279.22.1.el6.x86_64 x86_64 CentOS release 6.3 (Final)
auth_mechanisms = plain login
disable_plaintext_auth = no
mail_location = maildir:~/Maildir:INDEX=/var/lib/dovecot-virtualmin/index/%u:CONTROL=/var/lib/dovecot-virtualmin/control/%u
mbox_write_locks = fcntl
passdb {
  driver = pam
}
protocols = imap pop3
ssl_cert = </etc/pki/dovecot/certs/dovecot.pem
ssl_cipher_list = ECDHE-RSA-AES256-SHA384:AES256-SHA256:AES256-SHA256:RC4:HIGH:MEDIUM:+TLSv1:!MD5:!SSLv2:+SSLv3:!ADH:!aNULL:!eNULL:!NULL:!DH:!ADH:!EDH:!AESGCM
ssl_key = </etc/pki/dovecot/private/dovecot.pem
userdb {
  driver = passwd
}
protocol pop3 {
  pop3_uidl_format = %08Xu%08Xv
}

No 2. This is the problem http://www.imagehost.pitestinet.ro/images/3cj28iz4diyyo2c2hyc.png

No 3. I created the sub-folders from roundcube and squirellmail. I've tryed to download the folders on 2 different PC'S, both with Thunderbird, afaik i don't have anything else here.

Thank you for your reply.

Submitted by andreychek on Wed, 03/06/2013 - 21:50

No 1. it's easy to reproduce, just * get a fresh CentOS 6.3 minimal installed. * Run yum update * Run yum install wget mlocate perl * Install virtualmin and hit recheck config. Might need to create a virtualmin first.

I believe you that it's occurring on your system; the problem I'm having is that I've tried that on several systems, CentOS 6.3 and otherwise, and wasn't able to reproduce that.

The Re-Check Config is one of the more common things that we have folks perform, but we hadn't received reports of problems with that.

Just to clarify -- it didn't just happen once, but it happens every time you run the Re-Check Config?

Jamie, do you have any thoughts on what might cause that?

No 2. This is the problem

Thanks for the screenshot!

I don't imagine you see anything unusual in RoundCube's Settings -> User Interface -> Date Format?

One thing I saw was this bug report here:

http://trac.roundcube.net/ticket/1488696

It looks like someone saw a similar issue when the timezone in their php.ini file was set incorrectly.

For Virtualmin, that's in $HOME/etc/php.ini, and the field is named "date.timezone". What is that set to on your system?

No 3. I created the sub-folders from roundcube and squirellmail. I've tryed to download the folders on 2 different PC'S, both with Thunderbird, afaik i don't have anything else here.

I'll do some testing with that to see what I can figure out. We hadn't received any reports of folks having problems downloading folders though, I'm not quite sure what might cause that.

Submitted by JamieCameron on Wed, 03/06/2013 - 23:27

Regarding the problem with ORGMAIL and DEFAULT, this could be happening because /etc/postfix/main.cf on your system is setup to deliver to /var/mail/$username , and Virtualmin by default configures procmail to match that.

Based on the config you posted, it looks like the line home_mailbox=Maildir/ is missing from /etc/postfix/main.cf . If you add that, all should be good.

Submitted by ITCiao on Thu, 03/07/2013 - 03:58

Hello,

@andreycheck,

No 1. i doubt that it's from my system since it happened on 3 clean installs of http://mirrors.xservers.ro/centos/6.3/isos/x86_64/CentOS-6.3-x86_64-mini... Yes, it happens every time, without a doubt.

  • Update - adding home_mailbox=Maildir/ in /etc/postfix/main.cf fixed the problem. Thank you JamieCameron. This should be available by default.

No 2. Fixed. Thank you.

No 3. Ok. I can supply more info, just tell me what you need.

Submitted by andreychek on Thu, 03/07/2013 - 09:45

Update - adding home_mailbox=Maildir/ in /etc/postfix/main.cf fixed the problem. Thank you JamieCameron. This should be available by default

Well, in theory, it really should be set that way by default. I'll do some digging to see if I can figure out why you're seeing those issues on your systems -- I'll do some test installs with the ISO you mentioned.

What is the output of this command:

rpm -qa | grep postfix

Also, when you performed your Virtualmin installations, did you use the install.sh script?

Thanks!

Submitted by ITCiao on Thu, 03/07/2013 - 13:35

rpm -qa | grep postfix returns

postfix-2.6.6-2.2.el6_1.x86_64

Yes, i used the install.sh script. Same behaviour for Pro and GPL.

Any progress with the Thunderbird folders? That's what burning right now. I need to lunch this by tomorow.

Thank you.

Submitted by andreychek on Thu, 03/07/2013 - 15:17

Howdy! The issues you're describing are all significant, and we wouldn't want those to occur to folks using Virtualmin. I spent quite a bit of time today performing installs and changing settings in order to reproduce those, so we can make sure it never happens.

The problem is, despite starting with the same ISO you linked to above, and using all the default settings -- I haven't been able to reproduce the issues you're describing.

Since running the Re-Check Config is one of the first steps done on any new Virtualmin system (all users get a big yellow warning until that is run) -- we'd expect to be hearing from lots of folks about the problem with the email delivery settings you were seeing.

So, something odd is going on there, and I'm not quite sure what that is yet.

One possibility is that the installation didn't complete successfully, preventing the Postfix config from being updated.

Is there any chance you could attach a copy of your /root/virtualmin-install.log? That may have some clues as to what's going wrong.

Also, regarding the folder problems you're seeing --

I unfortunately haven't been able to reproduce that either.

What I did is, on a fresh CentOS 6.3 install, I installed Virtualmin and created a single VIrtual Server. And in that Virtual Server, I installed RoundCube.

Using RoundCube, I went into the Folder Manager, and created two new sub-folders, a sub-sub folder, and a new top-level folder.

Those were all visible immediately in RoundCube, and I verified on the filesystem (in $HOME/Maildir) that they were created by Dovecot.

Then, I loaded Thunderbird, and connected it to my account.

It immediately began showing any email in the Inbox.

In most cases, it didn't begin showing any other folders yet -- in order to get it to download the other folders, I either had to put an email in the folder, or I had to close and re-open Thunderbird.

But once I re-opened Thunderbird, it automatically downloaded the entire folder structure, including the sub-folders and sub-sub-folders.

I don't imagine the issue you're seeing is just a matter of restarting Thunderbird?

There's a slight variation in our Dovecot configs; it shouldn't make a difference, but just in case, mine does not have the "ssl_cipher_list" set. And my "mail_location" is storing the index and control files in a different location (in the user homedir):

mail_location = maildir:~/Maildir

Other than that, our Dovecot configs are the same.

Submitted by ITCiao on Thu, 03/07/2013 - 15:57

If you used CentOS 6.3 x86_64 minimal then the behaviour should have been easy replicable. Oh wait, did you run yum update before running install.sh ?

This is my http://pastebin.com/smq5szbb

Could this be related to the fact that i disabled IPv6 in /etc/sysctl.conf before running install.sh ?

# Disable IPv6
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1

I restarted Thunderbird more then a few times. I got Inbox, Sent, Trash but none of the custom folders. I added one email in every custom folder just to make sure it downloads something. Any logs that could point of what's wrong?

Submitted by andreychek on Thu, 03/07/2013 - 17:33

If you used CentOS 6.3 x86_64 minimal then the behaviour should have been easy replicable. Oh wait, did you run yum update before running install.sh ?

I used the same .iso as you used :-) I did indeed run "yum update" first.

I understand that all this is frustrating!

Remember though that thousands of people have performed installs using this install.sh script; it hasn't changed since May of 2012. And the .iso we both used hasn't changed since July of 2012.

Also, CentOS 6 is our most popular distro.

If what you were seeing was easy to reproduce, as serious as a problem as that is, we'd be seeing a ton of bug reports about that. However, you're the only one to report it.

That suggests something is different in your setup there, I'm just trying to figure out what.

Reviewing your virtualmin-install.log file, I see this output that occurred during the Virtualmin installation:

Error occurred while configuring Postfix: Failed to regenerate table /etc/postfix/virtual: postmap: fatal: parameter inet_interfaces: no local interface found for ::1
at ../web-lib-funcs.pl line 1361.

Enabling Postfix and disabling Sendmail
newaliases: fatal: parameter inet_interfaces: no local interface found for ::1
Failed to start Postfix!

According to that error, it looks like the ipv6 settings that were changed may indeed be involved in the problem you're seeing. I suspect that once the Postfix errors were seen, Virtualmin stopped attempting to configure Postfix, which caused the home_mailbox to not be set correctly.

Regarding the problem with the mail folders --

If you close RoundCube, and log back into it, does it correctly show all your folders?

If so, that should mean that Dovecot is correctly providing that folder information. RoundCube doesn't appear to cache the folder names, so that would mean it's pulling the names from Dovecot. You could also try the same with Squirrelmail just to be sure.

If that doesn't work, one thing you could try is to go into the "Maildir" folder within that user's home directory, and run "ls -la". Do you see the email folders listed there? And are they owned by the email user?

Also, if Dovecot is generating any errors, those would show up in /var/log/maillog.

In Thunderbird, if you click your account name on the left, and then click "Manage folder subscriptions", are you able to subscribe to your missing folders there?

Submitted by ITCiao on Thu, 03/07/2013 - 17:51

That explains everything. Note for future instalations. Don't disable IPv6 before install.

  • Roundcube and SqirellMail both show the folders correctly if i relog.
  • No errors in /var/log/maillog
  • ls -la is working correctly.

Omg you are right. If i right click in thunderbird and manage subscriptions from the client i can see the folders. But i think they should have been selected automaticly, no? Could it be related to autoconfig?

Thank you for your support. 10+

Submitted by andreychek on Thu, 03/07/2013 - 21:46

I agree, Thunderbird should normally retrieve that folder structure by default.

Since RoundCube and Squirrelmail are getting that folder structure, that means Dovecot is correctly providing it.

That may mean you're seeing some sort of Thunderbird issue... it's possible it's set to not automatically retrieve that. It also could be a problem with the particular Thunderbird version that you're using, since it does work in the various versions I've tried (I haven't changed any Thunderbird defaults).

I don't believe it's autoconfig related -- I don't think that configures anything related to folders, I think it's just the initial account details -- though I'll verify that with Jamie.

Submitted by andreychek on Thu, 03/07/2013 - 21:46

I agree, Thunderbird should normally retrieve that folder structure by default.

Since RoundCube and Squirrelmail are getting that folder structure, that means Dovecot is correctly providing it.

That may mean you're seeing some sort of Thunderbird issue... it's possible it's set to not automatically retrieve that. It also could be a problem with the particular Thunderbird version that you're using, since it does work in the various versions I've tried (I haven't changed any Thunderbird defaults).

I don't believe it's autoconfig related -- I don't think that configures anything related to folders, I think it's just the initial account details -- though I'll verify that with Jamie.

Submitted by andreychek on Fri, 03/08/2013 - 17:36

Jamie says that there isn't any folder related data stored in the autoconfig that Virtualmin performs.

It's only account setup information, such as the hostname, port, protocol, and login.

Thunderbird really should be showing you all the folders -- if you aren't using the most recent version, you may want to try upgrading, just to make sure it's not a bug.

However, it's also possible that the next Virtualmin version that comes out, which should be soon, may help, as it does some things differently with the Dovecot subscriptions file.

This really shouldn't matter, and I wasn't able to reproduce the problem you're seeing with the current Virtualmin version -- but if it is indeed an email client bug, the next Virtualmin version may help work around it.

Submitted by ITCiao on Fri, 03/08/2013 - 18:31

Ok Andrey,

That settles it. Thank you for your support, you can close the ticket.