User over quota, not receiving e-mail

Hi,

Today we had a user who is over quota. Unfortunately, the mail is being accepted but ( I guess) being thrown away because of the over-quota. Searching on this forum I see that that it is a known problem because of postfix.
What I don't understand is that all the virtual servers and users have only soft limits, hard limits are unlimited. We want to know which users are over-quota, but don't want these kind of problems to occur. That's why the hard limits are unlimited.

Are we doing something wrong? I thought that it would only be advisory quotas, but now it doesn't look like that.

Thanks.

Joshua.

Maillog:

Oct 3 11:23:40 beheer spamd[1205]: Error writing to /home/fital01/.spamassassin/bayes.lock.beheer.webinteractive.nl.1205: Disk quota exceeded at /usr/share/perl5/Mail/SpamAssassin/Locker/UnixNFSSafe.pm line 88, line 302.
Oct 3 11:23:41 beheer spamd[1205]: plugin: eval failed: bayes: (in learn) error closing /home/fital01/.spamassassin/bayes.lock.beheer.webinteractive.nl.1205: Disk quota exceeded at /usr/share/perl5/Mail/SpamAssassin/Locker/UnixNFSSafe.pm line 118, line 302.
Oct 3 11:32:31 beheer spamd[1205]: Error writing to /home/fital01/.spamassassin/bayes.lock.beheer.webinteractive.nl.1205: Disk quota exceeded at /usr/share/perl5/Mail/SpamAssassin/Locker/UnixNFSSafe.pm line 88, line 112.
Oct 3 11:36:59 beheer spamd[3356]: Error writing to /home/fital01/.spamassassin/bayes.lock.beheer.webinteractive.nl.3356: Disk quota exceeded at /usr/share/perl5/Mail/SpamAssassin/Locker/UnixNFSSafe.pm line 88, line 266.
Oct 3 11:36:59 beheer spamd[1205]: Error writing to /home/fital01/.spamassassin/bayes.lock.beheer.webinteractive.nl.1205: Disk quota exceeded at /usr/share/perl5/Mail/SpamAssassin/Locker/UnixNFSSafe.pm line 88, line 266.
Oct 3 11:37:07 beheer spamd[3356]: plugin: eval failed: bayes: (in learn) error closing /home/fital01/.spamassassin/bayes.lock.beheer.webinteractive.nl.3356: Disk quota exceeded at /usr/share/perl5/Mail/SpamAssassin/Locker/UnixNFSSafe.pm line 118, line 266.
Oct 3 12:04:24 beheer spamd[1205]: Error writing to /home/fital01/.spamassassin/bayes.lock.beheer.webinteractive.nl.1205: Disk quota exceeded at /usr/share/perl5/Mail/SpamAssassin/Locker/UnixNFSSafe.pm line 88, line 129.
Oct 3 12:06:43 beheer spamd[1205]: Error writing to /home/fital01/.spamassassin/bayes.lock.beheer.webinteractive.nl.1205: Disk quota exceeded at /usr/share/perl5/Mail/SpamAssassin/Locker/UnixNFSSafe.pm line 88, line 29.
Oct 3 12:11:15 beheer spamd[1205]: Error writing to /home/fital01/.spamassassin/bayes.lock.beheer.webinteractive.nl.1205: Disk quota exceeded at /usr/share/perl5/Mail/SpamAssassin/Locker/UnixNFSSafe.pm line 88, line 529.
Oct 3 12:12:24 beheer dovecot: IMAP(jornenyvet01): o_stream_send(/home/jornenyvet01/Maildir/.Notes/dovecot-uidlist.tmp) failed: Disk quota exceeded
Oct 3 12:31:27 beheer dovecot: IMAP(fital01): o_stream_send(/home/fital01/Maildir/.Drafts/dovecot-uidlist.tmp) failed: Disk quota exceeded

Status: 
Active

Comments

Howdy -- it looks like the user "fital01" is one of the users having trouble.

What output do you get if you run this command:

repquota / | grep fital01

Hi,

We changed the quota of fital01, but we have another user who has the same problems:
jornenyvet01

When I do this:
repquota /home | grep jornenyvet01
output is:
jornenyvet01 -- 548736 0 0 14908 0 0

Because
repquota / | grep jornenyvet01
gives

repquota: Mountpoint (or device) / not found or has no quota enabled.
repquota: Not all specified mountpoints are using quota.

Mmm, it doesn't appear that the jornenyvet01 user actually has a quota.

However, what I'm hoping to get from the fital01 user is whether it's using a hard quota or a soft quota, and what the grace period is set to (if anything).

So while we may need to figure out what's going on with this one user, maybe we can start by taking a look at the quota output for fital01 first.

And you're right, on your server, you'd want to run the repquota command with "/home" rather than just "/".

Thanks!

Hi!

The output for:
repquota /home | grep fital01
is
fital01 -- 2105168 0 0 29411 0 0

Hope you can find the problem.

Thanks in advance!

Hmm, that user doesn't appear to have quotas enabled either... just to make sure it's not an issue with repquota, what does this command return:

quota -v fital01

Also, if you look at that user in Virtualmin, does it show them as having a quota setup?

Okay, so that user does indeed seem to be set to "unlimited" -- that is, there isn't a quota setup for them.

Was that your intention?

The steps we're going through should hopefully assist in determining the root of the problem you're seeing... the issue here is that, according to the filesystem, the jornenyvet01 user didn't have a quota set either.

If you look at that user in Virtualmin, what is it's quota set to?

Hmm, it looks like that particular user has been setup with no quota either though... in Virtualmin, they're setup with an "Unlimited" disk quota, and it looks like the filesystem agrees there.

With soft quotas, there should still be a quota set, it would just be soft rather than hard.

But with those two users (jornenyvet01 and fital01), since they're set to unlimited, they shouldn't be running into problems with soft or hard quotas at the moment.

Do you have an example of a Virtual Server where there is a quota set?

Once we find one, we can look at the quota settings for that account and determine why you're seeing that behavior.

There are two cases where the behavior you described would come up:

  1. If hard quotas are being used

  2. If soft quotas are being used, and a grace period was set (and surpassed)

I reviewed your system, and I don't see any grace periods set for any of your current users. So #2 shouldn't be an issue.

But, I also don't see any users with hard quotas -- the few users that do have a quota are all set to use a soft quota.

I spoke with Jamie about about all this, and one of the things he pointed out was that changing the Server Templates from Hard quotas to Soft quotas would only affect new users, not existing users.

Is it possible that the user you had trouble with was setup before you had made the switch to soft quotas? If so, that would explain the problem you ran into.

Hi,

Thank you so much for your research.

I guess what Jamie might be of influence in how we setup Virtualmin. Let me explain.

When we installed Virtualmin for the first time, we were trying different configurations. In the end we decided to only use soft quota, because we don't want to be that strict. We made a new server template, cloned from the default template. The default template has hard quota enforced. We did however change the quota of the derived template to soft quota. After that we created the virtual servers. It looks like that what Jamie is saying kind of is related to the problems we run into.

What would be advisable to do to solve the problems?

What would be advisable to do to solve the problems?

Well, since all the quotas now appear to be soft quotas, and no grace periods are set -- it looks as if you won't run into that problem with any of your current users.

I suspect you ran into that before due to a hard quota being in use for some specific users.

I don't think you'll run into problems with that any longer though. If for some reason you do, let us know so we can look into that. If you're able to let us know before you change that quota, that would be helpful!

Hi,

The user jornenyvet01 are still having quota problems. Here is the maillog of about 20 minutes old:

Oct  5 13:56:18 beheer dovecot: IMAP(jornenyvet01): Disconnected for inactivity bytes=229/1032
Oct  5 13:56:18 beheer dovecot: IMAP(jornenyvet01): o_stream_send(/home/jornenyvet01/Maildir/.Notes/dovecot-uidlist.tmp) failed: Disk quota exceeded
Oct  5 14:06:51 beheer dovecot: imap-login: Login: user=<jornenyvet01>, method=PLAIN, rip=83.87.40.172, lip=172.16.1.20
Oct  5 14:07:02 beheer dovecot: imap-login: Login: user=<jornenyvet01>, method=PLAIN, rip=83.87.40.172, lip=172.16.1.20
Oct  5 14:07:41 beheer dovecot: IMAP(jornenyvet01): Disconnected: Logged out bytes=237/1041
Oct  5 14:11:50 beheer spamd[2434]: spamd: setuid to jornenyvet01 succeeded
Oct  5 14:11:50 beheer spamd[2434]: spamd: processing message <20111005120148.942.qmail@app1> for jornenyvet01:1129
Oct  5 14:11:51 beheer spamd[2434]: spamd: clean message (3.4/5.0) for jornenyvet01:1129 in 0.6 seconds, 11967 bytes.
Oct  5 14:11:51 beheer spamd[2434]: spamd: result: . 3 - HK_LOTTO,HTML_IMAGE_RATIO_04,HTML_MESSAGE,RCVD_IN_DNSWL_NONE scantime=0.6,size=11967,user=jornenyvet01,uid=1129,required_score=5.0,rhost=localhost,raddr=127.0.0.1,rport=40841,mid=<20111005120148.942.qmail@app1>,autolearn=no
Oct  5 14:11:51 beheer postfix/local[3409]: B04E529CD9: to=<jornenyvet01@beheer.webinteractive.nl>, orig_to=<topgear@jornenyvet.nl>, relay=local, delay=0.72, delays=0.04/0/0/0.69, dsn=2.0.0, status=sent (delivered to command: /usr/bin/procmail-wrapper -o -a $DOMAIN -d $LOGNAME)
Oct  5 14:37:04 beheer dovecot: IMAP(jornenyvet01): Disconnected for inactivity bytes=229/1032
Oct  5 14:37:04 beheer dovecot: IMAP(jornenyvet01): o_stream_send(/home/jornenyvet01/Maildir/.Notes/dovecot-uidlist.tmp) failed: Disk quota exceeded
Oct  5 14:40:28 beheer dovecot: imap-login: Login: user=<jornenyvet01>, method=PLAIN, rip=83.87.40.172, lip=172.16.1.20
Oct  5 14:41:00 beheer dovecot: IMAP(jornenyvet01): Disconnected: Logged out bytes=237/1041
Oct  5 14:41:39 beheer dovecot: imap-login: Login: user=<jornenyvet01>, method=PLAIN, rip=83.87.40.172, lip=172.16.1.20
Oct  5 15:11:41 beheer dovecot: IMAP(jornenyvet01): Disconnected for inactivity bytes=229/1032
Oct  5 15:11:41 beheer dovecot: IMAP(jornenyvet01): o_stream_send(/home/jornenyvet01/Maildir/.Notes/dovecot-uidlist.tmp) failed: Disk quota exceeded
Oct  5 15:23:42 beheer dovecot: imap-login: Login: user=<jornenyvet01>, method=PLAIN, rip=83.87.40.172, lip=172.16.1.20
Oct  5 15:24:12 beheer dovecot: imap-login: Login: user=<jornenyvet01>, method=PLAIN, rip=83.87.40.172, lip=172.16.1.20
Oct  5 15:24:52 beheer dovecot: IMAP(jornenyvet01): Disconnected: Logged out bytes=237/1041

So we are still having problems and why I asked you kindly to take a look at jornenyvet01.

So we are still having problems and why I asked you kindly to take a look at jornenyvet01.

Mmm, while I did look at that user, I didn't realize they were currently experiencing that problem. I figured they were okay since they didn't have a quota set.

Knowing that gave me an idea of where to look, and I think I see where the issue lies. I'll get back with you shortly regarding what to do about that.

Okay, so, it's correct that the user jornenyvet01 has no user quota set.

However, that user does have a group quota set -- since it's a Virtual Server owner, that's how the quota for the entire Virtual Server is enforced, so that no specific Virtual Server can exceed an allotted amount of space.

So, that particular Virtual Server as a whole is using too much space.

Now, in your case, you would like to use soft quotas.

Systems handle this differently, but some have a default grace period set. When a grace period is set along with a soft quota -- the user may only exceed their soft quota for the length of the grace period.

It looks like that's set to 7 days on your system... so that means once a user (or Virtual Server) surpasses their quota, they'd have 7 days to reduce how much space they're using, or else it would prevent them from adding new files.

Before we get to how to get things working the way you mentioned -- I did want to offer that we recommend using hard quotas. Without that, a user (or a malfunctioning application) could accidentally use up all the space on the server, which would cause a lot of trouble :-)

Using soft quotas and grace periods (what you have now) is a compromise to that... there's still a risk that a user could use up all the space on the server, but it would have to happen all within a week. If you're using the quota notifications (setup in Limits and Validation -> Disk Quota Monitoring), users would be notified that they're over quota, and would have a week to reduce their usage.

Knowing that, if you'd still like to use soft quotas and not use grace periods (ie, users can technically use as much space as they want for as long as they want), let me know and I'll work with Jamie to figure out what's needed to get all that changed for all of your users.

It's possible that's as simple as:

  1. Going into Webmin -> System -> Disk Quotas -> Users -> Edit Grace Times, and change both Block and Time to 0

  2. Going into Webmin -> System -> Disk Quotas -> Groups -> Edit Grace Times, and change both Block and Time to 0

But I need to verify that the above effects existing users as well as new ones.

Again though, we don't recommend that particular option, as it makes it easy for bad things to accidentally happen :-)

Hi Eric,

Thanks for your analysis and your detailed reply. We have deliberated for about a week now and we made the decision to follow your advice to enforce hard quota because of your mentioned arguments. Is it possible to set the hard quota on 5gb and set the soft quota on 1gb? We want to still have control over it, but don't want to enforce it. But you are right about if you set it to unlimited the virtual servers could bring the server down. That is of course something you don't want.

When I change the server template, it will only affect new virtual servers right? How can we apply it to the existing virtual servers?

Well, here's the problem... it's not easy to change this mid-stream, and it's also not possible to use both hard and soft quotas on the same account.

But, we can certainly discuss your options :-)

So, if you were to go into System Settings -> Server Templates -> Default -> Administrative User, and change the Disk Quota Type from Soft to Hard -- that will only affect new users.

The only way to change existing users would be to backup your mailbox users, change the quota format, then restore the mailbox users.

You can generate a backup of just the users, it doesn't need to be a full backup.

However, as far as using both a soft and hard quota for your users -- Virtualmin unfortunately doesn't support a setup such as that.... you would need to choose between either a hard quota or a soft quota.