load average very high with peaks, server far too slow

What "load" exactly are we talking about here? CPU? Network traffic? Connections per second? What unit exactly is "40" and "2 to 3"?

I suspect the units he's referring to are the output of "uptime"... so 40 is pretty high in that case.

When the load spikes, try running "top"... are there some processes that just seem to sit there at the top of the list, consuming a lot of resources?

Also, review the output of "mailq" -- are there a lot of emails in the queue? A lot of emails sitting there could mean someone is making heavy use of a newsletter. It could also mean a spammer figured out how to send email through someones account :-) So you'd want to make sure the mail in the queue seemed legitimate.

What also might be helpful is to run "ps auxw", and attach that output here. That will show a list of running processes, and we might be able to figure out if any seem out of the ordinary.

-Eric

Oh hell, yeah I just read up on the meaning of the "load average" output. As far as I understand it, a value of "40" kinda means that the system way overloaded about 40 times. :-) ("1" would mean that the CPU was busy working on one process all the time, while "2" means that while the CPU was loaded, another process was waiting to get CPU cycles all the time, and so on.)

And I agree, check the process list during "overload" times. atop is a nice alternative to top too, which has a somewhat better-arranged output, since it can be configured to only show processes that produce a notable load.

Some hints about atop: Press "i" to set a new refresh interval. "a" toggles between "show all process" and "show only those with load". "t" triggers refresh manually. "?" shows a help screen.

Okay, so it looks like you have 800+ emails in your mail queue. I'd classify that as quite a bit :-)

The next step is to determine if they're legitimate or not. To do that, you can log into Virtualmin, and go into Webmin -> Servers -> Postfix -> Mail Queue, and click on some of the messages.

Does the message appear to be a newsletter or mailing from one of your users? Or does it look "spammy"?

-Eric

There's a lot of connections to your web server. You might check your Apache logs for details, which URLs have been requested.

By reading the thread I draw a conclusion that a user sends a newsletter and recipients vist his/her website.
If that conclusion is right or false needs to be seen.

If I send out a newsletter to 4500+ recipients, it is not even noticible..
What specs does the server have?

Indeed, a mailserver should not cause such a noticeable CPU load when sending out mails. The ps output implies that it's Apache causing the high CPU and memory load. It seems like a lot of people are requesting web pages at the same time, which would also explain the high number of connections to port 80.

Hint: The atop application, in mode "p", will show all resources used by one application accumulated over all processes.

Interesting is the high count of Dovecot processes doing "pop3-login" and "imap-login". How many users does the server have that might concurrently be reading their mail? :)

Actually, your netstat shows only three direct connections to IMAP, and none to POP3. Are you maybe running webmail via your Apache? That might explain the great lot of Apache connects in combination with the Dovecot processes.

If the problems with mail logins persist, you might turn on the Linux firewall and have it record a log of connections, in addition to what the other servers themselves log.

You should be able to setup firewalling in Webmin without an additional module... just take a peek in Webmin -> Networking -> Linux Firewall.

As an aside, if you haven't already, I'd definitely take a look at some of those messages in your mail queue... if those are spam of some sort, you'd want to clear them out. You can do that from Webmin -> Servers -> Postfix -> Mail Queue.

If they're spam, that may mean that a spammer is sending those though a security vulnerability in one of your web apps. And in that case, a firewall isn't likely to help your issue, since all a firewall would see is incoming web traffic.

The key would be to determine what web app is the culprit, and fix the security issue.

When looking at messages in your mail queue, click the "View all headers" option on the right, and look for the "Received" header at the top. It should say what userid it was received from, if it was generated from your server. That should help you track down the culprit.

-Eric

I agree with Eric there... Make sure no spammer can abuse your server. Such a thing can result in quite some trouble if it persists.

The firewall suggestion was rather meant to log connections to syslog (which is one of iptables' features) and find out which IP addresses connect to where, to have something to work with.

So those mails in your queue are basically "undeliverable" replies from the mail-daemon due to nonexistent local email addresses? The "From" lines in the bounces look like it was an attempt to send spam to them. :)

It's a bit odd. Delivery attempts to unknown local addresses should be denied with a 550 error code during delivery, and not trigger a bounce-mail. You might want to check your /var/log/mail.log at the time when such a mail comes in, there might be hints there why that happens.

I don't know, that's why I suggested checking the logfile. :) Without further information about why those non-deliverable mails are sent, it's hard to say.

Okay, I'm not sure what caused that, but I think I fixed your two posts.

I already contacted Joe to see if we can prevent that from happening in the future :-)

-Eric

Wow, This forum support is great. Specially some guys are helping a lot