8 domains using all my memory?

14 posts / 0 new

Topic locked

Last post

#1 Fri, 03/28/2008 - 08:59

LHaig

8 domains using all my memory?

Hi,

I am running my virtualmin install on a VPS with the following spec

debian 4 720 MBram 3Gb hdd 512mb swap.

8 domains and about 15 accounts over all the domains.

The problem I have is that after a while the vps runs out of memory. and debian starts shutting down processes.

Where do i start to fault find this?

#2 Fri, 03/28/2008 - 09:04

Joe

Though your system doesn't count as "low memory" you might find some useful information in the guide for running Virtualmin on low memory systems--it discusses the memory hogs within a hosting system, and explains some of the reasons for their size, and how to reduce the size.

http://www.virtualmin.com/documentation/id,virtualmin_on_low_memory_syst...

top or the process list is a good place to start, though it can be tricky to interpret the data accurately (for example, a system with 8 domains might have 30 or 40 PHP processes, if you have a lot of applications running--but a lot of the memory they use is shared libraries, and so just adding up the 25MB or so each one looks like it is taking isn't actually what's happening (they actually take about 12MB per process).

I suspect mail processing is a culprit. If you get a lot of mail, at any given time you'll have multiple SpamAssassin and clamscan processes running--both of which can be quite resource-hungry. You might try switching over to spamc and clamdscan, instead, which are faster daemonized versions of the programs.

If you'd like to post the process list, we might be able to point you at the specific culprits.

Check out the forum guidelines!

#3 Sun, 06/07/2009 - 07:20 (Reply to #2)

LHaig

Tasks: 154 total, 19 running, 135 sleeping, 0 stopped, 0 zombie
Cpu(s): 87.7%us, 12.3%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 718600k total, 683228k used, 35372k free, 8352k buffers
Swap: 524280k total, 242940k used, 281340k free, 63332k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1914 root 20 0 10168 7940 7804 R 1.3 1.1 0:00.27 apt-cache
1470 1042 20 0 34704 8520 1088 R 1.0 1.2 0:24.08 clamscan
1480 casey@ha 20 0 34704 8744 1088 R 1.0 1.2 0:24.24 clamscan
1481 1042 20 0 34704 6100 1088 R 1.0 0.8 0:23.99 clamscan
1578 lhaig@ha 20 0 34704 28m 1088 R 1.0 4.1 0:23.30 clamscan
1588 frans@ha 20 0 34704 29m 1088 R 1.0 4.1 0:23.31 clamscan
1598 glenn@ha 20 0 34704 28m 1088 R 1.0 4.1 0:23.19 clamscan
1604 frans@ha 20 0 34704 29m 1088 R 1.0 4.2 0:23.23 clamscan
1608 glenn@ha 20 0 34704 28m 1088 R 1.0 4.1 0:23.22 clamscan
1614 lhaig@ha 20 0 34704 29m 1088 R 1.0 4.1 0:23.14 clamscan
1618 1018 20 0 34704 29m 1088 R 1.0 4.2 0:23.15 clamscan
1624 paul@hai 20 0 34704 29m 1088 R 1.0 4.1 0:23.05 clamscan
1630 1001 20 0 34704 29m 1088 R 1.0 4.2 0:24.10 clamscan
1666 casey@ha 20 0 34440 28m 1088 R 1.0 4.1 0:18.05 clamscan
1739 paul@hai 20 0 33648 28m 1088 R 1.0 4.0 0:11.08 clamscan
1805 nell@hai 20 0 32988 27m 1088 R 1.0 3.9 0:07.03 clamscan
1867 root 20 0 51832 49m 1576 R 1.0 7.0 0:01.34 collectinfo.pl
1886 fred@hai 20 0 24704 19m 1088 R 1.0 2.8 0:00.68 clamscan
1520 dovecot 20 0 3292 1156 1092 S 0.3 0.2 0:00.04 pop3-login
1669 root 20 0 2352 1124 812 R 0.3 0.2 0:00.79 top
1 root 20 0 1940 572 552 S 0.0 0.1 0:00.80 init
2 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 kthreadd
3 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/0
4 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 events/0
5 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 khelper
43 root 15 -5 0 0 0 S 0.0 0.0 0:00.03 kblockd/0
60 root 20 0 0 0 0 S 0.0 0.0 0:00.00 pdflush
61 root 20 0 0 0 0 S 0.0 0.0 0:00.00 pdflush
62 root 15 -5 0 0 0 S 0.0 0.0 0:00.22 kswapd0
63 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 aio/0
71 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 jfsIO
72 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 jfsCommit
73 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 jfsSync
74 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 xfslogd/0
75 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 xfsdatad/0
76 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 xfs_mru_cache
635 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 kcryptd/0
636 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 ksnapd
656 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 rpciod/0

#4 Sun, 06/07/2009 - 07:20 (Reply to #3)

LHaig

#5 Fri, 03/28/2008 - 09:45 (Reply to #4)

LHaig

Thanks Joe

I will make the changes

Lance

#6 Fri, 03/28/2008 - 09:49 (Reply to #5)

LHaig

Joe,

I get an error

The selected virus scanning command does not work :

connect(): No such file or directory
WARNING: Can't connect to clamd.

----------- SCAN SUMMARY -----------
Infected files: 0
Time: 0.020 sec (0 m 0 s)

but I can see clamd in the process list

Do I need to check anything?

#7 Fri, 03/28/2008 - 13:34 (Reply to #6)

LHaig

Joe,

Thanks a million. I had to turn it on and off a few time for it to work but now my server is sane and happy

Thanks

Lance

#8 Thu, 10/30/2008 - 06:46 (Reply to #7)

beat

hm, still no luck...Now it's clamd which uses 100% of one cpu core for 15 minutes (and it's a fast server) and it's still runing....disabling again :(

(debian 4.0 stable up-to-date).

Does ubuntu 8.04 LTS do same ?

#9 Thu, 10/30/2008 - 12:04 (Reply to #8)

Joe

<div class='quote'>hm, still no luck...Now it's clamd which uses 100% of one cpu core for 15 minutes (and it's a fast server) and it's still runing....disabling again :(

(debian 4.0 stable up-to-date).

Does ubuntu 8.04 LTS do same ?</div>

Probably. clamav has gotten extremely slow to startup in recent versions on some operating systems--Debian is definitely one, and since Ubuntu shares almost everything with Debian (except a bit of desktop-y stuff), it's almost certainly the same. Our CentOS packages are quite a bit faster, though I don't know why. Pre 0.9x versions were <i>dramatically</i> faster...so fast, in fact, that one could use clamscan on a reasonably high load server, and it worked just fine. 0.9x versions have wildly varying performance, and 0.90-0.93 versions seem to be the slowest.

But, once started clamd doesn't take much time at all--but it really can take 15 minutes or more to startup. On one customers VPS system it took over 30 minutes to load all of the virus definitions and start up. But once started, it ran fine and fast.

I don't know why clam, in some versions, is so stupidly slow. It just is, and I don't know what to do about it, except grin and bear it while it starts up...and make sure you don't have to start it up very often.

Check out the forum guidelines!

#10 Sat, 11/08/2008 - 03:59 (Reply to #9)

Karl

<b>LHaig wrote:</b>
<div class='quote'>Joe,

I get an error

The selected virus scanning command does not work :

connect(): No such file or directory
WARNING: Can't connect to clamd.</div>

I always get this error if I try to switch from "clamscan" to "clam<b>d</b>scan". And the 99% CPU effect with clamd too. Meanwhile on two different servers with Debian etch.

I found the following solution:

I updated ClamAV following this instruction:
http://www.clamav.net/download/packages/packages-linux/

After installing and updating everything runs well.

Karl

#11 Sat, 11/08/2008 - 10:16 (Reply to #10)

Joe

Yes, you have to turn on the clamd service (found in the Bootup and Shutdown module, or just on the command line). clamdscan has to have a server to talk to.

But, yes, I recommended the volatile repository for Debian in another thread. I'm pretty sure it was a thread you started a few days ago. ;-)

Check out the forum guidelines!

#12 Fri, 03/28/2008 - 09:09

Joe

Definitely mail processing. Switch over to clamdscan instead of clamscan--that'll save a lot of memory in your case.

clamscan on Debian is stupidly slow. It takes like three minutes to run on even a fast dedicated system...I'd hate to think how slow it is on a VPS. Switching to clamd and clamdscan means that the three (or maybe ten on a VPS) minute process only happens when the system boots and not every time an email comes in.

Check out the forum guidelines!

#13 Fri, 03/28/2008 - 09:15

LHaig

How do I switch?

#14 Fri, 03/28/2008 - 09:28 (Reply to #13)

Joe

Look in the Email Messages menu, and click on "Spam and Virus Scanning"

There you can switch to spamc and clamdscan. You'll need to click the "Enable ClamAV Scanner" button to start up clam. And you'll also need spamassassin to be running. It's a regular service that can be enabled in the Bootup and Shutdown module in Webmin.

Check out the forum guidelines!

Topic locked