Hi there,
I fell in love with Virtualmin (coming from WHM) quite recently and just moved all websites with low traffic to our new virtualmin powered server.
From the performance point of view, the server is amazing. All websites are really snappy.
However, the RAM usage is crazy (in my opinion). It usually averages around 80%, at the same time the highest CPU usage I saw so far is 24%. Screenshot: https://marketingbear.com/virtualmin/server.png Here are the server specifications:
Customizations I made so far:
When I click on the memory bar in Webmin, it shows mostly NGINX processes (4 to 6 per user), however, htop via SSH outputs this: https://marketingbear.com/virtualmin/htop.png
So based on htop it looks like MySQL is the culprit. I already tried to reconfigure MySQL (MariaDB). MySQL tuner outputs this: Maximum possible memory usage: 2.9G (38.00% of installed RAM) Which sounds reasonable to me.
For completion this is my MySQL configuration:
[mysqld]
skip-name-resolve
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
symbolic-links=0
innodb_file_per_table = 1
thread_concurrency = 8
query_cache_size = 0
query_cache_type = 0
thread_cache_size = 8
myisam_sort_buffer_size = 16M
read_rnd_buffer_size = 4M
read_buffer_size = 2M
sort_buffer_size = 2M
table_open_cache = 512
max_allowed_packet = 1M
key_buffer_size = 32M
# MyISAM #
key-buffer-size = 4M
# SAFETY #
max-allowed-packet = 2M
max-connect-errors = 1000000
# BINARY LOGGING #
expire-logs-days = 14
sync-binlog = 1
# CACHES AND LIMITS #
tmp-table-size = 16M
max-heap-table-size = 16M
query-cache-type = 0
query-cache-size = 0
max-connections = 300
thread-cache-size = 40
open-files-limit = 65535
table-definition-cache = 2048
table-open-cache = 2048
# INNODB #
innodb-flush-method = O_DIRECT
innodb-log-files-in-group = 2
innodb-log-file-size = 32M
innodb-flush-log-at-trx-commit = 1
innodb-file-per-table = 1
innodb-buffer-pool-size = 256M
[mysqld_safe]
log-error=/var/log/mariadb/mariadb.log
pid-file=/var/run/mariadb/mariadb.pid
!includedir /etc/my.cnf.d
Another item that came to my attention is that the fail2ban service comes up very often and with a large memory footprint when I run the "top" command. I wonder if there is any way to reduce the amount of memory it takes.
This is the output of the "free -m" command: https://marketingbear.com/virtualmin/free-m.png
I also noticed that the average percentage of the used memory increases by 3 to 5% for each virtual server I create, even if DNS for the new domains are not pointing at my server yet and there is no traffic for these new virtual servers.
In total I am having 11 virtual servers in the system, but only 9 are used with low traffic (less than 50 visitor per day).
My biggest concern is that once one of the websites gets actual traffic that the server is unable to handle the request, because it does not seem to utilize the swap space effectively and has a high memory footprint when it is pretty much idle as it is right now.
If anyone has an idea to lower the amount of memory used or tell me that I do not need to worry, I would really appreciate it.
Thank you very much.
I have found clamav and similar programs to be memory hogs. Try checking the for memory useage via linux command shell according to programs/processes currently running on the server...this will allow you to see which ones are sucking it all up.
https://ajecreative.com.au
Hi adamjedgar,
Thank you for your response. I do not think that this is the issue. If you have the chance, please see the output of "free-m" and my comment for "htop" command. ClamAV did not show up with high memory consumption and I hope it stays that way :D
Thank you very much!
WordPress enthusiast
Your MySQL conf is set to use up to 3.3GB of memory and this is MySQL alone. If you dont send or receive a lot of emails try to disable Spamassasin and ClamAV clients and instead use them as "standalone". It will go little heavier on CPU each time you get/send email but it will use much less memory. This is ok if your CPU isnt already maxed out or constantly over 70-80% in which case you already need to have a solution to migrate to better server. While you should always leave some free memory for server needs remember that unused memory is just waste of resources (literally). So to have 8GB memory and using 50% majority of time isnt productive, cost efficient and like i previously said - its just waste of server resources.
- I often come to the conclusion that my brain has too many tabs open. -
Failing at desktop publishing & graphic design since 1994.
Thank you for your response Diabolico. Spamassasin and ClamAV are already disabled (we are not using any email functionality).
However, I was able to decrease the average memory usage to roughly 65%, by switching most of the domains to PHP-FPM (it took me one and a half days to configure it probably...I think I made a PHP-FPM mess when configuring the server initially). Also, I optimized the MySQL configuration slightly, but most impact had the switch to PHP-FPM.
I guess 65% is a good average value, but if there is anything you can think of which could help to decrease the memory amount, please let me know: Specifically, I would be more interested in optimizing Fail2Ban as it sometimes comes up with 2GB memory usage.
Thank you so much.
WordPress enthusiast
The consumption of Fail2Ban isnt precise otherwise all my servers with 4GB of memory would be constantly OOM. Not sure but i think i saw some comment from one of the devs about this subject.
Next:
1. Not sure of the final result but my.cnf you posted really requires some attention.
2. On servers with less than 32GB memory you could set InnoDB (
innodb-buffer-pool-size) to 50% of system memory and for DB dedicated servers up to 80%. Just remember two things: backup (!!!) entire DB before any change to InnoDB andinnodb-log-file-sizeshould be the result ofinnodb_buffer_pool_size * innodb_log_files_in_group / 4.Not sure what you have on your server but if you are not using email function (e.g. 3rd party service) then the server should be fine with 1-1.5GB of free memory. Still you should keep an eye on your server more frequently at least for a week (or two) to be sure there is enough free memory. Everything else can be used by MySQL and caching.
Like i previously said, there is no point to have unused memory so dont be afraid to push the limits. You said now is sitting at 65% what would be around 5-5.1GB of 8 available. I would say go for 6GB because 2GB is more than enough for other functions, and if you see the server doesnt use at peak times more than 500MB then you can go up for another 1/2-1GB for MySQL or Apache.
- I often come to the conclusion that my brain has too many tabs open. -
Failing at desktop publishing & graphic design since 1994.