Hello everybody: I'm curious how Virtualmin calculates total disk space in general. The reason I ask, is I have a dedicated server I'm leasing froma datacenter on a monthly basis that I thought I'd install Virtualmin GPL on, at least for the moment. At some point down the road however, I hope to upgrade to a Pro subscription because I'd like the capability to install WordPress etc as a 1-click installation, as I'm honestly not a large fan of manual tweeking/configuration if it can be done through an automated system where possible. I don't have the time to attempt and write my own WordPress script for useage with GPL at this point either because I've got a lot bigger issues such as site redesign things to deal with at the moment that will be a bit more pressing as time goes on at this point. So what I'm wanting to know at least running my testing, hear is what I find. I install a minimal CentOS6 OS. Then because the server uses LVM automaticly, I want to use the entire drive so: lvextend -l +100%FREE /dev/mapper/vg00-home After extending that: resize2fs -p /dev/mapper/vg00-home once this successfully happens, I tend to install Virtualmin at that point. So: wget http://software.virtualmin.com/gpl/scripts/install.sh sh install.sh After answering all questions, I configure the primary setup through the wizard. But once everything is done before I've even setup a domain, I see Virtualmin is using 40 percent of my servers hosting drive? In otherwords 40GB! That is absolutely uncalled for, because du -hs and other commands don't even show that much space being used. I've even talked with CentOS devs and such, and not even they agree with me that Virtualmin is teling the actual truth as far as disk space percentages. Nobody that I talk with sees how this is possible. Can anyone here explain in a way that makes sense how therefore, Virtualmin comes to this conclusion? I'm wondering if you folks have some rounding-up mathimatical calculation system inside of Virtualmin's core codebase? Joe or Jamie might be best to answer this , unless providing details such as the install.log that the minimal CentOS automaticly installs packages wise etc would help. I'm not sure how this is happening. Is their a way to tel Virtualmin to actually "re-calculate" all disk space on the entire drive? Maybe Virtualmin is looking at the default server Raid1 setup somehow as different? fdisk -l shows the entire drive capassity of the server is 1TB, so I don't see how in the world 40GB wich is an extremely large number of GB on a hosting drive can be taken away simply by Virtualmin/webmin. Likewise, if I don't extend the logical volume, and just leave it as is, Firtualmin then says it's using 2GB out of 15. And no, I can't turn off RAID1, as this is part of the servers automatic setup regardless of how an OS is installed. It will always use RAID so I'm wondering if any bug obscure or not might have somehow gone unnoticed? I even tried looking at yum grouplist and such, and most of the groups like desktop, etc weren't even installed. Any thoughts would be appreciated. Should I try another OS image? My server provider is 1and1.com, and amungst other things they offer Debain6, Ubuntu, etc. Though the Debian images use XFS. Wonder if the Virtualmin installer would use less disk space on Debian? What's really interesting is when I've installed Virtulamin on Debain, Postgresql will still run at boot even when told not to run by the configuration wizard. The only way to make it not start is to remove the symlinks from /etc/init.d via rc-update or whatever. Take care folks.
I'm updating this with as much detail as possible. At present, hear is what I can provide: System hostname server1.keithnet.us (74.208.145.86) Operating system CentOS Linux 6.2 Webmin version 1.585 Virtualmin version 3.91.gpl GPL Theme version 8.4 Time on server 07/May/2012 05:31 , Up 2 hours, 43 minutes Kernel and CPU Linux 2.6.32-220.13.1.el6.x86_64 on x86_64 CPU load averages 0.15 (1 min) 0.07 (5 mins) 0.02 (15 mins)
Running processes 168 Real memory 3.71 GB total, 607.51 MB used
Virtual memory 1.87 GB total, 0 bytes used
Local disk space 900.58 GB total, 48.39 GB used
Package updates All Virtualmin packages are up to date.
Out of all that, I want to know how is 48GB being used? If I show you folks the install.log from my servers CentOS automatic stuff, I get this: cat install.log Installing libgcc-4.4.4-13.el6.x86_64 warning: libgcc-4.4.4-13.el6.x86_64: Header V3 RSA/SHA256 Signature, key ID c105b9de: NOKEY Installing setup-2.8.14-10.el6.noarch Installing filesystem-2.4.30-2.1.el6.x86_64 Installing basesystem-10.0-4.el6.noarch Installing ca-certificates-2010.63-3.el6.noarch Installing ncurses-base-5.7-3.20090208.el6.x86_64 Installing tzdata-2010l-1.el6.noarch Installing glibc-common-2.12-1.7.el6.x86_64 Installing nss-softokn-freebl-3.12.7-1.1.el6.x86_64 Installing glibc-2.12-1.7.el6.x86_64 Installing ncurses-libs-5.7-3.20090208.el6.x86_64 Installing bash-4.1.2-3.el6.x86_64 Installing zlib-1.2.3-25.el6.x86_64 Installing info-4.13a-8.el6.x86_64 Installing audit-libs-2.0.4-1.el6.x86_64 Installing popt-1.13-7.el6.x86_64 Installing chkconfig-1.3.47-1.el6.x86_64 Installing libcom_err-1.41.12-3.el6.x86_64 Installing db4-4.7.25-16.el6.x86_64 Installing libsepol-2.0.41-3.el6.x86_64 Installing libselinux-2.0.94-2.el6.x86_64 Installing glib2-2.22.5-5.el6.x86_64 Installing gamin-0.1.10-9.el6.x86_64 Installing shadow-utils-4.1.4.2-8.el6.x86_64 Installing sed-4.2.1-5.el6.x86_64 Installing nspr-4.8.6-1.el6.x86_64 Installing bzip2-libs-1.0.5-6.1.el6.x86_64 Installing nss-util-3.12.7-1.el6.x86_64 Installing readline-6.0-3.el6.x86_64 Installing libattr-2.4.44-4.el6.x86_64 Installing libacl-2.2.49-4.el6.x86_64 Installing libcap-2.16-5.2.el6.x86_64 Installing gawk-3.1.7-6.el6.x86_64 Installing libudev-147-2.29.el6.x86_64 Installing dbus-libs-1.2.24-3.el6.x86_64 Installing lua-5.1.4-4.1.el6.x86_64 Installing sqlite-3.6.20-1.el6.x86_64 Installing cyrus-sasl-lib-2.1.23-8.el6.x86_64 Installing iptables-1.4.7-3.el6.x86_64 Installing libidn-1.18-2.el6.x86_64 Installing file-libs-5.04-5.el6.x86_64 Installing libuuid-2.17.2-6.el6.x86_64 Installing libblkid-2.17.2-6.el6.x86_64 Installing elfutils-libelf-0.148-1.el6.x86_64 Installing xz-libs-4.999.9-0.3.beta.20091007git.el6.x86_64 Installing libstdc++-4.4.4-13.el6.x86_64 Installing pcre-7.8-3.1.el6.x86_64 Installing grep-2.6.3-2.el6.x86_64 Installing libgpg-error-1.7-3.el6.x86_64 Installing bzip2-1.0.5-6.1.el6.x86_64 Installing findutils-4.4.2-6.el6.x86_64 Installing libselinux-utils-2.0.94-2.el6.x86_64 Installing checkpolicy-2.0.22-1.el6.x86_64 Installing cpio-2.10-9.el6.x86_64 Installing tcp_wrappers-libs-7.6-56.3.el6.x86_64 Installing libcap-ng-0.6.4-3.el6.x86_64 Installing sysvinit-tools-2.87-3.dsf.el6.x86_64 Installing expat-2.0.1-9.1.el6.x86_64 Installing pth-2.0.7-9.3.el6.x86_64 Installing dbus-glib-0.86-5.el6.x86_64 Installing libgcrypt-1.4.5-3.el6.x86_64 Installing less-436-4.el6.x86_64 Installing gmp-4.3.1-7.el6.x86_64 Installing libusb-0.1.12-23.el6.x86_64 Installing grubby-7.0.15-2.el6.x86_64 Installing iproute-2.6.32-10.el6.x86_64 Installing iptables-ipv6-1.4.7-3.el6.x86_64 Installing nss-softokn-3.12.7-1.1.el6.x86_64 Installing libnih-1.0.1-6.el6.x86_64 Installing upstart-0.6.5-6.1.el6.x86_64 Installing pinentry-0.7.6-5.el6.x86_64 Installing vim-minimal-7.2.411-1.4.el6.x86_64 Installing tar-1.23-3.el6.x86_64 Installing libutempter-1.1.5-4.1.el6.x86_64 Installing MAKEDEV-3.24-6.el6.x86_64 Installing psmisc-22.6-15.el6.x86_64 Installing net-tools-1.60-102.el6.x86_64 Installing db4-utils-4.7.25-16.el6.x86_64 Installing e2fsprogs-libs-1.41.12-3.el6.x86_64 Installing libss-1.41.12-3.el6.x86_64 Installing diffutils-2.8.1-28.el6.x86_64 Installing binutils-2.20.51.0.2-5.11.el6.x86_64 Installing m4-1.4.13-5.el6.x86_64 Installing which-2.19-5.1.el6.x86_64 Installing libxml2-2.7.6-1.el6.x86_64 Installing ncurses-5.7-3.20090208.el6.x86_64 Installing coreutils-libs-8.4-9.el6.x86_64 Installing gzip-1.3.12-18.el6.x86_64 Installing cracklib-2.8.16-2.el6.x86_64 Installing cracklib-dicts-2.8.16-2.el6.x86_64 Installing coreutils-8.4-9.el6.x86_64 Installing pam-1.1.1-4.el6.x86_64 Installing module-init-tools-3.9-17.el6.x86_64 Installing logrotate-3.7.8-12.el6.x86_64 Installing nss-3.12.7-2.el6.x86_64 Installing nss-sysinit-3.12.7-2.el6.x86_64 Installing procps-3.2.8-14.el6.x86_64 Installing pciutils-libs-3.1.4-9.el6.x86_64 Installing ethtool-2.6.33-0.3.el6.x86_64 Installing mingetty-1.08-4.1.el6.x86_64 Installing keyutils-libs-1.4-1.el6.x86_64 Installing krb5-libs-1.8.2-3.el6.x86_64 Installing openssl-1.0.0-4.el6.x86_64 Installing openldap-2.4.19-15.el6.x86_64 Installing libssh2-1.2.2-7.el6.x86_64 Installing libcurl-7.19.7-16.el6.x86_64 Installing curl-7.19.7-16.el6.x86_64 Installing rpm-libs-4.8.0-12.el6.x86_64 Installing rpm-4.8.0-12.el6.x86_64 Installing gnupg2-2.0.14-4.el6.x86_64 Installing gpgme-1.1.8-3.el6.x86_64 Installing libuser-0.56.13-4.el6.x86_64 Installing mysql-libs-5.1.47-4.el6.x86_64 Installing fipscheck-lib-1.2.0-4.1.el6.x86_64 Installing fipscheck-1.2.0-4.1.el6.x86_64 Installing ustr-1.0.4-9.1.el6.x86_64 Installing libsemanage-2.0.43-4.el6.x86_64 Installing gdbm-1.8.0-36.el6.x86_64 Installing libffi-3.0.5-3.2.el6.x86_64 Installing python-libs-2.6.5-3.el6.x86_64 Installing python-2.6.5-3.el6.x86_64 Installing yum-metadata-parser-1.1.2-14.1.el6.x86_64 Installing pygpgme-0.1-18.20090824bzr68.el6.x86_64 Installing rpm-python-4.8.0-12.el6.x86_64 Installing python-pycurl-7.19.0-5.el6.x86_64 Installing dash-0.5.5.1-3.1.el6.x86_64 Installing slang-2.2.1-1.el6.x86_64 Installing newt-0.52.11-2.el6.x86_64 Installing newt-python-0.52.11-2.el6.x86_64 Installing plymouth-core-libs-0.8.3-17.el6.centos.x86_64 Installing redhat-logos-60.0.14-10.el6.noarch Installing plymouth-scripts-0.8.3-17.el6.centos.x86_64 Installing python-urlgrabber-3.9.1-7.el6.noarch Installing python-iniparse-0.3.1-2.1.el6.noarch Installing yum-plugin-fastestmirror-1.1.26-11.el6.noarch Installing yum-3.2.27-14.el6.centos.noarch Installing hwdata-0.233-1.el6.noarch Installing kbd-misc-1.15-11.el6.noarch Installing centos-release-6-0.el6.centos.5.x86_64 Installing iputils-20071127-13.el6.x86_64 Installing util-linux-ng-2.17.2-6.el6.x86_64 Installing initscripts-9.03.17-1.el6.centos.x86_64 Installing udev-147-2.29.el6.x86_64 Installing device-mapper-libs-1.02.53-8.el6.x86_64 Installing device-mapper-1.02.53-8.el6.x86_64 Installing device-mapper-event-libs-1.02.53-8.el6.x86_64 Installing device-mapper-event-1.02.53-8.el6.x86_64 Installing lvm2-libs-2.02.72-8.el6.x86_64 Installing kbd-1.15-11.el6.x86_64 Installing rsyslog-4.6.2-2.el6.x86_64 Installing openssh-5.3p1-20.el6.x86_64 Installing libcgroup-0.36.1-6.el6.x86_64 Installing policycoreutils-2.0.83-19.1.el6.x86_64 Installing cyrus-sasl-2.1.23-8.el6.x86_64 Installing postfix-2.6.6-2.el6.x86_64 Installing cronie-anacron-1.4.4-2.el6.x86_64 Installing cronie-1.4.4-2.el6.x86_64 Installing crontabs-1.10-32.1.el6.noarch Installing selinux-policy-3.7.19-54.el6.noarch Installing kernel-firmware-2.6.32-71.el6.noarch Installing openssh-server-5.3p1-20.el6.x86_64 Installing lvm2-2.02.72-8.el6.x86_64 Installing dhclient-4.1.1-12.P1.el6.x86_64 Installing authconfig-6.1.4-6.el6.x86_64 Installing passwd-0.77-4.el6.x86_64 Installing sudo-1.7.2p2-9.el6.x86_64 Installing efibootmgr-0.5.4-8.el6.x86_64 Installing audit-2.0.4-1.el6.x86_64 Installing e2fsprogs-1.41.12-3.el6.x86_64 Installing acl-2.2.49-4.el6.x86_64 Installing attr-2.4.44-4.el6.x86_64 Installing libdrm-2.4.20-2.el6.x86_64 Installing plymouth-0.8.3-17.el6.centos.x86_64 Installing dracut-004-32.el6.noarch Installing dracut-kernel-004-32.el6.noarch Installing kernel-2.6.32-71.el6.x86_64 Installing selinux-policy-targeted-3.7.19-54.el6.noarch Installing bfa-firmware-2.1.2.1-2.el6.noarch Installing iwl5150-firmware-8.24.2.2-1.el6.noarch Installing iwl6050-firmware-9.201.4.1-2.el6.noarch Installing b43-openfwwf-5.2-4.el6.noarch Installing iwl6000-firmware-9.176.4.1-2.el6.noarch Installing aic94xx-firmware-30-2.el6.noarch Installing grub-0.97-68.el6.x86_64 Installing system-config-firewall-base-1.2.27-1.el6.noarch Installing ql2400-firmware-5.03.02-1.el6.noarch Installing iwl5000-firmware-8.24.2.12-3.el6.noarch Installing libertas-usb8388-firmware-5.110.22.p23-3.1.el6.noarch Installing rootfiles-8.1-6.1.el6.noarch Installing xorg-x11-drv-ati-firmware-6.13.0-6.el6.noarch Installing atmel-firmware-1.3-7.el6.noarch Installing iwl4965-firmware-228.61.2.24-2.1.el6.noarch Installing iwl1000-firmware-128.50.3.1-1.1.el6.noarch Installing iwl3945-firmware-15.32.2.9-4.el6.noarch Installing ql2200-firmware-2.02.08-3.1.el6.noarch Installing rt61pci-firmware-1.2-7.el6.noarch Installing rt73usb-firmware-1.8-7.el6.noarch Installing ql2100-firmware-1.19.38-3.1.el6.noarch Installing ql2500-firmware-5.03.02-2.el6.noarch Installing zd1211-firmware-1.4-4.el6.noarch Installing ipw2100-firmware-1.3-11.el6.noarch Installing ql23xx-firmware-3.03.27-3.1.el6.noarch Installing ipw2200-firmware-3.1-4.el6.noarch Installing ivtv-firmware-20080701-20.2.noarch *** FINISHED INSTALLING PACKAGES ***[root@server1 ~]# That output is from my servers console directly, not an SSH user. I am listing that stuff in case it may help. I don't know if all those packages are taking up space (as I hoave not a single virtual host set up at all, etc. Also, I wonder why my RAM seems to be on the rise? I don't really feel llike putting an I686 version of CentOS on the box if I don't have to. Also if I do that I'll need to do whatever to reduce disk space. This isn't a VPS template of CentOS though, so I probably have tons of extra junk that I do not need at all. If we run just for the heck of it an lsmod: lsmod Module Size Used by ipt_REJECT 2383 2 nf_conntrack_ipv4 9506 2 nf_defrag_ipv4 1483 1 nf_conntrack_ipv4 iptable_filter 2793 1 ip_tables 17831 1 iptable_filter ip6t_REJECT 4628 2 nf_conntrack_ipv6 8748 2 nf_defrag_ipv6 12182 1 nf_conntrack_ipv6 xt_state 1492 4 nf_conntrack 79453 3 nf_conntrack_ipv4,nf_conntrack_ipv6,xt_state ip6table_filter 2889 1 ip6_tables 19458 1 ip6table_filter ipv6 322029 41 ip6t_REJECT,nf_conntrack_ipv6,nf_defrag_ipv6 ext4 364410 3 jbd2 88834 1 ext4 ppdev 8729 0 parport_pc 22978 0 parport 37265 2 ppdev,parport_pc sg 30124 0 tg3 140883 0 serio_raw 4818 0 k10temp 3513 0 edac_core 46773 0 edac_mce_amd 15488 0 shpchp 33482 0 i2c_nforce2 6904 0 ext3 235341 1 jbd 80433 1 ext3 mbcache 8144 2 ext4,ext3 sd_mod 39520 4 crc_t10dif 1541 1 sd_mod arcmsr 30158 3 pata_acpi 3701 0 ata_generic 3837 0 ahci 40455 0 nouveau 708412 1 ttm 70328 1 nouveau drm_kms_helper 33236 1 nouveau drm 230675 3 nouveau,ttm,drm_kms_helper i2c_algo_bit 5762 1 nouveau i2c_core 31276 5 i2c_nforce2,nouveau,drm_kms_helper,drm,i2c_algo_bit mxm_wmi 1925 1 nouveau wmi 6287 1 mxm_wmi video 21032 1 nouveau output 2505 1 video dm_mirror 14101 0 dm_region_hash 12170 1 dm_mirror dm_log 10122 2 dm_mirror,dm_region_hash dm_mod 81692 11 dm_mirror,dm_log [root@server1 ~]# What else can I show you folks... If you folks have any thing else that may even prove remotely useful in tracking down why so much space is being used (as I wish this amount to be drasticly reduced without erasing the mbr in the process), please let me know.
Let me know if you want anymore details.
Well, I'm sorry if I didn't read through your whole lengthy posts, which are very hard to read, being about five screenfuls of text without any paragraph or organization. :)
I can say though that Virtualmin most definitely does NOT allocate 40 GB of disk space. A fresh installation of VM on in my case Ubuntu uses about 2 GB.
To find out more about your issue, I'd like to ask for some more structured and concise information about your setup. You can use
dfto find out disk allocation per mount point, anddu -sh *to summarize allocation per subdirectory. Where exactly do you see that Virtualmin uses 40 GB?Sorry about the organization lacking, apparently hitting my enter key once or twice didn't do any paragraphs.
Suggestions welcome. :)
Now, hear is what you wanted:
df /
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda1 3850292 735712 3114580 20% /
The /usr and /home partitions:
df /usr
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/vg00-usr 4062912 948244 2904956 25% /usr
df /home
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/vg00-home
932354300 146160 884094916 1% /home
The swap partition is only 1GB in size, so I won't bother listing that. As for the rest:
According to du -sh * on a per-directory setup, the / partition uses 1.6GB.
/home uses: 36k? interesting.
I did use LVM to expand the /home logical volume to use up all remaining free space and resized the ext4 FS on it, but I still don't see how Virtualmin thinks about 48GB is in use.
According to du -sh /usr that's about 900M and /var is about 100M and only 9 percent in use.
Thanks!
The allocation figures you get from df and du look all okay for a regular CentOS.
So I need to ask again, where exactly do you see that Virtualmin says "40 GB used"?
Specificly, I'm seeing the large amount of GB being used in the system information section:
System hostname server1.keithnet.us (74.208.145.86) Operating system CentOS Linux 6.2
Webmin version 1.585 Virtualmin version 3.91.gpl GPL
Theme version 8.4 Time on server 07/May/2012 08:54 , Up 0 hours, 55 minutes
Kernel and CPU Linux 2.6.32-220.13.1.el6.x86_64 on x86_64 CPU load averages 0.00 (1 min) 0.00 (5 mins) 0.00 (15 mins)
Running processes 163 Real memory 3.71 GB total, 595.14 MB used
Virtual memory 1.87 GB total, 0 bytes used
Local disk space 900.58 GB total, 48.33 GB used
Package updates All Virtualmin packages are up to date.
So now that you know exactly where I'm seeing this, I'm wondering how Virtualmin is deriving that much?
Hmm, let's start here -- can you run this command, and let us know what output you're seeing:
df -h
That will show all your partitions at once, and the 'h' will help make the output a little quicker to parse.
Thanks!
-Eric
Yep, that, and put the output in
[code][/code] tags please. :)Hi, folks:
Hear's the output you requested with the proper tags.
Thanks for getting back to me on this.
[code] df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 3.7G 573M 3.2G 16% /
/dev/mapper/vg00-usr 3.9G 959M 2.8G 26% /usr
/dev/mapper/vg00-var 3.9G 301M 3.4G 8% /var
/dev/mapper/vg00-home
890G 143M 844G 1% /home
none 2.0G 0 2.0G 0% /tmp
[/code]
I hope this helps!
Oops: forgot to put the [/code] tag, but hopefully, you still saw the code. Just ignore my typo on the ending tag.
Thanks again-will check back soon.
Okay, so that appears to be showing your system using less than 2GB of space in all.
Which makes that Virtualmin output you saw appear a bit bug-like :-)
I'll ask Jamie to take a peek into this issue. Thanks!
-Eric
Thanks, Eric.
Okay, I learned something new today!
Jamie explained that Webmin and Virtualmin include space reserved for the root user as used disk space.
Notice how when you run "df -h", it shows the partition as being 890GB, but only 844GB being available.
However, only 143MB is "used".
That's because 46GB is reserved for root and not available for general use.
So Webmin and Virtualmin are each showing that space as being used.
-Eric
Thanks Eric!
Now I understand, you know..I totally forgot about reserved disk space!
However, while this isn't related, would you mind passing this along to Jamie for me?
It appears to be a small bug issue or osmething, with the very latest virtualmin release.
Would you mind trying this on your own system assuming you have ProFTP installed?
Please try shutting it down and starting it back up again.
In both cases you should see the following output prior to the script actually doing what it should.
[code] /etc/rc0.d/K30proftpd: line 44: [: =: unary operator expected
Shutting down proftpd: [ OK ]
[/code]
You should see that happening-at least I am, and no I didn't modify anything special on my CentOS box.
I don't know why it's occuring, is this a bug?
Or can this easily be fixed with an easy workaround solution?
Thanks!
That FTP issue doesn't occur on my system... it's also not likely to be a Virtualmin related issue -- that looks like either an issue with your particular ProFTP package, the related config, or maybe an issue with the image if you're using an ISP-provided image.
If you want to discuss that, open up a new forum thread, and tell us about your system, whether you're using any third party repositories, and whether you're using an OS image provided by an ISP, or whether it's an install you performed. Thanks!
-Eric
I ran into the same issue wondering why Virtualmin said that 150GB was used while df showed only 20GB used.
Request:
or
That would help a lot for the less experienced virtualmin users like me.