After installing suggested update, Virtualmin 3.99 update failed, I think because of broken webmin. At upgrade I noticed that Debian moved their Lenny repositories to archive, as Lenny is EOL. After fixing repositories apt-get update finished fine. But apt-get upgrade (with webmin-virtual-server as only package) failed with error that "Webmin does not appear to be installed on your system.". And installing Webmin with apt-get says that its already installed.
Earlier at initial upgrade, this the error that Webmin generated:
Preparing to replace webmin 1.620 (using .../archives/webmin_1.630_all.deb) ...
Unpacking replacement webmin ...
dpkg: error processing /var/cache/apt/archives/webmin_1.630_all.deb (--unpack):
unable to create `./usr/share/webmin/mysql/help/table_form.nl.html': No space left on device
dpkg-deb: subprocess paste killed by signal (Broken pipe)
grep: /etc/webmin/miniserv.conf: No such file or directory
Webmin install complete. You can now login to http://ns1:/
as root with your root password, or as any user who can use sudo
to run commands as root.
Errors were encountered while processing:
/var/cache/apt/archives/webmin_1.630_all.deb
Also this is the error of Virtualmin:
(Reading database ... 88720 files and directories currently installed.)
Preparing to replace usermin 1.540 (using .../archives/usermin_1.550_all.deb) ...
Unpacking replacement usermin ...
Preparing to replace webmin 1.620 (using .../archives/webmin_1.630_all.deb) ...
Unpacking replacement webmin ...
Preparing to replace webmin-virtual-server 3.99.gpl (using .../webmin-virtual-server_4.00.gpl_all.deb) ...
Webmin does not appear to be installed on your system.
This package cannot be installed unless the Debian version of Webmin
is installed first.
dpkg: error processing /var/cache/apt/archives/webmin-virtual-server_4.00.gpl_all.deb (--unpack):
subprocess pre-installation script returned error exit status 1
grep: /etc/webmin/config: No such file or directory
grep: /etc/webmin/config: No such file or directory
usage: copyconfig.pl <os>[/real-os] <version>[/real-version] <webmin-dir> <config-dir> [module ...] at /usr/share/webmin/copyconfig.pl line 7.
grep: /etc/webmin/webmin.acl: No such file or directory
head: cannot open `/etc/webmin/webmin.acl' for reading: No such file or directory
grep: /etc/webmin/webmin.acl: No such file or directory
grep: /etc/webmin/webmin.acl: No such file or directory
Error: Script was not run with full path (failed to find /usr/share/webmin/run-postinstalls.pl under )
-----
Script was not run with full path (failed to find /usr/share/webmin/run-postinstalls.pl under )
-----
Errors were encountered while processing:
/var/cache/apt/archives/webmin-virtual-server_4.00.gpl_all.deb
I did not want to try to remove/install Webmin, because I don't want to make more trouble. Now my Virtualmin does not work and I also client receive an email with error, but email actually works:
Command died with status 127: "/usr/bin/procmail-wrapper -o -a $DOMAIN -d $LOGNAME'
And also error in procmail log:
From MAILER-DAEMON Wed Jun 12 11:25:02 2013
Subject: Undelivered Mail Returned to Sender
Folder: /root/Maildir/new/1371029102.18138_0.ns1 2685
/bin/sh: /etc/webmin/virtual-server/procmail-logger.pl: No such file or directory
Could you help me how to fix this ?
Regards, Petar
Comments
Submitted by andreychek on Wed, 06/12/2013 - 07:34 Comment #1
Howdy -- it looks like the initial issue is that your hard drive was out of space. I see this error while upgrading Webmin:
No space left on deviceThat may have triggered a problem with the Webmin package that is causing the additional issues that you are seeing.
The first question is -- do you have backups of your Virtual Servers and perhaps your filesystem?
Second, what output does this command produce:
ls /etc/webmin/Submitted by petar.j on Wed, 06/12/2013 - 08:24 Comment #2
I was thinking the same, but obviously there is enough space, unless its needed more than 600 MB on root.
Filesystem Size Used Avail Use% Mounted on/dev/xvda2 1.9G 1.2G 592M 67% /
tmpfs 641M 0 641M 0% /lib/init/rw
udev 10M 532K 9.5M 6% /dev
tmpfs 641M 0 641M 0% /dev/shm
/dev/xvda1 183M 43M 131M 25% /boot
/dev/xvdc1 9.9G 5.4G 4.5G 55% /home
/dev/xvdb1 3.0G 634M 2.3G 22% /var
/dev/xvdf1 20G 16G 3.7G 81% /backup
Here is the output of /etc/webmin/:
uninstall.sh webmin.aclI have a backup of Virtualmin servers few days ago, when upgrade failed. Also there is Virtualmin configuration backup and file system backup week or two ago
Submitted by andreychek on Wed, 06/12/2013 - 08:33 Comment #3
Hmm, what does "df -i" show?
And is this a dedicated server, or VPS? If a VPS, what kind of VPS is it -- Xen, OpenVZ, KVM?
Submitted by petar.j on Wed, 06/12/2013 - 08:54 Comment #4
df -iFilesystem Inodes IUsed IFree IUse% Mounted on
/dev/xvda2 122160 99032 23128 82% /
tmpfs 164006 3 164003 1% /lib/init/rw
udev 164006 981 163025 1% /dev
tmpfs 164006 1 164005 1% /dev/shm
/dev/xvda1 48192 40 48152 1% /boot
/dev/xvdc1 655360 222785 432575 34% /home
/dev/xvdb1 196608 9325 187283 5% /var
/dev/xvdf1 1310720 2347 1308373 1% /backup
It's dedicated server with Citrix XenServer and beside this Virtualmin, there is also some Windows machine.
Submitted by andreychek on Wed, 06/12/2013 - 09:14 Comment #5
Hrm, it's tough to say why you were getting that error, as it looks like there's plenty of space in there now. It's possible something else was going on at the time which caused space, or perhaps inodes, to temporarily be used up.
However, it appears that the key at the moment is going to be getting everything else back online.
The first step I'd take is to reinstall Webmin, and see if that works properly.
You can do that with this command:
apt-get install --reinstall webminDoes that install cleanly, without any errors?
If so, the next step would be to do the same for Virtualmin and related modules.
You can see all the Webmin/Virtualmin packages that need reinstalled by running this command:
dpkg -l 'webmin*' | grep iiAfter doing that -- are you then able to log into Virtualmin on port 10000? (note that you likely won't see your Virtual Servers yet, that'll be the next step)
Submitted by petar.j on Wed, 06/12/2013 - 09:46 Comment #6
Reinstalling Webmin was successful and and I able to login with Webmin. Should I reinstall with same command every module, or maybe is enough to just upgrade with apt-get, because webmin-virtual-server is the only package that needs to be upgraded.
Also is it safe to reinstall the modules, because I need to keep all virtualmin configurations.
Submitted by andreychek on Wed, 06/12/2013 - 09:54 Comment #7
Unfortunately, all of the data from /etc/webmin is gone, which is where all those modules store their config information.
In order to begin the process of getting it all back, you'll need to upgrade and/or reinstall all of those packages, which will regenerate a lot of the config data.
So if you can upgrade some of the packages, that's great, but those that you can't upgrade you'd need to reinstall them.
Submitted by petar.j on Wed, 06/12/2013 - 11:22 Comment #8
I upgraded webmin-virtual-server and reinstalled all other modules. I can access now Virtualmin, but as you mentioned all configuration is gone. Can I just restore Virtualmin Settings previoulsy created with "Backup and Restore > Scheduled Backups > Features and settings > Virtualmin settings to also backup", to restore only configuration and not touching any of system services and data (MySQL, Apache Websites, Mailboxes ...). If so, is the option "Just re-import server details?" in "Other Restore options" is the thing that I need. I also have backup of Webmin configurations.
Submitted by andreychek on Wed, 06/12/2013 - 11:51 Comment #9
There's a few different ways to move forward from here, we'll work with you to find the simplest one.
When you say you have a backup of your Webmin configuration -- what exactly does that backup consist of?
Submitted by petar.j on Wed, 06/12/2013 - 12:42 Comment #10
I'm not sure, but I think that there are created with Webmin > Webmin > Backup Configuration Files and all the modules selected from "Modules to backup".
ns1:/# ls /backup/temp/boot etc tmp var
ns1:/# ls /backup/temp/etc/apache2 default group inetd.conf localtime mailname nsswitch.conf postgresql rpc ssh webalizer
bind dovecot gshadow init.d logrotate.conf mime.types pam.d procmailrc rsyslog.conf sysctl.conf webmin
cron.d fstab hostname inittab logrotate.d mysql passwd proftpd services timezone
crontab ftpusers hosts iptables.up.rules mailcap network postfix resolv.conf shadow usermin
content of /etc/webmin (excluded *.acl files)
ns1:/# ls -C /backup/temp/etc/webmin/ | grep -v ".acl"at inittab postfix telnet
config ldap-server proftpd usermin
cron logrotate qmailadmin vgetty
fetchmail mon sentry webalizer
file mount servers webmin
grub sshd
And also Virtualmin configurations:
ns1:/# ls /backup/temp/virtualmin/virtualmin_config virtualmin_custom_shells virtualmin_mailserver_dkimkey virtualmin_mailserver_maincf virtualmin_templates
virtualmin_custom virtualmin_email virtualmin_mailserver_grey virtualmin_mailserver_mastercf virtualmin_templates_global
virtualmin_custom_linkcats virtualmin_mailserver virtualmin_mailserver_greyclients virtualmin_scheds virtualmin_templates_plans
virtualmin_custom_links virtualmin_mailserver_dkim virtualmin_mailserver_greyrecipients virtualmin_scripts virtualmin_templates_skel_0
I think this are all the necessary configuration.
Submitted by andreychek on Wed, 06/12/2013 - 14:00 Comment #11
Thanks! I've asked Jamie for some input on a couple of issues; once we get those issues sorted out, I'll respond here and we'll finish getting your system up and running again.
In the meantime, if your email is bouncing -- you can always temporarily stop Postfix. Mail servers will retry the mail delivery for some time, usually every 20 minutes for a week. And we'll have your system fixed up long before then :-)
Submitted by andreychek on Wed, 06/12/2013 - 14:16 Comment #12
Okay, so all of your websites are working, what you're missing is the Virtualmin metadata from /etc/webmin.
What we need to do is extract that from your Virtualmin backups, without making changes to the user data.
Jamie's suggestion of the best way to do that is to use this command:
virtualmin restore-domain --source /path/to/backups/ --all-domains --feature virtualmin --fixThat will restore the metadata from all the backups files located in "/path/to/backups/".
Submitted by petar.j on Wed, 06/12/2013 - 17:20 Comment #13
After this command I can see domains in Virtualmin, but some configurations are missing. Where should I start from:
MySQL does not appear to be installed and running on your system, or has not yet been set up properly in Webmin's MySQL Database module. If your system does not use MySQL, it should be disabled in Virtualmin's module configuration page.
During command execution there was and error:.. your system is not ready for use by Virtualmin.
/usr/bin/mysqlshow: Access denied for user 'root'@'localhost' (using password: NO)Webalizer reporting : Webalizer configuration file /etc/webmin/webalizer/home_domain_logs_access_log.conf was not foundMySQL database : MySQL database domain does not exist
Webmin login : Webmin user domain does not exist
Submitted by andreychek on Wed, 06/12/2013 - 17:54 Comment #14
The command you ran re-populated the metadata for your various Virtual Servers... that will enable us to perform these next steps to finish fixing things up.
The next step -- do you have a backup file named virtualmin.tar.gz?
If so, go ahead and perform a restore of that file... and then let us know when that's finished.
Submitted by petar.j on Wed, 06/12/2013 - 18:22 Comment #15
Just restored the virtualmin.tar.gz, re-checked the configuration and it passed. Also I put the password for MySQL modul, and I can see the databases from Virtualmin.
Submitted by andreychek on Wed, 06/12/2013 - 19:20 Comment #16
Great! We're almost finished.
Did restoring virtualmin.tar.gz also restore your Webmin/admin users for your various Virtual Servers? Or are those still missing?
Anything else that's missing, we'll need to restore from the Virtualmin backups... going into Backup and Restore -> Restore Backups, pointing it at the backup directory, and restoring the features that are missing.
You won't need to restore databases or home directory data -- but you may need to restore "Virtual server password, description and other details", "Administration user's password, quota and Cron jobs", and "Webmin ACL files".
Feel free to let us know what's missing, and we can help with which features you need to restore.
Submitted by petar.j on Fri, 06/14/2013 - 14:01 Comment #17
Everything is restored properly, thank a lot for the help :) I also restored configurations of "Virtualmin Virtual Servers", Webmin Configuration" and "Webmin users" from Webmin backup. Domain validation was showing some Webalizer errors on few domains, but after re-enabling the option, all domain validation completed successfully.
There is one small thing that I need to restore: Webmin > Filesystem Backup Schedules. There were few customized schedules, then I can always create again, but honestly, I'm little lazy after this :)
Thanks again for the help :)
Submitted by andreychek on Fri, 06/14/2013 - 14:08 Comment #18
Super, I'm glad all that is working!
We're working on a way to prevent this all from occurring; that should never happen simply because you're low on disk space.
We'll have a fix for that available in the next couple of days... but in the meantime, just to be safe, you may want to make a backup of your /etc/webmin directory. Then, in the off-chance that it occurs again during an update, you can simply replace that directory.