Webmin hacked

Update to Webmin 1.800. There's a security bug in Authentic Theme as shipped with Webmin devel versions 1.794 and 1.794 (which we rolled out to Virtualmin users to address Ubuntu fixes and Let's Encrypt enhancements).

It's been fixed in 1.800. We learned of the bug about two hours ago. We'll talk about specifics of the problem in a few days after folks have had a chance to update. You can open a private ticket in the issue tracker to discuss it in more detail, if you'd like us to assist with assessing the scope of the problem on your system.

I have seen the same thing on a few servers 3 days ago.

They seem to be using a shell exploit in the [edit: we'll discuss the exact security issue later :-)], at first I though it was a shellshock hack (but of course those have been patched for years). It looked like it was escaping out some sort of login alert mail out routine.

If you reboot the server the bad processes will go away, and you can delete the junk files left behind in the webmin directory.

I have not found any permanent changes to the system files, so once the theme is updated the issue should be gone for good, however this exploit could have been very destructive since it runs as root.

Got the malware in one of our servers. It runs after reboot and takes all server CPU. Thats how we spotted it.

I am very disappointed of Webmin team. Not because of getting our server hacked, but because it could be avoided easily. We got hacked almost 48h after the 1.800 patch went public, however the release did not get to the official Webmin mailing list, so I was unaware of the patch.

Another bad luck is that Centos 6.8 upgrade come in the same time, so spotting webmin between 135 other packages manually is not simple task.

On webmin website, all the forums and mailing list go through source forge. Are they update? Why are there forums there and here.... Please centralize the information.

Please advise me how can I immediately know that there is a webmin/virtualmin update have their changelog.

Cursor, can you please share your experience with the hack. What was the exact impact?

I have managed to save lots of its code. Its combination of PHP/Pbot.D troyan, Linux/Tsunami IRC bot and Kaitan IRC DDOS client.

It seems it tried to brute force the root password even when the .X0-unix process was running root

my webmin too was hacked. so i will not buy virtualmin i just only has one server.

Hey guys,

Sorry for all the trouble, I assure you all of this is the last thing any of us want.

For those of you asking how you might learn about things like this quicker --

We do post about things like this on Twitter, that might be a good way to monitor for updates:

https://twitter.com/virtualmin

Also, you could configure Virtualmin to send notifications about available updates. Now, it can be difficult to understand the severity of an issue from that, but it would be an opportunity to know that updates are waiting and available.

Another thing you could do is subscribe to the News Forum here, which would provide you with email updates for Webmin and Virtualmin related news:

https://www.virtualmin.com/forum/274

These are the files that have been uploaded/compiled to webmin directory

These are the one in /tmp

mg seems to be the first file of the hack that downloads, compiles and runs the rest of the code.

The hack seems to be at least 3 different IRC controlled DDOS bots: Linux/Tsunami, Kaitan and PHP/PBot. The control server is irc.ugotownedz.org and he files were downloaded from ugotownedz.org.

It seems have attempted to do local password brute force.

Read here how to check your system: https://github.com/qooob/authentic-theme/issues/471

I have the sources of all of them if you need them.

Hiya,

Got hit also and this is bad and unacceptable.

Look in /etc/rc.local also because a ssh scanner was installed on my machine "/usr/share/webmin/.lssh"

Francesca

Some tips for others since none of my machines where affected.
1. I was not using Authentic theme. (never liked the slowness, virtmin mobile is way better on mobile) 2, Changed port to something other than 10000. (any internet facing service should never be on the default port) 3, Only accept ssh from VPN ip or a specific IP, or at least don't leave ssh on port 22. 4. Ossec bans for failed password attempts, for ssh, mail, http, etc. (install some type of active response monitor) 5 Ossec monitors /etc and other folders for changes.

2 and 3 will prevent most attacks. 4 and 5 will prevent the others if they get that far, or at least notify you so you can act. This goes for any web hosting platform, not just webmin.

In addition to https://www.virtualmin.com/comment/757529#comment-757529, I also had these files

I also had this in root crontab

Hi, here is what I would suggest..

1. deploy fail2ban at least for ssh and setup ban for day or month as a minimum.. or close port 22 to outside world
2. disable ssh logon with passwords and usernames globally and enable ssh keys logon only
3. close port 10000 to accessible from outside - connect to it via vpn or within your network - if that is not possible, you can setup ip access for webmin login page and change default port for webmin (that is not that helpful but it does slow attacks down)
4. limit how many times user can fail to logon to webmin and then ban it for day or so (the ip address or user or both)
5. check for any unknown user accounts on your system
6. change passwords for everyone - as well for root account too
7. check access logs - how they did came in
8. check ssh config (if any) for any amomalies
9. check start up scripts and remove anything suspicious
10. check cron jobs for any user - including for root and remove anything suspicious
11. check any unknown ssh keys in your authorized_keys file(s) - remove anything that its not yours perhaps best options would be to generate new file with just your keys..
12. disable ftp - if you run any, just use sftp which is ssh - if you must run it make sure you do deploy fail2ban (at least or something similar), fail 3 times logon = ban for day, month or year..
13. deploy notification script for ssh logins !! - on all srvs I've ever touched, I've deployed my own script which will send me google hangout message (aim - anything like icq, google hangouts, irc or xmpp) with ip address, username with time of logon at the very moment that user (any user using ssh) is logged in. It also sends those details to my client (owner of the srv) to email as well so we both are informed asap. This will allow you to see things in real time like who just connected from where..really great idea is this and I would greatly suggest to you to use this as well. Writing script like it is very easy! So if anything is happening on srv, be it attack or regular usage - you will know in advance as by this way ssh will not connect anyone before the quiet (discreet) notification is not send out to you (be it aim or email msg or both)..

....that should be enough to stop new hacks and you can continue to investigate from there.. I strongly believe that you will be just fine with those points mentioned above perhaps its overkill, but from personal experience - this is what always worked at lest for me past 7 years..

This apply to virtualmin too, however if you will detect new hack it would be the time to revise your website scripts in a bit different manner.. nothing hard to accomplish..

Have good day all.

But just moving from default port will cut down more than 90% of all bruteforce bots so i think is worth to change default port to something else (1024 < X).

@coderinthebox exactly..you are very right there - as I mentioned earlier, ssh keys will reduce this to 100% within fail2ban properly deployed even more (it will save some bandwidth you know).. changing ports is just not enough perhaps (speaking for me only personally) waste of time.

Thanks - all very good - i like the script/hangouts/login idea... disabling the root login is another thing i use, then if anyone tries a root login - they are banned instantly.

I used to use fail2ban - and still have it set up as a fall-back - but now i use OSSEC which does more of the same thing, but as a network tool - including a tripwire type scan - login notifications etc- means if someone does something annoying on one machine - i can block that ip/user on all machines i have responsibility for.

Changing port is not waste of time but should never be used as only prevention to hacking attempts. Its a fact that majority of hacking/scanning bots are hitting default ports and to use non-standard ports will eliminate most of them. Using keys instead of passwords, closing unused ports, fail2ban or other security software, all this will help to increase your overall security so moving to non-default ports is just one small piece of the bigger picture.

Only reason to leave default ports is in cases like shared hosting, some technical requirement or client for whatever reason want to use them.

both my servers have been hit too my real server and my home/backup server. not sure if it is the same issue or not,

Webmin version 1.801 Virtualmin version 5.04 Operating system CentOS Linux 6.8

I run chkrootkit and see 'suckit' infected.: Searching for Suckit rootkit... Warning: /sbin/init INFECTED

I see that I had a (hacked) script running on server /etc/webmin/status/monitor.pl and it produces files in /tmp/.webmin

d---------     2 root root  4096 Aug  7 11:25 204159_2211_2_status.pl
d---------     2 root root  4096 Aug  7 11:25 24501_2089_2_monitor.pl
d---------     2 root root  4096 Aug  7 11:10 248937_25009_2_status.pl
d---------     2 root root  4096 Aug  7 11:45 289317_6865_2_status.pl
d---------     2 root root  4096 Aug  7 11:15 333563_32736_2_status.pl
d---------     2 root root  4096 Aug  7 11:15 371546_32619_2_monitor.pl
d---------     2 root root  4096 Aug  7 11:30 469862_3129_2_monitor.pl
d---------     2 root root  4096 Aug  7 11:20 474562_1179_2_monitor.pl 

Selinux has caused major problems too, still trying to sort that out

I rebooted my home server and now unable to boot up it due to kernel panic. I can cet access through terminal but only in limited shell mode. tried USB live distro but still cannot get in.

I also get rm command not found. means I can't delete any of the hackers files. so now I have a script changing permissions to 000 that stops the files getting accessed.

how were they able to hack into the /sbin/init file ? seems they used Selinux security which caused home server to boot up with a kernel panic - not syncing : Attempted to kill init!

I also found a .scan folder and a 'scan' user

does look like I'll have to setup both servers from scratch now.

joe thanks for feedback.

it happened at least 7-10 days or more ago. I probably didn't notice due to working heavily on a project for the last month