I had several distinct installations of Virtualmin 6.07 installed on a minimal fresh Debian 9 (cloud servers at hetzner.de). On non of them fail2ban worked as supposed. For example it claimed to have found and banned IP's on postfix-sasl, but in fact the ban did not work so the same IP's continued happily trying to login to the smtp server day and night. (I did not check jails for every service though, maybe others do work.)
I tried everything suggested here in the forums to get it working, but to no success. Something must be broken in Virtualmin configurations for Debian. Probably because FirewallD is used instead of Iptables?
What I finally did was set up all my servers new on CentOS 7 because there FirewallD is default, widespread and working. Not like Debian where FirewallD is not popular at all. Now on CentOS everything is working as expected, just out of the box.
I consider this a serious security issue. Wonder how many Debian+Virtualmin servers out there are also unprotected exposed to bot attacks.
fail2ban is working nicely here with debian 9, but at a custom setup with csf. maybe webmin by-default can try to detect firewall used and change fail2ban action accordingly (?)
in your scenario, you should check action configuration most probably, to find what's not working... for csf you'd put : action = csf-ip-deny[name=sasl] for example, with an /etc/fail2ban/action.d/csf-ip-deny.conf like :
CSF / fail2ban integration[Definition] actionstart = actionstop = actioncheck = actionban = csf -d Added by Fail2Ban for actionunban = csf -dr
[Init] name = default
Thanks for your remarks dimirist.
In fact I think it's a issue with firewalld which does not work properly in debian, so thats why you got it working with csf.
I can understand the idea to use firewalld on debian to get a more unique virtualmin environment across linux distros. But then it should be fixed to be working.
Anyway I moved to CentOS now to stay out of this problem. So for me the issue is solved, but all others who still use Debian should be aware.