BIND exiting and shutting down for no apparent reason - random RCODE REFUSED

Howdy,

Are you by chance using an OpenVZ-based VPS?

If so, can you paste in the contents of your /proc/user_beancounters file?

-Eric

I don't think the error message and the exiting is directly related. In your first example, there's over a minute between these events. Can you check how much time passed between each of these incidences?

I'd also think that BIND is hitting some kind of resource limit. You might check the syslog for further errors, maybe OOM messages. Short of a programming bug, it should not simply exit like that. You might check with "tcpdump udp port 53" what the load of your BIND is.

The resolver errors you see are reverse lookups, probably triggered by some service on your machine that wants to know the hostname for an IP address that is trying to connect (e.g. SSHD and Postfix can do that). You might want to check your syslog, auth-log or mail-log for an unusually high amount of connection attempts / login failures, which indicate dictionary attacks.

Still, BIND should not simply exit from something like this. It's meant to process a considerable amount of requests per second.

Did you check the syslog or kernel log for additional errors like OOMs? Otherwise, I'm fairly certain that it's a problem with the virtualization software in use for your server. What's your used and free memory? You might want to use the tool "atop" to record and playback performance data, that should give you an idea if something has a memory spike or similar.

It's not uncommon that we see unusual issues and service that die unexpectedly on OpenVZ-based systems... but typically when that occurs, the "failcnt" field of the user_beancounters file shows some non-0 values in it.

On your system, they all appear to be "0".

How much RAM is in your server, and what is the current usage?

You can determine that by running this command:

free -m

I can only repeat what I wrote last time, and what Eric said too now: We need to know what your RAM situation is. free and atop are recommended to provide that information, and to check historical memory data when BIND crashes.

This is really odd. Memory should be plenty, and no beancounter violation.

Two things I can think of right now: Intermittent memory problems. To find out about those, install "atop". Or the BIND version you're using (being an RC) might be buggy. Ubuntu officially uses 9.8.1, yours is 9.8.2-RC1.

You could try turning on BIND's own logging, including debug logging. Or you might want to check if there's an earlier, non-RC version of BIND available. Are you using third-party repositories for your CentOS? Those often contain versions of software that make trouble when used with Virtualmin.

You'd definitely want to limit recursive queries to your local server, and not allow it for any user on the internet. That's done with the "allow-recursion" directive, or there's also an edit box in Webmin's BIND module for it. So you should set "Do fully recursive lookups" to "Default", and in "Addresses and Topology", set "Allow recursive queries from" to "Listed..." and enter in the box

localnets
localhost

As for the RAM filling up, did that happen when your BIND crashed? You really need ATOP, to have historical data about processes. It could be a script going haywire and spawning many processes, or BIND itself using too much RAM. But you won't find that out without ATOP.

When you have it installed, go to /etc/init.d/atop and modify the line "DARGS=", the number at the end is the interval in seconds that atop uses to store resource data. You might want to lower the default of 10 minutes to e.g. one minute, for your debugging purposes.

Good to hear you found the issue! You see, atop really helps in such cases. Also for me it has helped greatly in the past when nailing down processes running rampant. :)