Auto edit DNS Records if primary server goes dead

Submitted by sgrayban on Sat, 12/03/2011 - 02:34

I like the idea of the "Roundrobin DNS Records" but I would like to see a new feature added that will auto update the dns if 1 server goes down and then once the primary is back again it would change the DNS A records back. Of course it would need to edit more then just the "domain.com A IP"

I thought about heartbeat but that's way beyond what I am willing to take on and learn. I figured if you can write the "Roundrobin DNS Records" code you could add "Fallover DNS Records" to do what I would like.

It would be a lot simple then trying to setup heartbeat.

Status: 
Active

Comments

Submitted by JamieCameron on Sat, 12/03/2011 - 12:37

That's pretty much what the roundrobin DNS records feature does already though - you can define a list of systems whose IPs will be included in A records, but only if they are up or meet some other conditions..

Submitted by sgrayban on Sat, 12/03/2011 - 21:12

No it doesn't... it adds whatever record type to use round robin dns.

Lets say I have the following servers added to cloudmin...

master1 with IP 127.0.0.2 and www uses 127.0.0.3
master2 with IP 127.0.0.4 and www uses 127.0.0.5

Now master1 is the primary and master2 is the fallover encase master1 fails.

This isn't supported with "Roundrobin DNS Records" at all, In fact it isn't even close. It adds the IP the Cloudmin server is using. So say I add a the WWW record now the WWW record has the wrong IP because roundrobin is using the IP associated with webmin server; EG 127.0.0.2 and 127.0.0.4

What I was proposing with "Fallover DNS Records" is if ping *and* port 80 fails after X checks it edits "domain.com.hosts" and replaces the the A record for www with 127.0.0.5 or whatever IP I set it to.

Now once ping *and port 80 responds "Fallover DNS Records" will reverse that and replace the A record for www with 127.0.0.3.

Round Robin is just that -- its a load balancer and it doesn't work the way I described unless I don't understand what Round Robin dns is which I highly doubt.

Submitted by JamieCameron on Sun, 12/04/2011 - 00:55

Ok, so your systems actually have multiple IPs and you are trying to use the non-primary IP in DNS?

Submitted by sgrayban on Sun, 12/04/2011 - 01:25

Correct but that isn't the whole problem with "Roundrobin DNS Records"

It does actually what round robin does and that is make www answer on 2 IP's

www.domain.com IN A 127.0.0.3
www.domain.com IN A 127.0.0.5

That's round robin dns -- it sorta acts like a load balancer and it's what I was asking for.

I wanted my version "Fallover DNS Records" to only edit the A records for

www.domain.com IN A 127.0.0.3
domain.com IN A 127.0.0.3

And change those records to the IP user provides for it in the settings.

Submitted by JamieCameron on Sun, 12/04/2011 - 13:53

So in your failover case, if both the machines on .3 and .5 were up, which one would go into the DNS record? Would it always prefer .3 unless it was down, in which case .5 would be used?

Submitted by sgrayban on Sun, 12/04/2011 - 14:55

.3 the primary would be preferred unless port 80 AND ping failed after X checks then .5 would takew over and the only records changed would be for www and domain.com.

Submitted by JamieCameron on Sun, 12/04/2011 - 19:19

OK, so it sounds like what you want is for Cloudmin to have an option to only create an IP record for the IP of the first usable system, rather than all of them. I will include that in the next release..

Submitted by sgrayban on Sun, 12/04/2011 - 23:02

It will work if you can define the fallover IP and record to modify

Submitted by JamieCameron on Sun, 12/04/2011 - 23:35

The next release will also let you enter arbitrary IP addresses to check and include in the roundrobin, which could be systems that aren't even managed by Cloudmin if you like..

Submitted by sgrayban on Wed, 12/14/2011 - 19:00

I still don't see where to put the IP to use if the selected server is down.

The screen for round robin is really confusing and there is no help files or hints for any settings.

I don't how to explain this feature request any better.

I know how much you hate talking to people but I think its time for a actual phone call to get our thoughts in sync on this one.

Submitted by JamieCameron on Wed, 12/14/2011 - 19:18

What you want is to create a roundrobin with the primary and backup IPs in the "Systems with IP addresses" field, and "Address records to include" set to 1.

A little clunky, but this feature wasn't really designed for DNS-based single-IP failovers like this.

See https://www.virtualmin.com/documentation/cloudmin/vm/roundrobin for more docs.

Submitted by sgrayban on Wed, 12/14/2011 - 23:54

Not working like I expected then

www.centerforsecuritypolicy.org IP addresses 74.208.113.42 50.23.11.112 0 up, 2 down (DNS error : No usable systems found - leaving DNS record un-changed)

I guess I need to figure out how to use heartbeat to do this.

Submitted by sgrayban on Thu, 12/15/2011 - 00:07

Ya it's definitely not working the way I explained.

I made it fail and the secondary IP I used was 50.23.11.112 but it still forced the IP for master2 instead of what I told it to use.

I can't believe something this simple is so hard to understand.

If IP 74.208.113.42 goes dead I want IP 50.23.11.112 to be used... you just can't get that any simpler. I don't care about if the hostname for the server is up all I care is if the website for a domain is..

Submitted by JamieCameron on Thu, 12/15/2011 - 01:11

If you click on the roundrobin and open the section that shows the status of the hosts, what does it say as the reason why the hosts were rejected?

Submitted by sgrayban on Thu, 12/15/2011 - 01:21

That wasn't a issue after I figured it out...

The issue was that it was still wanting to use the physical IP for master2(50.23.47.2) and NOT the IP(50.23.11.112) assigned to the website that I told it it to use in IP section.

EG if the main IP failed it did this...

www.centerforsecuritypolicy.org. IN A 50.23.47.2

and that is wrong, it should be 50.23.11.112

I have given up trying to explain this (not rocket science) and instead looked for another option like a script which I did find and works exactly how I was trying to explain here. Maybe if you look at the code you will see what I have been trying to get cloudmin to accomplish.

http://gregsowell.com/?p=191

<?php
//
//servers
$Main = "74.208.113.42";
$Backup = "50.23.11.112";

//path to named folder
$Path = '/etc/bind/';
//domains to check through for the IP replacement
$Domains = 'centerforsecuritypolicy.org';
//A record to look for
$Arecord = 'www.centerforsecuritypolicy.org';
//check for a command line parameter
if ($argc >= 2) {
        if ($argv[1] == 'up') {

                //set the A record back to main
                echo "up \n";
                $FindIT = $Backup;
                $SetIT = $Main;
        } elseif ($argv[1] == 'down') {
                //set the A record to backup

                echo "down \n";
                $FindIT = $Main;
                $SetIT = $Backup;
        }
        $lineNum = 0;
        echo $Path . $Domains . '.hosts' . "\n";

        //open the domain file
        $lines = file($Path . $Domains . '.hosts');
        //open the file for editing
        $fp=fopen($Path . $Domains . '.hosts',"w+");
        foreach ($lines as $line_num => $line) {
                $lineNum = $lineNum + 1;

                //increment sequence #
//              if ($lineNum == 3) {
//                      $line = $line + 1;
//                      $line = "                       " . $line . "\n";
//              }
                $pos = 0;
                //loop through each line checking if it matches $FindIT exactly with the
                //last values at the end of the line.  If found, change it to $SetIT
                $pos = strpos('test' . $line, $FindIT);
                if ($pos > 0) {
                        //we found the line
                        $line = str_replace($FindIT, $SetIT, $line);

                }
                //write each line as we loop through
                fwrite($fp,$line);
        }
}
        //reload bind service
        exec('dnstouch /etc/bind/centerforsecuritypolicy.org.hosts');
        exec('rndc reload');
?>

Submitted by JamieCameron on Thu, 12/15/2011 - 23:41

I see the bug that causes this - will fix in the next release. Let me know if you'd like a patch ..

Submitted by sgrayban on Sun, 12/18/2011 - 13:29

Sure you can send a rpm and I'll test it.

Submitted by sgrayban on Mon, 12/19/2011 - 13:06

Opps I meant deb package.. sorry

Submitted by sgrayban on Thu, 12/22/2011 - 03:17

Ok that works but that php script works much better as it replaces all the same IP's with the fallover IP and visa-versa when the primary server is back online.

Maybe you guys should post that script on the wiki err docs. Man I miss that wiki... I really liked the idea I could maintain my own docs.