Replications with remote MySQL Failure

I found another potential issue with Cloudmin (Physical Server) with replication and MySQL.

I have cweb1 (primary, and cweb2 setup. They both work with LDAP-stored users that virtualmin handles management of when creating virtual-servers.

My problem right now is with replication and MySQL. As per documentation here:

https://www.virtualmin.com/documentation/id,running_mysql_on_a_remote_sy...

I setup MySQL to be on a remote host (and that host gets replicated to a secondary database server).

With my replication options set to:

Virtualmin global settings to replicate: Module Configuration, FTP Directory Restrictions, Server Templates and Plans, Custom fields, links and shells (all other options turned off)

Virtualmin features to replicate: Virtual server configuration, Bind DNS Domain, Apache Website, SSL Website, Webmin login. (all other features turned off)

Note that MySQL Database is not turned on in the replicate features, but the problem is the following:

Failed to restore on cweb2.omitted : Checking for missing features .. .. all features in backup are supported Starting restore.. Extracting backup archive files .. .. done Re-creating virtual server domain1 .. .. a clash was detected : A MySQL database named omitted already exists Restoring Virtualmin settings (Module configuration, Server templates and plans, Custom fields, links and shells, FTP directory restrictions) .. .. done Applying FTP server configuration .. .. done Restore failed!

As-in: It's trying to re-create the mysql database on the secondary node, even though that "feature" is specifically not included?

My setup is very similar to how the documentation is from the before-mentioned website on setting up MySQL on a Remote host. The differences are:

On cweb1's Virtualmin -> Server Templates -> Default Settings -> MySQL Database -> Allowed MySQL client hosts: Includes the IP address of cweb1 and cweb2 specifically, space-separated. (seems while this is setup, when creating the initial mysql user, it creates a user with cweb1's FQDN, along with the two IP addressed defined here).

Status: 
Active

Comments

Ok, it looks like the issue is the creation of the default MySQL database for the domain when the backup is restored. The correct fix for this is it configure Virtualmin to not create that initial DB, which can be done at System Settings -> Server Templates -> Default Settings -> MySQL Database, by changing "Create database as well as login?" to "No".

Okay, I did that, turned off the creation of the default database (didn't want that anyway).

Now another issue popped up:

Failed to restore on cweb2.omitted : Checking for missing features .. .. all features in backup are supported Starting restore.. Extracting backup archive files .. .. done Re-creating virtual server domain1 .. .. a clash was detected : A MySQL user named user1 already exists Restoring Virtualmin settings (Module configuration, Server templates and plans, Custom fields, links and shells, FTP directory restrictions) .. .. done Applying FTP server configuration .. .. done Restore failed!

Now it's the USER that it's trying to do. This is beginning to seem like it's completely ignoring the feature of MySQL Database replication. I'm almost certain now that if I were to create a database as said user it would resort to going back to that again. Or still remain a combination there-in.

Furthermore. I went and deleted user1 from the database manually while keeping the actual virtual-server in-place on cweb1.omitted, and here's the results of that:

Failed to restore on cweb2.omitted : Checking for missing features .. .. all features in backup are supported Starting restore.. Extracting backup archive files .. .. done Re-creating virtual server domain1 .. Creating administration group user1 .. .. failed to create administration group : ldap-useradmin::create_group failed : Failed to add group to LDAP database : Already exists at /usr/share/webmin/web-lib-funcs.pl line 1331. Restoring backup for virtual server domain1 .. Content-type: text/html; Charset=iso-8859-1

That's from the LDAP, and do note, in the replication features, "Administration user" is not enabled, which I believe this relates to?

Basically, so far, it seems that using anything "shared", with the "seperate" backup format is going to fail miserably, including NIS/LDAP users, MySQL users/databases, likely even my next up, PostgreSQL users/databases.

Any updates regarding this issue?

I'm afraid if this issue is unresolved and continued to not even be regarded, then I'll be asking for my money back on this. I'm not one to pay for something that advertises it can do things that it flat out does not do because of multiple broken issues that really shouldn't have been there if it was truely a "ready" product.

I need at least some reasonable reassurance this is being looked into and preferably even an ETA when there might be a solution for it. I'm even more than willing to try out immediate patches to insure they work so at least I can use what I was expecting to use in the first place.

This waiting game of absolutely little to no response is bothering me. I'm a man of business and I am a serious programmer and server administrator. I need something to go on here.

If you really need this feature soon and Cloudmin's current method of replication doesn't work for you, I think a refund would be best. Unfortunately the replication feature was not designed to work in the way you are planning to use it, and changing that is going to require quite a few code changes. You can request a refund by sending email to refunds@virtualmin.com

The only option I can offer right now is to use LDAP and NFS, as I mentioned in comment #4 above.

There is no comment #4. It goes 1-3, then to 5-6 and now 7 for this one. There is n

But the problem is, even if in what you say in #6 is true, I've already proven LDAP won't work as it tries to re-create the user/group in LDAP as well on the replicated nodes, which again breaks everything.

NFS is already in-use for shared file storage itself, as it's connected to a glusterfs-based SAN with replication on two NAS servers. The problem is where Cloudmin is having issues with is shared databases, which when setup manually between two virtualmin installations /can/ work, however not with any form of ease just the same, because it too tries to do basically what Cloudmin does. Backup and restore everything, including databases, users and-all.

The entire logic of "Cloud" services is redundancy and high availability, in multiple places, not just in backup solutions, which is kind of what it seems like Cloudmin only does is backup redundancy only, presently.

From what it sounds like, so far, it's not really just Cloudmin that's ultimately failing, but the design of it combined with existing backup/restore solutions of Virtualmin GPL and Virtualmin Pro. They don't have the options to exclude what Cloudmin suggests if could do.

If my analysis is at least reasonably accurate in that regards, I might at least be able to get a target to personally work on to get what I need done myself to get around the problem I'm having. I'd at least ask to get some points of reference so I can see what I can do to fix the problems.

I opted for Virtualmin + Cloudmin instead of writing my own system, because it's a good system.. Well Virtualmin is anyway. Cloudmin's not much more than Webmin with broken backup & restore based replication with some monitoring capability as I see it, so far.

And of course... If I do wait, and not ask for a refund, and even do some work on my own to attempt to resolve the issues myself. I would ask that my license be extended to a point to a year after the official release is done that corrects the issues at hand. If that's a reasonable request to you, then great.

From what I'm seeing in Virtualmin itself.... This is interesting.

Virtualmin itself seems to allow to make backups with the same limits and exclusions as Cloudmin (or at least most of). But It's Cloudmin itself that's just not abiding them exactly.

With that, and a few exceptions to exactly what to restore is in concern, that's ALL that seems to be wrong at first glance.

Such things I'm mentioning that are not accounted for are, for example:

Mail/FTP users and mail aliases -- With this turned off, I'd assume it would not manage the actual user accounts themselves, which includes Postfix's definitions. (this can be gotten around by using unison, or something, on the server to replicate the postfix configuration, and periodically restarting it on the replicated servers).

Contents of server's MySQL databases -- This is only for the databases themselves I would guess? The users associated to the account are not included? This is a possibility of where the problem could be for the MySQL issue.

Spam filtering -- This option is missing from cloudmin completely. I'm now sure how Virtualmin does it's spam rules, but if it's done within the account holder's home directory, this would already be done no problem, on shared storage anyway.

DAV users file -- Again, missing in cloudmin itself, but shared-storage-wise, this likely isn't an issue.

Mailman mailing lists -- Missing again from Cloudmin. Not sure if this is needed like the others mentioned, since part of it would be handled by the mail server routines.

All the global settings seem to match up, for Virtualmin GPL, plus the extras that Virtualmin Pro includes in Cloudmin.

Seems this problem is occurring only during the initial "recreation" stage of the restore process. If the virtual-server is already setup on both nodes it has no issues so far that I have determined.

I found this out while investigating on issue ticket https://www.virtualmin.com/node/15183 I decided to disable LDAP completely from both nodes reverting back to using local users and groups, backed up manually, restored manually, and it worked. Once that was done, making a few changes to the node1 virtualmin settings for the virtual-server then using Cloudmin to replicate had no issues.

Is there possibly some way to "trick" Cloudmin's replication to not enable the "recreation" mode of the restoration process?

Odd, I can see comment #4 ... it reads :

I am wondering if replication as done by Cloudmin is really what you want. Instead, you might want to consider using shared home directories (via NFS), a remote MySQL server, and users and groups in LDAP. You could have a primary system which runs Virtualmin, and multiple load-balancing secondary servers. Apache configs could be shared via NFS, or rsynced via a cron job.

Cloudmin's replication is more designed for backup systems which are totally separate.

You are correct that the re-creation of MySQL and LDAP users is the problem - when a domain is restored, if the user and group already exist, the restore will fail. If Virtualmin was changed to not re-create them when restoring if they already exist, I think this would solve most of your issues ..

From the Cloudmin Replication documentation:

Replicating websites for load-balancing purposes, so that multiple machines can serve the content for a single website.

As per http://www.virtualmin.com/documentation/cloudmin/vm/vsync this is what it's supposed to be /able/ to do, and it doesn't because of all these errors. So I will not accept the excuse that maybe Cloudmin wasn't supposed to be able to do /exactly/ what I'm trying to do.

What I want, are two live load-balanced servers that are replicated as per what was defined it was capable of. Without at least this, then I won't be using Cloudmin and trying to jump through hoops trying to get things working as they were supposed to have been working.

I'm not happy about paying for something that's documented to be able to do something, and the support says that's not how it was designed.

The load balancing would only work when the replicas contain either static content, or are essentially read-only - replication where there is some changable storage or database is more complex.

We would be more than happy to issue a full refund if Cloudmin doesn't do what you expect..

Have the replication issues ever been resolved to-date yet?

Not to support what you are trying to set up, sorry ..