Do I Really Need Cloud Min to Migrate Servers from One Host to Another

ha... I just found that page before I got your response by email.

but it doesn't work to do a "copy all domains" so yes, questions:

1) two of them are huge... 80GB each (most data in public_html/media) where we have lots of audio and image files) so does the back up command bundle all of the into 1 single giant tar ?

2) I need to put two of the domains each on their own independent "instance" /server.

would we still use the

mkdir /root/backups virtualmin backup-domain --dest /root/backups/ --all-domains --all-features --newformat --all-virtualmin

cmd?

what if we don't want to do "all-domains" at once? There are a lot of timing issues with data being generated daily on the blog of the old site... that will be missing on the new site... so we want to go carefully one by one... and then shut down any updates on the old site until DNS migrates (this seems safer than expecting "incremental" back up to just pick up the most recent in the MySQL databases)

3) What about php.ini if the PHP version is different?

4) can I use my current 10 domain Virtual min license on all three cloud servers? (I will still only admin 9 domains total)

5) Does it make sense to do individual back ups first of the "little" sites (not much data) and test those first?

6) Does your "incremental" back up just examine the changes and then port new changes from the old site to the new site? And, does this include additions to MySQL databases?

Andreychek wrote:

"You just need to generate a backup for the domain, you can do that any way you want, and copy that to your preferred server."

"anyway you want" ... I presume in the virtualMin GUI we can use

Backup and Restore --> Backup Virtual Servers (panel dialog) --> select a server --> "Any plan" [leave as default] --> Features to back up (Select All) --> Virtualmin settings to back up (select all)

and we get the same result as the command line?

I'm looking at your docs for cmd line operation -- does this look right? (where, in this example, the single domain would be hinduismtoday.com)

virtualmin backup-domain --dest /root/backups/ --domains hinduismtoday.com --all-features --newformat --all-virtualmin

Did I get that right?

and what about the tar for VM itself?

virtualmin restore-domain --source /root/backups/virtualmin.tar.gz --all-virtualmin

is this generated when we use

[--all-virtualmin] | [--virtualmin config]

the first time? and then for subsequent domain back ups we don't need that?

Also why don't we need " [--as-owner]" if you back up as root... is the owner/group of all the files preserved? and recreated on restore?

??

Backup Virtual Servers

One last question and I'm good to go

VirtualMin GPL should work fine on the two servers with one domain each, right? I guess I would need to restore a copy of the Vm tar file on each of the three servers (where there was only one before) VM GPL will restore the back up made from VMPro?

Or maybe we let the VM GPL start fresh on those two?

Alright we made progress, spun up a new "linode" and installed virtual min and tried to do a back up of one small domain:

Faced some issues:

1) on the fresh in stall on ubuntu, the final step to verify VirtualMin config failed due to lack of a Mailman list of some sort. "Virtual Min Not Ready to use" was the report at the bottom.

2) we then went over to the current/old server and ran

[root@sat backups]# virtualmin backup-domain --dest /root/backups/ --domain wailuarivernursery.com --all-features --newformat --all-virtualmin

And we got a failure (see below)

what now?

Starting backup.. Creating backup for virtual server wailuarivernursery.com ..     Copying virtual server configuration ..     .. done

    Backing up Cron jobs ..     .. none defined.

    Copying records in DNS domain ..     .. domain not found!

             Saving Virtualmin configuration ..         .. done

        Saving templates and plans ..         .. done

        Saving resellers ..         .. done

        Saving email templates ..         .. done

        Saving custom fields, links, categories and shells ..         .. done

        Saving custom script installers ..         .. done

        Saving custom styles ..         .. done

        Saving scheduled backups ..         .. done

        Saving FTP directory restrictions ..         .. done

        Saving DKIM settings ..         .. not installed

        Saving greylisting settings ..         .. not installed

        Save mail rate limiting configuration ..         .. not installed

        Saving mail server configuration ..         .. done

    .. done

    Backup failed!

DNS IS hosted by a third party

What about the inability verify VirtualMin on the new linode because of missing mailman list (it seems like VM want a list to be created...)

Sorry I was reading peddle to the middle and missed your mailman response.

And, yes we had DNS Bind on for some sites when it is not needed, will uncheck and try again.

andreychek:

I just wanted to say that we succeeded with the new VirtualMin Install on the new host server at linode, backed up several small domains on the old box, ran the scp to push the tar's over to the new server, ran restore one of them late this afternoon and all went well! getting the site in preview...

That site only required Apache and nothing else, so we will see how things go with the more complicated ones, but so far, so good!

Thanks for all your guidance!

We have ported five small domains to the new server and they are all on a shared IP, the restore process seems to have gone without a hitch. We are getting new home folders for each new user, and httpd.conf directives all look good with the correct doc roots pointing to the correct doc roots for each user.

BUT...when we try to preview the sites we restored today, they all default to the doc root of the first site that we restored this morning. Just guessing, but it appears to us that virtual min is somehow "stuck" on the doc root for one user/site. We dont know how to fix this.

All the settings that we see are unique for each restored site. But we are seeing the homepage for Domain A even when trying to preview Domain B.

root@hostname:/home/wailuanursery/public_html# grep -i <virtualhost /etc/apache2/sites-enabled/*.conf -bash: virtualhost: No such file or directory

root@hostname:/etc/apache2/sites-enabled# grep -i '<virtualhost' /etc/apache2/sites-enabled/*.conf /etc/apache2/sites-enabled/000-default.conf: /etc/apache2/sites-enabled/0-wiki.hindu.org.conf: /etc/apache2/sites-enabled/0-wiki.hindu.org.conf: /etc/apache2/sites-enabled/eastkauaiwater.org.conf: /etc/apache2/sites-enabled/jnanadana.com.conf: /etc/apache2/sites-enabled/kauaialohaendowment.org.conf: /etc/apache2/sites-enabled/wailuarivernursery.com.conf:

interesting... I see that in the GUI for the Webmin-Apache Server... only the Wailuarivernursery has an actual IP assigned... and that is the one that is showing for all the others that we try to preview. the asterisk is translated as "Any"

also... 0-wiki.hindu.org.conf is wrong... why a "0-" in front of the domain?

FYI these were all backed up on the server in SF (using VirtualMin) scp to the new linode image and then restored there

They all had and asterisk before the port:80 (btw... where is a like to your markdown rules for these posts?)

we got this:

root@hostname:/etc/apache2/sites-enabled#  grep -i  virtualhost' /etc/apache2/sites-enabled/*.conf
/etc/apache2/sites-enabled/000-default.conf:<VirtualHost *:80>
/etc/apache2/sites-enabled/0-wiki.hindu.org.conf:<VirtualHost *:80>
/etc/apache2/sites-enabled/0-wiki.hindu.org.conf:<VirtualHost 45.56.82.123:443 [2600:3c01::f03c:91ff:fe37:416]:443>
/etc/apache2/sites-enabled/eastkauaiwater.org.conf:<VirtualHost *:80>
/etc/apache2/sites-enabled/jnanadana.com.conf:<VirtualHost *:80>
/etc/apache2/sites-enabled/kauaialohaendowment.org.conf:<VirtualHost *:80>
/etc/apache2/sites-enabled/wailuarivernursery.com.conf:<VirtualHost 45.56.82.123:80 [2600:3c01::f03c:91ff:fe37:416]:80>
[code]
 
I went in and manually edited all those *conf files and entered the full IP string from the nursery site and restarted apache... now they all work... but it would help us to understand if we did something wrong upon restoring the sites.

I fixed the conf files manually form with nano... they all preview as expected now. Must have been some quirk in the restore process that we missed.

But still I would like to preview sites form the IP so that we can avoid and strange behaviors... I looked up this issue... seems it is not recommend as a long term strategy for security reasons, but for the period of this migration it would be good if I could test sites on a shared IP using

http://44.44.44.192/~user

or something like that... is that possible? in some of these PHP CMS's (XOOPS, DokuWiki and others) they want to set a baseURL... since DNS is not cut over... would like to be able to enter that BaseURL in that form temporarily.

is that possible? put another way:

is the documentation on this still valid as of May 2015:

http://www.virtualmin.com/documentation/web/faq

But this documention is either outdated or unclear as I am unable to locate the "Click on the Automatic Virtual Hosts icon."

[FYI: I understand the security risks and would turn this all off as soon as testing is finished. this is a not a long term strategy for reselling space to let clients see their sites before going live... it's just to get me thru this week.]

Those issues are increased exponentially by allowing CGI/PHP scripts to run in this way, and we strongly discourage use of UserDir with executable applications, as it introduces serious security problems.

docs say:

However, you can use this mode, if you wish.

To switch your Virtualmin server to behaving the way you're used to, you just need to turn it on in Apache. Here's how:

Browse to the Apache Webmin module.

Click on the Default virtual server (just be sure you know what your default Domain is so you can point folks to the right place).

Click on the Automatic Virtual Hosts icon.

Switch the radio button by Automatic virtual host root to the one beside the empty field, and fill in the field with, /home/%0/public_html. Note that if you've configured Virtualmin to create domains with a different path convention (e.g. "/home/a/abc") things get more complicated and you'll need to construct a path using the variables documented here:

http://httpd.apache.org/docs/1.3/mod/mod_vhost_alias.html

Not sure I understand that very well... let me feed back to you:

in the DNS... we would take an unused domain... say jnanadana.org

point this to the new IP on the new cloud instance... which is shared by 5 different domains.

So then if I want to preview one of the domains that was migrated... I just create an alias domain

"www.jnanadana.org"

the we can use that domain to preview the migrated site?

Did I get that right?

I guess obviously you could only do this one at a time since 5 domains share the same IP (but that's OK...)

OR I will set up 9 subdomains just for test the 9 sites we are migrating

Yes that make sense.

1) of course the alias domain needs to be pointing to the IP that is propagated globally...right? OR could onejust set that up in his local hosts file on hgis own box and map

45.46.96.123 example.jnanadana.org

2) I've already imported all domains except one... :-( so how do i set up the alias domains now?

I went ahead and tried the "intuitive" route... navigated to a domain the tab under virtual Min

Create Virtual Server --> Click on link for "alias of wailuarivernursery.com"
--> enter "test.wailuarivernursery.com --> uncheck "Create website --> save

now go to my local host file on my own box

enter

45.85.46.128 test.wailuarivernursery.com

then go to my browser and to go for it.

OK so, it works to the point of getting back to the Linode instance but I get:

"Forbidden

You don't have permission to access / on this server."

So... still missing something.

Awesome!

I created

test.domaina.com test.domainb.com

etc. for the domains on the shared IP

and added them to my local hosts file

45.83.123.257 test.domaina.com

45.83.123.257 test.domainb.com

it works!

You have been so patient! Can I help by writing up the documentation for you on this one?

What is the mark up we use here for code? you use standard Markdown? on this forum?

So far so good, migration is going well. Most web sites are up and running on the new linode instance and the setting of an alias and adding that to the host file on my own machine works great for previews

But I have on "stubborn" case... where my.domain.org

returns this on any attempt to preview or look at any file in any directory.

Forbidden You don't have permission to access / on this server.

I restored this domain from a VirtualMin back up, like all the others. But even if I create a simple /home/mydomain/public_html/index.html file with "Hello, My Domain is Up and Running."

and try to preview, either externally using the "test.mydomain.org" method or from the "Preview web site" option inside VirtualMin we get the same error.

Forbidden You don't have permission to access /index.html on this server.

I'm trying to examine carefully all settings in VirtualMin for this domain, and compare them with settings for the domains that do work on the same shared IP... but I can't find any differences. (Obviously there must be some difference) the home/mydomain owneruser and the owner and group for all files match... permissions all look correct...

So why is access forbidden?

hhmm. I'll next look at the HTTPD directives...

I just found one discrepancy... on this one "myDomain.org" in the web site configuration, if I check the "Users and Groups" it shows "run CGI as unix user 1009 group 1004

But the other sites have the same number in both of these fields. Could there be some confusion at the unix user level?

cd /home/wikihinduorg/logs tail -fn100 access_log

I get a line appended on each attempt to get to the "test.wikihindu.org" alias that I set up (as suggest above) which works for other domains.

Switching to the error_log we see:

which is a totally new thing for me! I looked thru as many settings as I could find in VirtualMin for this domain and I'm not seeing something that would cause this (frankly though, I may still not be looking in the right place)

Interesting...I decided to leave my tail of the error log running...

I'm also seeing another strange error on this virtual server/domain (which was backed up and restored using exact same procedure as all the ones that do work...)