I have a second (new) server. I want to migrate all domains from existing server to the new server. I didn't find anything on the documentation wiki to cover this. I also tried searches in the forum, but either I'm not a great searcher (likely), or the topic doesn't already exist...
Couple questions:
1) Should I create a single virtual site before migrating any of the existing sites? (I.E. the server currently has zero sites).
2) Is there a special order to migration?.
3) Is there a batch migration option?
4) I have some subdomains/subservers within the same home directory as the parent, and some as separate home directories.
So, what happens when I migrate the parent directory in those cases? What is the recommended way to move those domains, etc.
5) The most basic question: How to do the migration, exactly? I.E. a) backup and transfer the backup? b) import on the new server?
When would I "turn off"/delete the site on the old server, in this case?
(I'd wish there was a single point to administer this on the NEW server, which would connect and migrate automatically, in a transaction)
Same question here im migrating to a new server from FC3 x86 to Centos5 x64
And with all the diff's like mysql 4 /5 php 4 /5 etc is pretty hard.
But also with plain website it doesnt work that great to backup and restore what is the best way to do this ?
Maybe Joe can give a write up on his thoughts...
As I think I understand virtualmin & webmin little bit I think that everything is written into $prefix/etc/webmin configuration files (for webmin and virtualmin) - you can inspect this folder and move necessary configuration files (but I don't think it is neccessary) - other files are configuration files for each server (like Squid, Apache, ProFTPd etc.) and webmin is parsing them and writing configuration into them instead of admin hands. So if you will move those configs onto your new server it should work cause newer versions of software should understand older config files.
Of course there are rules like BACKUP EVERYTHING and TEST IT WELL. If you are moving on new server I think that there may be no problem with this as you can undo whatever you did and try it again. When you will be ready with you new setuped server you may start with changing DNS records. Maybe there can be a problem with TTL as been mentioned before if you have many users and they are changing site contents often - but this can be solved by lowering TTL for DNS records to minimum and than writing something - site is moving to new one into each website on old server for those people whom will come to old server.
I did quite a couple of these virtualmin to virtualmin migrations in the last couple of month and often had problems, though all server were the same OS (Debian w. apache2, postfix, mysql).
My hint if things fail: turn an option off an back on. Make sure you have identical templates. Do serious testing (esp. w. mail accounts)
I would very much appreciate a feature that eases V2V migrations and which checks when don, that REALLY EVERYTHING is transferred, in place and working.
H.
<div class='quote'>I did quite a couple of these virtualmin to virtualmin migrations in the last couple of month and often had problems, though all server were the same OS (Debian w. apache2, postfix, mysql).</div>
File bugs! We can't fix it, if we don't know it's broken.
It works in our tests...so we need you to tell us when things don't work.
--
Check out the forum guidelines!
So joe.. you confirmed that andreychek how to works in your test, right?
andreychek is usually smarter than I am. You can trust his advice, generally, and if not, I'll usually amend or correct. I see nothing to complain about in this howto.
Since I have far fewer sites to move than some folks, I usually do make the move one virtual server at a time, which means I can focus on making sure everything is right for each one. But, it's merely a matter of order of operations, not any particular right or wrong.
--
Check out the forum guidelines!
Joe, how to do this?
"Lower the DNS TTLs so prevent DNS from being cached for too long (the $TTL number at the top of each domain.com.hosts file)"
Thanks.
It's possible Virtualmin has some fancier way of handling this that I haven't yet run across, but I just did it using a terribly dangerous command I ran from the command line :-)
This command changes a whole ton of files at once, and there's no going back! So if you use this, please please back up all the files first :-)
If you look at your DNS zone files (mine are in /etc/bind/), you'll see some number at the top -- something like "$ttl 38400". You just need to change it to a lower number.
I did that using something like this command:
perl -pi.bak -e 's!\$ttl (\d+)!\$ttl 1800!' /path/to/bind/files/*.hosts
That changes all the files from it's currently value to 1800 (meaning, 30 minutes).
Then restart Bind.
I highly recommend backing up all your files before doing that, and second, I'd test the command on a dummy copy of the zone files before making changes to your production system :-)
Oh, and after your move is finished, you can use a command similar to the above to change the numbers back to how they are now.
-Eric
<b>PatrickLambooy wrote:</b>
<div class='quote'><div class='quote'>I did quite a couple of these virtualmin to virtualmin migrations in the last couple of month and often had problems, though all server were the same OS (Debian w. apache2, postfix, mysql).</div>
File bugs! We can't fix it, if we don't know it's broken.
It works in our tests...so we need you to tell us when things don't work..</div>
Hey, 1 year + w. no need to move Virtualmin :)
The problem I had today is w. templates.
On the old box the template all virtual servers were created with is ... lets say "xyz" now, on the new box I created "xyz" as clone from the default template and compared/adjusted all settings to be as on the old box.
When I restore the backup, it shows all virtual servers that will be imported (and created), but when I hit Restore Now, I get:
[code:1]Starting restore of 128 domains from local file /root/backup.xyz ..
Extracting backup archive files ..
.. done
Re-creating virtual server domain.com ..
.. the server template specified in the backup does not exist
.. failed! See the progress output above for the reason why.[/code:1]
How can I transfer the template from the old to the new server?
Cheers!
There may be a better way than what I'm about to suggest. But, I also suspect this will work just fine :-)
It looks like the templates are stored in this directory:
/etc/webmin/virtual-server/templates/
The various files in "domains" reference those templates by their filename.
By copying over the files in the templates directory, that should handle the error you're seeing above.
-Eric
thanks a bunch, Eric, that worked perfectly! :)
Though the restore ended w.: "... failed! See the progress output above for the reason why.", no errors were shown and a option "Validate Virtual Servers" reported that everything is o.k.
I reckon the developers should take a look at V2V Transfers resp. V2V Migrations, cheers!
Hans
I don't know if this will be a thread resurect or high-jack & if it's a problem if I do so.
But, I am going to state clearly what I've inferred is the process :
On my old Virtualmin server : 3.48 gpl. I save a backup & ssh it right across to my new server.
Then on new server I run the restore backup option, tell it which file to grab & it seemed to do what I was hoping it might do.
I have 2 concern's with this process, 1) I have one user with about 60GB worth of data. It's all on an internal network, so moving it isn't a huge problem w/cost's or speeds, but I was wondering if I could backup just the config, but not the home folder, & then Rsync the data myself. -Is there a better way? or is that what others do?
I thought I'd try that with another site & right now it seems to hung at the "updating webmin user" section of the Restore. - I guess that's my secdond thing.
By VM2, what do you mean Sgrayban?
My imported user mentioned before.. never seems to have imported correctly. I try loggin in now & check the Virtual server for the user settings I restored & it causes my VM interface to seem to crash. When I type TOP while ssh'd, Perl seems to be taking over the system.
Is there a command-line way I can undo this restore?
<div class='quote'>By VM2, what do you mean Sgrayban?</div>
Scott's talking about Virtualmin Machine Manager, which is a tool for mass server management (including virtualized and dedicated servers, both with and without Virtualmin). It is currently in private beta with a few of our larger customers--folks who have more than a few Virtualmin systems.
You can perform migrations without VM2, but it does make it moderately simpler. There will be a low end version of VM2, with no virtualization support that I believe will be priced at $198. I hope to launch it publicly in a week or two.
--
Check out the forum guidelines!
Joe, what do you mean by W/o virtualization?
Not to confuse the definition of virtualization, but do you mean virtualized OS/hardware, the whole 9 yards that my little GPL version doesn't do? (that I am aware of i.e. xen, VMware) Or do you mean the virtualization existing GPL user's are using/used to.
I haven't looked at the full feature spec's of the Pay V-min since it became for profit, didn't know it did full on Virtualization.
Well, the "restore" I did seems to have caused a problem. (mind you, I know it was a screwed up backup I made to begin with) I think it's responsible for my box repetitively crashing now. I've got to kick the reset button & see if I can fix it via SSH, because I cannot get Virtualmin to even display the config web-terface if I choose that user.
<div class='quote'>Joe, what do you mean by W/o virtualization?
Not to confuse the definition of virtualization, but do you mean virtualized OS/hardware, the whole 9 yards that my little GPL version doesn't do?</div>
I mean OS/hardware virtualization. When I'm talking about what Virtualmin does (GPL and Professional) I use the phrase "virtual hosting". Not the same thing, though that darned word "virtual" comes up in both cases. It sounds like you're thinking VM2 is somehow part of Virtualmin Professional...it isn't.
<div class='quote'>I haven't looked at the full feature spec's of the Pay V-min since it became for profit, didn't know it did full on Virtualization.</div>
VM2 is not Virtualmin (GPL or Professional). They are two separate products--though they happen to work well together and when used together they are more powerful, you don't need one for the other or vice versa.
Virtualmin does no hardware virtualization, and never will. It is not at all operating at that level. It manages virtual hosting websites and the related users.
VM2 manages lots of systems--including virtualized systems of many types. In its simplest (and cheapest) form, it manages regular old dedicated servers. It just makes it easier to keep up with what all of your Virtualmin systems are doing. In the more advanced forms, it creates and manages Xen, vserver, Solaris Zones, and EC2 instances across any number of machines.
<div class='quote'>Well, the "restore" I did seems to have caused a problem. (mind you, I know it was a screwed up backup I made to begin with) I think it's responsible for my box repetitively crashing now.</div>
That doesn't make any sense. A restored virtual host couldn't possibly cause a system crash--the system would have to have serious problems for anything in userspace to be able to crash it.
If you can give us some idea of the errors, we can probably guide you on getting things straightened out...but if your box is crashing, I can assure you that you have bigger problems than your restored websites.
Also, you'll want to start a new thread specific to your problem. We get a lot of queries every day, and if the threads get too long and convoluted we have a hard time keeping up and figuring out what problem we're talking about.
--
Check out the forum guidelines!
Joe,
Thanks for the clarification about VM2, I will Definitly have to check that out, as a huge fan of Webmin from the day I found out about it & onto know, it's cool that it's grown to this level.
Your 100% right about my box, it's not the V-host that crash's it or virtual min, its something to do with disk activity, (or network activity) I will investigate that & make a new thread, if I actually do end up having prob's that are truly relavent. I'll also leave this thread alone.
Howdy,
I just did something similar! A few thoughts --
<b>tabletguy wrote:</b>
<div class='quote'>
1) Should I create a single virtual site before migrating any of the existing sites? (I.E. the server currently has zero sites).
</div>
So long as you have a decent way to get to your server, I'm not sure it matters that you have a site setup there.
<div class='quote'>
2) Is there a special order to migration?.
</div>
There shouldn't be. If there is, it's likely a bug -- save the backup file so you can give it to Jamie and allow him to fix it :-)
If you run into an error importing one of your virtual servers, simply skip that one and go back to it later, it might just work once everything else is done.
<div class='quote'>
3) Is there a batch migration option?
5) The most basic question: How to do the migration, exactly? I.E.
</div>
Where I would start is to perform a normal backup of your Virtual Servers. I like the "One file per server" option (which creates a tar.gz file for each domain you have hosted there).
When you're done, you'll have a directory full of tar.gz files. Copy that entire directory to the new server.
Then go into Webmin on the new server, choose "Restore Backup", and point it to the directory full of your tar.gz files.
At that point, it will give you a list of what all it plans to do during the restore.
<div class='quote'>
4) I have some subdomains/subservers within the same home directory as the parent, and some as separate home directories.
</div>
I suspect you'll be fine here, but you might need to give it a shot and find out ;-) You don't need to make it live until you see things working.
<div class='quote'>
So, what happens when I migrate the parent directory in those cases? What is the recommended way to move those domains, etc.
</div>
If you're able to, I might recommend doing it all at once (ie, perform the backup of everything, copy, restore).
<div class='quote'>
When would I "turn off"/delete the site on the old server, in this case?
</div>
I'll offer the steps I used for my migration, I think that might help answer your question:
* Warn users that stuff is moving :-)
* Lower the DNS TTLs so prevent DNS from being cached for too long (the $TTL number at the top of each domain.com.hosts file)
* Backup Virtual Servers
* Copy Virtual Server backups to new Server
* Restore Virtual Servers
* Log into Webmin on the new server, and for each Virtual Server, click Services -> View Website via Webmin, in order to verify that each site is working (remember, the sites aren't live yet, but this Webmin feature allows you to see them even when not live)
* Once things appear to be working reasonably well, log into the registrar and update the "glue records" -- that is, make sure the IP address assigned to the nameservers being used is pointing at the new box, rather than the old one
* Put the DNS TTL's back to normal
I wouldn't get rid of the sites on your old server for at least a few days after the transition. Some folks may still be hitting your old server for a few days since DNS records are cached, it may take a bit before everyone is hitting the new server. Lowering the DNS TTL's should help with this, but still... :-)
Also, how well the above works depends on how much stuff you need to transfer and how long that transfer will take. If you can do all that in a few hours in the middle of the night, that's one thing -- but if it takes days, you might want to consider running rsync after the restore to pull over anything from /home on your old server that may have changed since you performed the backup.
There's also the option of migrating one Virtual Server at a time, but I personally find that cumbersome :-)
Good luck!
-Eric
Thanks for the response.
One more aspect of the migration (or perhaps a couple)
There are other settings (customized letter formats, custom defaults, etc) for Virtualmin, as well as customized Squid settings and firewall rules.
Is it possible to migrate all of those items (maybe cron jobs as well).
Most of the above was setup by other than me, and some by installs (optional steps outside of Virtualmin scripts).
Can anybody confirms that this how to work? By the way.. Joe, do you have official how to on migrating virtualmin to another virtualmin box which is newly set-up using the install.sh? Hope to hear from you. THanks.
Another migration issue that was not obvious to me, anyways, was a move from Apache 2.0.xx to Apache 2.2.xx Some of the Mod Rewrite handling in .htaccess changed. (I'm sure many other things changed too, but this is one that specifically had subtle effects for me).
Because of that change, one version of a URL rewrite stopped working, redirecting to the base URL instead of the blog URL when the "www" was removed, while other versions continued to work. It happened to be with Wordpress blogs, so that a large number of incoming links were "broken" for a couple weeks.
So, part of testing should be to check things like this, and if you have an analytics package (Google, others), to monitor them for changes.
In this case, I moved 3-4 sites every couple days, which let me run some tests for each site. One site in particular I had to switch back twice for various reasons.
I do VM -> VM moves as part of my business using the VM2 module which really does a wonderful 99.9% of the time. The only thing I usually check is the templates/plugins and make sure they match after that I simply do a move from the old server directly to the new one using SSH and only disable the site on the older server.
50 domains takes about 6 hours to totally move and make sure they are operating correctly.
Hello,
I'm running Virtualmin pro on two Ubuntu 10.04 LTS servers, one or more of which I am fairly certain has been compromised (they're both behind a NAT, and their shared IP address was listed as a spam-sending address in the Composite Blocking List). I may eventually be doing a total backup/reinstall of both systems, because I have been logging and monitoring every single network socket over the time periods that the CBL has reported there being suspicious activity / traffic from the IP address, and haven't been able to find anything out of the ordinary -- which to me suggests the possibility of a very creative blackhat who is very good at covering his/her tracks.
Despite how reinstalling would typically be a last resort, one would think that making provisions for such a scenario would be necessary since sometimes it is inevitable (it's even recommended by the network admins that blocked us; "If all else fails, you may need to have your machine's software re-installed from scratch.") but also because migrating from one machine to another is not an atypical IT task.
It has been three years since this thread was made and I'm wondering if there's anything in the documentation yet that I'm missing on how to migrate/reinstall from scratch and keep existing VPS / Webmin / Virtualmin configurations, since I can't find any.
Most spam that we see is due to a web application that has been compromised... a spammer will break in via that web app, drop in a malicious file, and then use that malicious file in order to send spam.
So, if you migrate all your websites to a new server, it's plausible that you'd just migrate your problems to that other server :-)
One thing you may want to try is running a malware scanner, such as this one:
http://www.rfxn.com/projects/linux-malware-detect/
However, if you're interested in migrating to a new server, you can use these instructions here:
http://www.virtualmin.com/documentation/system/migrate
Thanks for responding, and for those helpful links. I did find (and fixed) a WordPress vulnerability (the Timthumb hack, which left a /tmp/.tmp/ directory), but am not 100% certain that it was the cause of the spamming. This is especially so in lieu of the CBL's listing for the IP address showing that it was detected as having sent spam over ten hours after I had rectified that issue.