BACKUPS Failing to write .serv files

Submitted by xtremeservices on Fri, 04/20/2012 - 03:56

Hi Guys,

Hope you are doing well, I came across an issue in Cloudmin 6.2 that I think is pretty serious. I have my XEN instances backed up nightly as a precaution after having my data store corrupt about 10 days ago and have noticed the following in the logs recently:

Backing up ns1.xtsuniverse.net to /backups/xen-images/nightly/ns1.xtsuniverse.net.gz on FTP server 10.1.10.10 as manager ..
    Compressing LVM disks for ns1.xtsuniverse.net ..
    Sending backup for ns1.xtsuniverse.net to /backups/xen-images/nightly/ns1.xtsuniverse.net.gz on FTP server 10.1.10.10 as manager ..
. created backup of 537.48 MB

Saving details of system ns1.xtsuniverse.net to /backups/xen-images/nightly/ns1.xtsuniverse.net.serv on FTP server 10.1.10.10 as manager ..
. saving failed : command-line line 0: Missing argument.
lost connection

Also, I was attempting to restore backups of images using the Backup Log yesterday morning after upgrading to Cloudmin 6.2 and kept getting a Connection Timed out message instantly as shown in the attachment. I tested FTP logins from the host Cloudmin master node to the backup and had no issues connecting. I did however get the following message which I can't say for sure is new or not :

[root@gx3 ~]# ftp
ftp> open 10.1.10.10
Connected to 10.1.10.10.
220 Welcome to blah FTP service.
530 Please login with USER and PASS.
530 Please login with USER and PASS.
KERBEROS_V4 rejected as an authentication type
Name (10.1.10.10:root):

I ended up having to manually transfer a backup file at 3am yesterday via ssh to the host I was trying to restore to and unzip it manually. No .cfg file was inside the gz or on the backup host so I was glad I had a copy. I am imagining that the .serv file contains the .cfg file for xen and allows restoring on hosts that do not have the original .cfg file.

The restore from backup log has worked in the past (nothing has changed on my end). I know that it worked great on 6.1.

Anyways, I it is a known issue and an easy fix since it is pretty darn important : ).

Take it easy guys,

~Jeremy

Status: 
Closed (fixed)

Comments

Submitted by JamieCameron on Fri, 04/20/2012 - 11:24

Looks like this is a bug that only effects FTP backups, and only impacts the .serv file that contains meta-information about the VM which is used if it is deleted and needs to be re-created.

I am working on a fix for this now. The work-around is to backup and restore using SSH if you can, rather than FTP. Is that possible?

Submitted by xtremeservices on Fri, 04/20/2012 - 11:39

Hi Jamie,

Thx for looking into this, I will change to SSH for now.

~Jeremy

Submitted by JamieCameron on Fri, 04/20/2012 - 12:46

I have implemented a fix for this, which will be in the next Cloudmin release.

Submitted by xtremeservices on Fri, 04/20/2012 - 13:53

Getting new errors with SSH Backups (See attached)

I have a theory, I know that with Cloudmin 6.2 swap img files are not backed up by default. I know that With the 2 servers that are having an issue, I had manually setup their swap drives to not backup (prior to 6.2). For whatever reason the Cloudmin Backup is trying to tar those swap drives in the backup and I can't tell it not to with Cloudmin 6.2.

~Jeremy

Submitted by JamieCameron on Fri, 04/20/2012 - 15:14

That looks like a separate issue .. does that .gz.2 file mentioned in the error exist on the destination system, and if so how large is it?

Submitted by xtremeservices on Fri, 04/20/2012 - 15:17

Hi Jamie,

No, it does not get created on the destination server.

Submitted by JamieCameron on Fri, 04/20/2012 - 15:26

Ok, I see the cause now - that's another bug :-(

Fortunately it isn't too harmful, as you are only missing the backup for a disk that would have been skipped anyway.

Submitted by xtremeservices on Fri, 04/20/2012 - 15:27

Please see the VPS-BACKUP-FILES.PNG for the files that are being created on the Destination host.

Both of the systems failing have 3 drives (see attached)

The 3rd Partition of 2GB is the Swap (Virtual Memory - See attached)

My Theory is that since I had manually set these VPS systems not to backup the Swap drives, that somehow when that became the defacto method in 6.2 it is causing a reverse affect (Why not bring back that yes/no option? And set to No by default).

~Jeremy

Submitted by xtremeservices on Fri, 04/20/2012 - 15:30

Awesome, glad you found it.

When is the next release scheduled? Can you share the fix with me to implement?

Please mark this as fixed again.

~Jeremy

Submitted by JamieCameron on Fri, 04/20/2012 - 15:30

That confirms my suspicions - the "missing" file is the one ending in .2 , which is for the 3rd (swap) disk.

I can send you a fix for these issues if you like?

Submitted by JamieCameron on Fri, 04/20/2012 - 16:55

Ok, I have sent you an update .deb file.

Submitted by xtremeservices on Wed, 04/25/2012 - 14:45

Hi Jamie,

The final fix worked like a charm, please mark this as fixed.

~Jeremy

Submitted by Issues on Wed, 05/09/2012 - 15:18

Automatically closed -- issue fixed for 2 weeks with no activity.