VPS - ftp backup failure

Submitted by rulez22 on Fri, 01/04/2013 - 00:43

Centos 5.8 (64bit) (GPL) on VPS installed have scheduled backup for all domains (one file per domain) on to external ftp address it does not work throwing this (backup takes around 30 minutes due to domains folder sizes, logs etc):

"Uploading archive to FTP server ip.ip.ip.ip .. .. upload failed! Failed to read reply to PASV"

where is the problem? Virtualmin Settings backup works on to the same ftp with no problem.

Status: 
Closed (fixed)

Comments

Submitted by JamieCameron on Fri, 01/04/2013 - 02:00

You might want to check that both the FTP and FTP-data ports (21 and 20) are open on any firewalls between the Virtualmin system and the back destination.

Also, if possible check the logs on the destination to see if any error is reported by the FTP server when you try a backup..

Submitted by rulez22 on Fri, 01/04/2013 - 02:43

destination FTP has firewall disabled (that's why it backs up virtualmin settings without a problem)
Webmin has nothing in IPtables - it's all open as well..
Also according to logs and backup filesystem it DOES back up some small domain folders, but struggle to save big ones

FTP log reads like this

2013-01-03 21:23:03 - vps [000076] [ip.ip.ip.ip] - INFO: logged in.
2013-01-03 21:23:03 - vps [000076] [ip.ip.ip.ip] - 230 User vps logged in.
2013-01-03 21:23:03 - vps [000076] [ip.ip.ip.ip] - TYPE I
2013-01-03 21:23:03 - vps [000076] [ip.ip.ip.ip] - 200 Type set to I.
2013-01-03 21:23:04 - vps [000076] [ip.ip.ip.ip] - PASV
2013-01-03 21:23:04 - vps [000076] [ip.ip.ip.ip] - 227 Entering Passive Mode (192,168,1,65,117,104)
2013-01-03 21:23:06 - vps [000076] [ip.ip.ip.ip] - INFO: user disconnected gracefully. (00:00:02)

so it looks like it's struggling with something, timeout, CPU/memory overload (VPS this is)?
================
but just before that it has no problem uploading small domains (small in size):

2013-01-03 21:23:02 - vps [000075] [ip.ip.ip.ip] - INFO: logged in.
2013-01-03 21:23:02 - vps [000075] [ip.ip.ip.ip] - 230 User vps logged in.
2013-01-03 21:23:02 - vps [000075] [ip.ip.ip.ip] - TYPE I
2013-01-03 21:23:02 - vps [000075] [ip.ip.ip.ip] - 200 Type set to I.
2013-01-03 21:23:02 - vps [000075] [ip.ip.ip.ip] - PASV
2013-01-03 21:23:02 - vps [000075] [ip.ip.ip.ip] - 227 Entering Passive Mode (192,168,1,65,117,48)
2013-01-03 21:23:02 - vps [000075] [ip.ip.ip.ip] - STOR /centos_webmin/backup_domains/04-01-2013/domain.com.tar.gz.dom
2013-01-03 21:23:02 - vps [000075] [ip.ip.ip.ip] - INFO: request ALLOWED to upload '\centos_webmin\backup_domains\04-01-2013\domain.com.tar.gz.dom' into 'Z:\FTP\!VPS\backup\'.
2013-01-03 21:23:02 - vps [000075] [ip.ip.ip.ip] - 150 Data connection accepted from ip.ip.ip.ip:53214; transfer starting for domain.com.tar.gz.dom.
2013-01-03 21:23:02 - vps [000075] [ip.ip.ip.ip] - INFO: upload started of '\centos_webmin\backup_domains\04-01-2013\domain.com.tar.gz.dom' into 'Z:\FTP\!VPS\backup\'.
2013-01-03 21:23:03 - vps [000075] [ip.ip.ip.ip] - 226 File received ok.
2013-01-03 21:23:03 - vps [000075] [ip.ip.ip.ip] - INFO: upload finished of '\centos_webmin\backup_domains\04-01-2013\domain.com.tar.gz.dom' into 'Z:\FTP\!VPS\backup\'. Stats: (00:00:01 - 6.318 KB - 6.318 KBytes/s).
2013-01-03 21:23:03 - vps [000075] [ip.ip.ip.ip] - QUIT
2013-01-03 21:23:03 - vps [000075] [ip.ip.ip.ip] - 221 Goodbye.

To remind again, this is VPS, which I have no control really. This is 1024 package
Guaranteed 1024Mb RAM and 1536Mb "busrtable"
6-8 cores equal share
OpenVZ

Submitted by andreychek on Fri, 01/04/2013 - 10:25

Can you paste in the contents of your /proc/user_beancounters file? I'm curious if it shows any limits that have been reached.

It's possible that one of the OpenVZ bandwidth limits is being triggered.

Submitted by rulez22 on Fri, 01/04/2013 - 12:34


Version: 2.5
uid resource held maxheld barrier limit failcnt
74217: kmemsize 26429647 55766748 2147483646 2147483646 0
lockedpages 9731 10266 999999 999999 0
privvmpages 244829 338932 393216 393216 0
shmpages 3588 3604 262144 262144 0
dummy 0 0 0 0 0
numproc 98 212 999999 999999 0
physpages 165069 225821 0 2147483647 0
vmguarpages 0 0 262144 2147483647 0
oomguarpages 165159 225908 262144 2147483647 0
numtcpsock 29 236 7999992 7999992 0
numflock 9 107 999999 999999 0
numpty 2 2 500000 500000 0
numsiginfo 0 8 999999 999999 0
tcpsndbuf 518696 3708504 214748160 396774400 0
tcprcvbuf 475136 3565760 214748160 396774400 0
othersockbuf 313536 1519024 214748160 396774400 0
dgramrcvbuf 0 279136 214748160 396774400 0
numothersock 175 799 7999992 7999992 0
dcachesize 985011 1554045 2147483646 2147483646 0
numfile 3942 10520 23999976 23999976 0
dummy 0 0 0 0 0
dummy 0 0 0 0 0
dummy 0 0 0 0 0
numiptent 41 41 999999 999999 0

Submitted by JamieCameron on Fri, 01/04/2013 - 13:16

Does the backup always fail on the same domain, or after the same number of domains?

It looks a lot to me like the temporary port allocated by the FTP server for the data transfer is blocked.

Submitted by rulez22 on Sun, 01/06/2013 - 09:55

ok, here is some progress.
In scheduled backups I've selected "transfer each domain after backup"
It uploaded more domains now
those which fail are strangely fail with same time "upload finished in 17 minutes and .. seconds" (coincidence? i got about 3 of those)
largest domain uploaded (tar.gz) is 52Mb in size
so it's either size or timeout of some kind that doesn't allow other domains to be uploaded?
destination FTP server has disconnect timeout disabled all together.

Submitted by andreychek on Sun, 01/06/2013 - 10:40

There shouldn't be any sort of size or time limits on your Virtualmin system that would cause a FTP backup to fail.

Could you try a different backup type?

For example, often SSH-based backups will run into less problems... plus they're more secure. Is using SSH backups an option?

Submitted by rulez22 on Sun, 01/06/2013 - 15:59

It would be no problem if you could explain how to make it easy and automated (like daily backup 1 domain per file with last 10 versions of backup folder to keep). I don't know how to do it other than manually. Could you help? Thanks a lot.

Submitted by andreychek on Sun, 01/06/2013 - 16:07

The delete backup feature works the same for FTP as it does with SSH.

You can see the option for that in Backup and Restore -> Scheduled Backup -> BACKUP_NAME, and review the option "Delete old backups".

You can click the "Delete old backups" link to get help text on how to use it.

Submitted by rulez22 on Sun, 01/06/2013 - 16:11

oh i see, but this means i have to have SSH server to back it up to. External FTP is on Windows and I'm not sure how to make a SSH server on win machine?

Submitted by andreychek on Sun, 01/06/2013 - 16:27

Although it is possible to setup an SSH service on Windows, I'm not familiar with the procedure to do that, and it's not a standard Windows service.

The FTP problem you're seeing doesn't appear to be a Virtualmin bug though; we haven't been able to reproduce it, and it hasn't been reported by anyone else.

Unfortunately, that sounds like an issue with either the Windows FTP service, Windows itself, or maybe a device between your Linux and Windows server -- a router could potentially cause that, perhaps due to a restrictive timeout.

Submitted by rulez22 on Fri, 01/11/2013 - 22:56

SOLVED! problem was in destination ftp server

Submitted by Issues on Fri, 01/25/2013 - 23:08

Automatically closed -- issue fixed for 2 weeks with no activity.