Hi,
For the most part the S3 backup system has been running well, but recently we've seen some errors in the email reports. First of all a quick overview of what we are doing. We use the file per virtualmin server in the new format, we make use of the reduced redundancy option, along with the strtime folder paths.
We run a full backup once a week with the file patch full-%Y-%m-%d and on all the other days an incremental backup with the path incremental-%Y-%m-%d. On the full backup we define that backups older than 2 days are deleted, and for the incrementals, this is 7 days.
All in all the backups work, but each week on the full backup we see that a file upload to Amazon fails. It's only ever the full backup and only ever the upload process. I don't have the exact error message to hand right now as a backup is currently running, however it is to do with the multi-part uploads and a transfer error is reported. Any ideas? I will of course post an exact error message as soon as it occurs on the current backup.
Thanks.
Ok, after a few weeks of testing we are still having problems with the Amazon S3 upload. As explained we run a full backup once a week and between that we have a daily incremental. It doesn't seem to matter on the domain/host that is being backed up or if it's the full or incremental job, we get an error like this:
Creating backup for virtual server blanked-out-domain.com .. Copying virtual server configuration .. .. done
.. completed in 1 minutes, 33 seconds
This is totally random, sometimes the same domain will backup fine, other times it'll fail. Any ideas or things to look at. We really need to have the regular backups working flawlessly. This is the last step of out evaluation before we go pro with Virtualmin. :)
Thanks.
Howdy,
Does there seem to be any correlation to file size? Is it backups that are over a certain size that are failing?
-Eric
Hey,
There doesn't seem to be a correlation that I can see. For example the log I posted is for an incremental backup of a few hundred kilobytes. On other fails it can be a 1.8GB virtual server, from week to week that server might backup just fine, or fail. I cannot see a pattern right now.
Initially some of the S3 failures had a more descriptive error message, but in the last two weeks they are all as posted. Anywhere I can look specifically for issues?
Thanks.
Here we go, another incremental backuop fired this morning, and the same virtual server backed up perfectly with no config changes at all:
Creating backup for virtual server domain.com ..
Copying virtual server configuration ..
.. done
Backing up Cron jobs ..
.. none defined.
Saving mail aliases ..
.. done
Saving mail and FTP users ..
.. done
Backing up mail and FTP user Cron jobs ..
.. none to backup
Copying Apache virtual host configuration ..
.. done
Copying Apache log files ..
.. done
Copying SSL Apache virtual host configuration and certificate ..
.. done
Copying Logrotate configuration ..
.. done
Dumping MySQL database blah ..
.. done
Copying Procmail and SpamAssassin configuration files ..
.. done
Backing up Webmin ACL files ..
.. done
Backing up AWstats configuration file ..
.. done
Creating incremental TAR file of home directory ..
.. done
Uploading archive to Amazon's S3 service ..
.. done
.. completed in 1 minutes, 4 seconds
So, the question is what could cause seemingly random upload errors to S3?
Another week, another failed backup, this time a totally different virtual server, but the same error:
Uploading archive to Amazon's S3 service .... upload failed! Upload failed :
I can't seem to find anything that is causing the problem... :-/
I'm seeing the exact same issue. Occasionally an upload to S3 will fail for no apparent reason, different domains, different sizes, and not every day.
Also there seems to be a bug in the error notification email that gets sent out. The email tells me that every domain in the backup failed, but reading down the list of backup actions all but one have completed successfully.
Chris
There is indeed a bug in the backup notification email - it lists all domains as failed in some cases, when really only one failed. I'll fix this in the next Virtualmin release.
As for the S3 failures, one thing you can try is increasing the number of re-tries Virtualmin does for each upload. This is set at System Settings -> Virtualmin Configuration -> Backup and Restore.
''