Submitted by xkeywee on Sun, 03/01/2015 - 17:11
Hi
We are getting random backup fails, never the same virtual server and not everyday and not always on the same server.
Getting
Uploading archive to Amazon's S3 service ..
.. upload failed! Empty response to HTTP request. Headers were :
.. completed in 9 minutes, 47 seconds
Have set "Number of times to re-try FTP or S3 uploads" to 10 as per other Issues and that reduced the number of fails but has not completely resolved the issue.
Cheers Greg
Status:
Needs work
Comments
Submitted by JamieCameron on Sun, 03/01/2015 - 18:28 Comment #1
Are you backing up to a bucket in a region close to your Virtualmin system?
Submitted by xkeywee on Sun, 03/01/2015 - 19:13 Comment #2
Hi Jamie
Yes the EC2 Instances we are backing up are in Sydney as the S3 Bucket is also in Sydney.
Cheers Greg
Submitted by JamieCameron on Sun, 03/01/2015 - 20:46 Comment #3
This is unfortunately a problem that a few other users have reported, and for which we haven't found a solution yet - sometimes the S3 service simply fails an upload with zero information sent back in response :-(
I am hoping that an update to use a newer S3 client library in Virtualmin which is currently underway will fix the issue though.
Submitted by xkeywee on Sun, 03/01/2015 - 20:54 Comment #4
Thanks Jamie
We will wait for the update, luckily it seems to pick different Virtual Servers each day to fail on so we are getting on average 6 out of our 7 daily backups for each one.
Cheers Greg
Submitted by JamieCameron on Sun, 03/08/2015 - 19:53 Comment #5
Ok, the next release of Virtualmin (4.16) will completely work around this problem by using the AWS command line tool, if installed. See http://aws.amazon.com/cli/ for instructions on how to download it.
Submitted by vbgscloud on Mon, 03/09/2015 - 18:00 Comment #6
Jamie helped me out with the same issue. I had my attempts to contact AWS set by default to 3. Since upping it to 12, I have had 0 fails, I used to get multiple fails a day...
"You may be seeing an interruption due to a transient network error - one possible work around is to increase the number of times Virtualmin re-tries an upload, at System Settings -> Virtualmin Configuration -> Backup and restore -> Number of times to re-try FTP or S3 uploads."
Submitted by Issues on Mon, 03/23/2015 - 18:00 Comment #7
Automatically closed -- issue fixed for 2 weeks with no activity.
Submitted by Issues on Mon, 03/23/2015 - 18:00 Comment #8
Automatically closed -- issue fixed for 2 weeks with no activity.
Submitted by acosonic on Wed, 01/08/2020 - 05:42 Comment #9
You should add awscli installation to install.sh script to avoid problems with s3
Once installed, all works...
Did you install it using
pip3 install awsclicommand?Actually it works as well with
yum install awscli(CentOS 7 and 8), and on Debian and Ubuntu worksapt-get install awscli;Joe, could we add this to the
install.shscript as a dependency?Submitted by acosonic on Wed, 01/08/2020 - 08:10 Comment #12
Yes, exactly I've used apt install awscli