Amazon S3 backup fails when region is not US Standard

Submitted by ucaqld on Wed, 01/30/2013 - 23:21 Pro Licensee

When trying to backup to an Amazon S3 bucket that is created in regions other than US Standard (we have tried Oregon and Sydney) through the virtualmin backup module, we get the following error (bucket name replaced with xxxx's):

File does not exist: Can't connect to xxxxx.xxxxxx.xxxxx.s3.amazonaws.com:443 (certificate verify failed)

LWP::Protocol::https::Socket: SSL connect attempt failed with unknown errorerror:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed at /usr/share/perl5/LWP/Protocol/http.pm line 51. at S3/ListBucketResponse.pm line 26

If we create an identical bucket in the US standard region, the backup works correctly.

Status: 
Active

Comments

Submitted by JamieCameron on Wed, 01/30/2013 - 23:35

Does your bucket name perhaps have dots in it? That is going to cause problems due to the way Amazon redirects requests for non-US buckets. For example, if your bucket is named foo.bar you may get directed to the API endpoint https://foo.bar.s3.amazonaws.com by the S3 server. However, if Amazon only has an SSL cert for *.s3.amazonaws.com , this hostname will not match the cert and you'll get an error like the one you saw.

The only work-around is to have a bucket without dots in the name. Does that work for you?

Submitted by ucaqld on Thu, 01/31/2013 - 00:07 Pro Licensee

Hi Jamie,

Ah ok.... yes it does have dots in it. Changing the bucket name to not include dots should be ok - I will test and confirm that it works.

Submitted by ucaqld on Thu, 01/31/2013 - 02:01 Pro Licensee

Hi Jamie,

As a follow up, taking out the dots in the bucket name has resolved the issue.

Thanks for the quick response.

Submitted by Issues on Thu, 02/14/2013 - 10:46

Automatically closed -- issue fixed for 2 weeks with no activity.

Submitted by djgillard on Sat, 10/24/2015 - 16:16

This is still a bug in 4.18gpl no validation to prevent the use of a bucket with dots in - this just fails. I believe this either needs to be validated against or the at S3/ListBucketResponse.pm file updated to handle buckets with dots in

Submitted by JamieCameron on Sat, 10/24/2015 - 19:14

The 5.0 Virtualmin release will fix this.

Submitted by cento on Sun, 03/19/2017 - 19:28

Removing the dots from bucket names resolved the error on my machine

Operating system Ubuntu Linux 16.04.2 Webmin version 1.831 Virtualmin version 5.07

Submitted by colech on Thu, 01/10/2019 - 21:12

Does the dot issue still exist in Webmin 1.900? I'm not having trouble doing backup to S3 but am having issues restoring from S3 at DigitalOcean and have a dot in my object like...

bucket-name/object.1/somedomain.com.tar.gz

I have tried both the Virtualmin GUI method as well as the Virtualmin command line like this...

virtualmin restore-domain --all-features --all-virtualmin --source s3://ACCESSKEY:PRIVATEKEY@bucket-name/object.1/somedomain.com.tar.gz

Submitted by JamieCameron on Thu, 01/10/2019 - 22:48

@colech - what's the exact error message that you are getting?

Submitted by JamieCameron on Fri, 01/11/2019 - 21:33

I just did some testing with the latest versions of Virtualmin and Webmin and a bucket named foo.bar.jamie in the Singapore S3 region, and was able to backup to it just fine.

Submitted by init-s on Sun, 06/07/2020 - 09:30

Unfortunately, I still have this issue. I cannot backup or list content and this is the error.

HTTP/1.0 500 Perl execution failed Server: MiniServ/1.942 Date: Sun, 7 Jun 2020 14:23:31 GMT Content-type: text/html; Charset=iso-8859-1 Connection: close Error - Perl execution failed File does not exist: Can't connect to the-name.domain.sub.domain.it:443 (certificate verify failed) LWP::Protocol::https::Socket: SSL connect attempt failed with unknown error error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed at /usr/share/perl5/LWP/Protocol/http.pm line 41. at S3/ListBucketResponse.pm line 26.

S3 account key UGITFIYTFIYTFTYFI Bucket name the-name.domain.sub.domain.it Storage location eu-central-1

This is what I can check on DNS side:

the-name.domain.sub.domain.it.s3.amazonaws.com Server: 127.0.0.1 Address: 127.0.0.1#53

Non-authoritative answer: the-name.domain.sub.domain.it.s3.amazonaws.com canonical name = s3-1-w.amazonaws.com. Name: s3-1-w.amazonaws.com Address: 52.216.161.211

Submitted by JamieCameron on Sun, 06/07/2020 - 13:51

Make sure you have the aws CLI command installed on your system, it provides a more reliable client for the S3 API.

Submitted by init-s on Mon, 06/08/2020 - 02:06

Hi, thanks, installing the AWS cli solved the problem.

Submitted by colech on Mon, 02/01/2021 - 13:38

JetBackup 5 is supporting a lot of new admin panels... don't think they have done Virtualmin yet but if they do it's a pretty affordable and very robust backup solution. I'd keep checking in on it and I bet they will support it eventually as that's where they are heading.