Unable to restore S3 backups

Submitted by sanjoseweb on Sat, 12/15/2012 - 01:30 Pro Licensee

I'm trying to restore S3 backups to a move sites to a new server and I get this message:

Restore failed : The specified source does not contain any domains, or is not a Virtualmin backup

Even if I try and restore to the server that created the backup, I get this message.

Status: 
Closed (fixed)

Comments

Submitted by JamieCameron on Sat, 12/15/2012 - 12:04

Are you trying to restore from an S3 bucket, a sub-directory of the bucket, or a single file?

Submitted by sanjoseweb on Sat, 12/15/2012 - 14:31 Pro Licensee

A sub-directory of a bucket or individual backup within the subdirectory

Submitted by JamieCameron on Sat, 12/15/2012 - 16:30

What are you entering in the "Bucket and path" field on the restore form?

Submitted by sanjoseweb on Sat, 12/15/2012 - 17:59 Pro Licensee

When trying to restore server1 backups on server2 I'm using

bucketname/path-121211

or

bucketname/path-121211/domain.com.tar.gz

Neither one works.

If I try and restore the server1 backup on server1 by click on the backup log on server1 it also fails with the same message.

Submitted by JamieCameron on Sat, 12/15/2012 - 19:42

That looks OK...

One thing you could try is checking if Virtualmin can list that S3 directory using the command-line tools. SSH into your system as root and run :

virtualmin list-s3-files --bucket bucketname --access-key XXX --secret-key YYYY

replacing the bucket name, access key and secret key with the correct values.

Submitted by sanjoseweb on Sat, 12/15/2012 - 21:02 Pro Licensee

This lists the folders, but the filenames of the backups are truncated. Could this be the problem?

Submitted by JamieCameron on Sat, 12/15/2012 - 21:29

No, that's just how the listing is displayed by default. You can use the parameter --multiline to show a more detailed list, including permissions (which might be preventing your backups from being read).

Submitted by sanjoseweb on Sat, 12/15/2012 - 21:37 Pro Licensee

I noticed that it only lists 1000 files and is not listing my newest folders and backups. Could that be a problem? I tried to restore an older backup that was in the listing and that one works.

Submitted by JamieCameron on Sat, 12/15/2012 - 21:59

That could be a problem. Can you see the newer files with other tools, like Amazon's s3 console?

Submitted by sanjoseweb on Sat, 12/15/2012 - 22:10 Pro Licensee

Yes, I see the newer files in the S3 console and I can retrieve them with s3cmd.

Submitted by JamieCameron on Sat, 12/15/2012 - 22:23

You might want to try moving the files you want to restore to another bucket, and then having Virtualmin restore from that.

Submitted by sanjoseweb on Sat, 12/15/2012 - 23:53 Pro Licensee

Yes, that works.

Submitted by JamieCameron on Sun, 12/16/2012 - 01:32

Ok, I found the cause of this - the S3 API by default only returns around 1000 files in a bucket. To get the full list of files, multiple API calls are needed, which Virtualmin wasn't doing.

This will be fixed in the 3.98 release.

Submitted by Issues on Mon, 12/31/2012 - 23:58

Automatically closed -- issue fixed for 2 weeks with no activity.