Hi,
I've been using a scheduled backup to Amazon S3 for years with no problems. I backup several domains (1 file per domain). All domains are moderately sized (couple of GB at max), but one domain is large, about 50GB. I will call it LARGEDOMAIN.com hereon.
3 weeks ago I first received a "failed" email notification due to no space left. Since then I created more space on my local drive. However, the next time that the backup was supposed to run, it failed again (no files were sent to S3). I didn't receive any email at all, but I found the following in "Backup Logs". The same LARGEDOMAIN.com has failed again, but for a different reason (which I don't understand what it means). Also, the domain that was supposed to be backed up after it (domain2.com), also failed for another reason. This is the log:
Creating backup for virtual server domain1.com ..
Copying virtual server configuration ..
.. done
Backing up Cron jobs ..
.. none defined.
Saving mail aliases ..
.. done
Saving mail and FTP users ..
.. done
Backing up mail and FTP user Cron jobs ..
.. none to backup
Copying Apache virtual host configuration ..
.. done
Copying Apache log files ..
.. done
Copying Logrotate configuration ..
.. done
Dumping MySQL database storejrl ..
.. done
Backing up Webmin ACL files ..
.. done
Backing up AWstats configuration file ..
.. done
Creating TAR file of home directory ..
.. done
Uploading archive to Amazon's S3 service ..
.. done
.. completed in 2 minutes, 4 seconds
Creating backup for virtual server LARGEDOMAIN.com ..
Copying virtual server configuration ..
.. done
Backing up Cron jobs ..
.. none defined.
Saving mail aliases ..
.. done
Saving mail and FTP users ..
.. done
Backing up mail and FTP user Cron jobs ..
.. none to backup
Copying Apache virtual host configuration ..
.. done
Copying Apache log files ..
.. done
Copying SSL Apache virtual host configuration and certificate ..
.. done
Copying Logrotate configuration ..
.. done
Dumping MySQL database XXX ..
.. done
Backing up Webmin ACL files ..
.. done
Backing up AWstats configuration file ..
.. done
Creating TAR file of home directory ..
.. done
Uploading archive to Amazon's S3 service ..
.. upload failed! Part 1 failed at 0 : Read failed for 5242880 : Bad file descriptor
.. completed in 5 hours, 25:23 minutes
Creating backup for virtual server domain3.com ..
Copying virtual server configuration ..
Failed to open /home/tpsmy/.backup/domain3.com_virtualmin_scripts for writing : No such file or directory at ../web-lib-funcs.pl line 1397.
At the next time that the backup was supposed to run again, there was no email notification and no logs at all created at "Backup Logs". And again, no files were sent to S3. However, while the backup was running, I could see it at "Running Backups", but once it crashed, logs were nowhere to be found. There is also a file left behind from this backup attempt, at: /backup/.webmin/17587-/ there is a LARGEDOMAIN.tar.gz file which is 49GB (roughly matches the size of the whole LARGEDOMAIN).
Something that I also tried is to increase the chunk size to 15mb since I read somewhere that S3 can only handle 10000 parts - the default chunk size being 5mb, I thought I was appoaching the limit for my 50GB domain. But it didn't help.
Could you please advise what is happening and how do I fix this?
Can you run the backup to a different location...like Onedrive (i also have office 365 so often use onedrive) or another local disk on your pc?
https://ajecreative.com.au
I was able to perform an s3 full backup on 2017/11/12, however the following schedule incremental s3 backups have failed every day since as OP describes. There are no logs in Backup Logs. When I perform a manual backup from the Scheduled Backups page, virtualmin hangs with the message: 'Starting backup of 5 domains to backup_incremental/%Y-%m-%d-%H%M in Amazon S3 bucket bucket.name ..' and no status on the Running Backups page.
I was able to successfully backup locally with no issue today, but s3 backup schedules and manual backups are broken.
The backup eventually fails with: "Backup failed : Failed to open /home/[virtualhost]/.backup/[virtualhost].net_virtualmin_scripts for writing : No such file or directory"