5GB+ Backups Fail to Cloud Files

Hi Guys,

I just updated Virtualmin to 3.95PRO and tried testing out the backup of Virtual Servers larger than 5GB to Cloud Files. Well, apparently this isn't supported or something else went wrong. This is the output I received in the log (there was PAGES of the gobligook so this is just a snippet) :

Starting backup of 5 domains to andromeda/full/%Y-%m-%d-%a in Rackspace container Backups ..

Creating backup for virtual server vital-soft.com ..

    Copying virtual server configuration ..
    .. done

    Backing up Cron jobs ..
    .. none defined.

    Copying records in DNS domain ..
    .. done

    Saving mail aliases ..
    .. done

    Saving mail and FTP users ..
    .. done

    Backing up mail and FTP user Cron jobs ..
    .. done

    Copying Apache virtual host configuration ..
    .. done

    Copying Apache log files ..
    .. done

    Copying Webalizer configuration files ..
    .. done

    Copying SSL Apache virtual host configuration and certificate ..
    .. done

    Copying Logrotate configuration ..
    .. done

    Dumping MySQL database vital_soft_com ..
    .. done

    Copying Procmail and SpamAssassin configuration files ..
    .. done

    Backing up Webmin ACL files ..
    .. done

    Creating backup of Mailman mailing lists ..
    .. done

    Creating TAR file of home directory ..
    .. done

    Uploading archive to Rackspace's Cloud Files service ..
    .. upload failed! Upload failed at 855638016 : Source file /ê¯Sð„ bE9 >¨#º"žtÐ-øüÅ_,·=ƒ†)—nëÙQ£ý5º!|ÏKÛžÄÎ ¦U‹î–ÊõÕ˜p»ŠN†¬aâÒ +†ÈÇ3@ºHY”*Ô’rÕ@$Do¯:ñ‘Œ/:‡3åp´ÙKÁÜôGÒU9lÂ+M£ý!w¸Bfñ7 ´0îÆ(~¿ÖP[©á¢ñÔKU¹âï'^·)µ«‰Ák!‰kd×ë:ë¶2Ïó$ ]¹q½¶áŽÌ!ÜnOúqCÅPY†ð@b¬“5c.Sä ^ñGƒ§¼ø}þ÷²aÄùVŸsõüuÆÏÛ:»_4èv㬲¼l–Žv %ìEbñàªFi"'àjÛ'e½Úˆ|/¹øßÀà з£ù¦ŽAŽ! næ*ÎL(wQhƒâ!Ø$6è``PsðŠ¡| ,K˜a¼‘3c3*¡£¬ŸÌ\,?æœÃ†Œr´R!ÈÂ+rxh|m ß-oñÚ«*›•oÌ|ó­V¸düÛ1â¹´AmzÝäûpï ëWã’çVXhõojX)vòcëÄ!bbŒŸ=·z½;ZiMïÕwf!x

A full backup locally to the /home directory works great.

~Jeremy

Status: 
Closed (fixed)

Comments

That's odd, as 3.95 is supposed to include the change to support 5 GB files.

Do you see any error when uploading a 2-3 GB file?

Actually, I see the cause of this bug, and will include a fix in the 3.96 Virtualmin release.

Let me know if you'd like to get a patch sooner though.

Hi Jamie,

I am very glad to hear that the issue is resolved. Please email me what to patch so I can try it out.

Thank you,

~Jeremy

Ok, just save the attached file as /usr/share/webmin/virtual-server/rs-lib.pl and then run /etc/webmin/restart

Thx Jamie, I will test it out and get back to you.

Hi Jamie,

I ran a full backup 2 times this evening with the patch file and it failed both times with the following message :

    Uploading archive to Rackspace's Cloud Files service ..
    .. upload failed! Upload failed at 3399483392 : Invalid HTTP response :

.. completed in 1 hours, 28:0 minutes

Here is a screen shot of what I am seeing in the Cloudfiles front end. It appears like each section is 1MB.

Those 1MB files are expected - Cloud files requires that large files be uploaded as a series of parts, which are then automatically combined.

Does the backup upload always fail at the same place (3399483392) ?

Hi Jamie,

File segments below 100-200MB are NOT recommended. 1MB File segments are impossible to manage if you wanted to manually copy the backup offsite or purge them. Would it be possible to incorporate the Max file segment size of 5GB as mentioned in the following :

http://www.rackspace.com/knowledge_center/frequently-asked-question/does...

Here is the error from the first failure in the Full backup yesterday, it is earlier in the backup.

Uploading archive to Rackspace's Cloud Files service ..
    .. upload failed! Upload failed at 2814377984 : Invalid HTTP response :

~Jeremy

Ok, attached is a further update to that same file which uploads in 200M chunks, and should fix the issue you have been seeing.

Great news Jamie,

The latest patchwork did not fail.

Is it possible to make the Segment size an option in the Virtualmin Configuration -> Backup and restore area?

Hope you have a great day,

~Jeremy

Sure, I will make that customizable in the next release.

Automatically closed -- issue fixed for 2 weeks with no activity.