Dropbox backups failing with output "v1_retired"

Dropbox backups are failing with output message "v1_retired". It looks like Dropbox has retired V1 of their API. See:

https://www.dropbox.com/developers/reference/migration-guide

As a result, VirtalminPro cloud backups to Dropbox are failing. Thanks, stevep

Status: 
Closed (fixed)

Comments

jimr's picture
Submitted by jimr on Fri, 09/29/2017 - 09:52 Pro Licensee

I'm getting this too ... mind you there appears to a lot of work to do

Thanks for the heads up guys!

Yeah this does indeed appear to be a bug (or perhaps a missing feature). I just ran into this last night on my own personal server.

Damn, I guess dropbox changed their API without telling us :-( We'll work on a fix ..

jimr's picture
Submitted by jimr on Fri, 09/29/2017 - 19:48 Pro Licensee

Dropbox did notify that v1 api was retiring some while a go looking at the changes you may have some work to do let's hope you can do it quite quickly

We started getting this too as of midnight Friday )09/29) in regards to our daily backups. Dropbox is our primary backup source as space is cheap so I hope Virtualmin is quickly updated to use the new Dropbox API.

Good news - support for the new Dropbox API has been implemented, and will be included in the upcoming 6.1 Virtualmin release.

Status: Active » Fixed

Thanks for getting this done quickly. Is there any ETA on when we can get our hands on it?

Couple of days at most

jimr's picture
Submitted by jimr on Wed, 10/04/2017 - 08:21 Pro Licensee

When using the new code I get the following error :- .. upload failed! HTTP/1.1 409 path/malformed_path/

when the backup module tries to 'upload' the data to dropbox is there a change required in the backup job settings ?

@jimr - did you upgrade to 6.01 already? If so, what destination path are you backing up to?

jimr's picture
Submitted by jimr on Thu, 10/05/2017 - 07:20 Pro Licensee

Webmin handled the upgrade to 6.01 so yes I am running 6.01 with the original version the dropbox path required was something like webmin//%y.%m.%d with v2 the path required is now webmin/%y.%m.%d . I made the change to the schedule to reflect the new path however domains greater than 1GB backup fail with the following :- upload failed! HTTP/1.1 409 incorrect_offset/... (performed 3 attempts) all the smaller domains backup correctly however I can see the archives in the correct dropbox folder for the larger domains so perhaps there is an issue with the V2 api that is returning and error ?

jimr's picture
Submitted by jimr on Thu, 10/05/2017 - 11:24 Pro Licensee

I notice that the 1gb archive is about 2kb on drop box so the previous point is valid

I think there are still a few issues around the new Dropbox API calls. My backups get copied up to Dropbox after updating to 6.01, however, the cleanup of old backups does not function:

"servers backed up successfully, 0 had errors. Backup is complete. Final size was 322.29 MB.

Deleting backups from web2_%d-%m-%Y in Dropbox folder vs_abvmp older than 10 days .. .. failed to list Dropbox files : Error in call to API function "files/list_folder": request body: path: 'vs_abvmp' did not match pattern '(/(.|[\r\n]))?|id:.|(ns:[0-9]+(/.*)?)'"

jimr's picture
Submitted by jimr on Fri, 10/06/2017 - 06:24 Pro Licensee

I looked on the dropbox website and found :- NOTES /files_put has a maximum file size limit of 150 MB and does not support uploads with chunked encoding. To upload larger files, use /chunked_upload instead.

as every domain backup that is smaller than 150mb is uploaded correctly & those bigger than 150mb fail this may shed some light on my problem

jimr's picture
Submitted by jimr on Fri, 10/06/2017 - 21:23 Pro Licensee

trying again I got :- Uploading archive to Dropbox .. .. upload failed! JSON decoding failed : malformed JSON string, neither array, object, number, string or atom, at character offset 0 (before "(end of string)") at /usr/share/webmin/virtual-server/pro/dropbox-lib.pl line 246. (performed 3 attempts)

Backup failed! See the progress output above for the reason why.

Deleting backups from %d.%m.%y in Dropbox folder noideersoftware older than 3 days .. .. failed to list Dropbox files : Error in call to API function "files/list_folder": request body: path: 'noideersoftware' did not match pattern '(/(.|[\r\n]))?|id:.|(ns:[0-9]+(/.*)?)'

That's odd, because Virtualmin does use the chunked upload feature for files above 100 MB. Let me see if I can re-produce this, and we will do a patch release with a fix for it and the purge issue.

jimr's picture
Submitted by jimr on Sat, 10/07/2017 - 07:58 Pro Licensee

I could give you access to one of the virtualmin servers if you like so you can see for yourself if that helps

this looks like what is happening to me too Jamie (see my other help ticket) - some upload, some do not.??

jimr's picture
Submitted by jimr on Wed, 10/11/2017 - 09:56 Pro Licensee

As of version 6.01-2 dropbox backups now appear to work. I forced a back up and all files were uploaded to dropbox however it was a good hour or so to transfer 3.5 GB of files I guess when the unattended back up will work .. I'll have to wait till 3:00 am to be sure .. I'll report back

jimr's picture
Submitted by jimr on Wed, 10/11/2017 - 10:12 Pro Licensee

just a thought .. does restore work ?

It does in our automated tests. Restores are less complex to support in the Dropbox API, as there's no special code needed for large downloads.

jimr's picture
Submitted by jimr on Thu, 10/12/2017 - 09:38 Pro Licensee

OK I can report that both servers backed up fine over night using the scheduled backup task to dropbox. I have tried to do a restore .. restoring from dropbox failed with the larger domains .. so I tried from a local file (upload to server) this also failed with a chrome error of 'connection reset' I'll try a restore from google see if that works

jimr's picture
Submitted by jimr on Wed, 10/25/2017 - 11:09 Pro Licensee

Hi Just tried to restore from dropbox .. this failed with the message 'Restore failed : The specified source is not a Virtualmin backup : Download incomplete' this applies if you either use the restore function or try to restore from a back up log .. I guess the need work title is still valid ;)

@jimr - how large was this backup you were attempting to restore?

I have not filed an item in the bug tracker regarding this issue, but have continued to test and no matter what Dropbox backup I attempt to restore, I always see the "Download incomplete" message. The backups range anywhere from 10 to 16 GB in size.

jimr's picture
Submitted by jimr on Thu, 10/26/2017 - 04:38 Pro Licensee

After finding out the restore doesn't work I have tried to restore backups of all sizes from 2mb up to 4.3gb

When restoring, are you entering the full path to a tar.gz file (like folder/domain.com.tar.gz), or just a directory on Dropbox?

jimr's picture
Submitted by jimr on Fri, 10/27/2017 - 05:17 Pro Licensee

I have tried both entering the location and the exact file neither work also trying to restore from the backup logs also fails. To date I have not manually uploaded the tar to the server from Dropbox and used the local file option to restore .. But I guess this option will work

Does the restore fail immediately, or does it take some time after the process starts for that "Download failed" message to appear?

jimr's picture
Submitted by jimr on Fri, 10/27/2017 - 17:49 Pro Licensee

I would guess about 3~5 seconds, using authentic the little red line across the top gets to about 50% of the screen width after you hit the "show what will be restored" button

Ok, so definitely not long enough to download a significant amount of a 10 GB backup.

If you SSH in as root and use the command virtualmin download-dropbox-file to fetch the same tar.gz from Dropbox to your system, does that work OK?

The GUI restore tool runs for about 15 seconds for me then fails with the "Download incomplete" message; running the download-dropbox-file command fails after even less time, only about 5 seconds.

jimr's picture
Submitted by jimr on Fri, 10/27/2017 - 18:02 Pro Licensee

command line fails with :- Unsupported file or mode >xx.tar.gz at WebminCore::../web-lib-funcs.pl line 2413

Make sure you enter a full path for the destination, like /tmp/domain.com.tar.gz

jimr's picture
Submitted by jimr on Tue, 10/31/2017 - 15:29 Pro Licensee

I ran :- virtualmin download-dropbox-file --file noideersoftware/27-10-17/ickleh.co.uk.tar.gz --dest /home/ickleh.co.uk.tar.gz and the response was ERROR: Download incomplete within about 5 seconds

How large is that noideersoftware/27-10-17/ickleh.co.uk.tar.gz file?

I have tried every permutation of every parameter on Backup Virtual Servers and Scheduled Backups page and I can not get Backup to work with Dropbox. I also used forward and backward slashes.

jimr's picture
Submitted by jimr on Thu, 11/02/2017 - 05:31 Pro Licensee

Ickleh. Archive size is about 68mb... I did try with larger archives but hit the same issue

As a test, I tried a few Dropbox uploads and downloads with a 522 MB file, but it worked fine.

Any chance we could login to your system to see what's going wrong here?

jimr's picture
Submitted by jimr on Fri, 11/03/2017 - 07:24 Pro Licensee

I have sent remote log in privs for you

Thanks, but I wasn't able to SSH in as root to the IP you granted access to. Perhaps root logins are blocked?

You can also send me login details at jcameron@virtualmin.com

Thanks for the mention, Jamie! How would one go about applying this patch immediately to the current installed version of Webmin? Or, since you already have my login details, would you mind applying it for me? Thanks!

jimr's picture
Submitted by jimr on Sat, 11/04/2017 - 22:24 Pro Licensee

I added the patch and it Works !!! so the last thing to do is to get the older archives removed e.g stuff older than X days then we will be back to a working system

That should happen automatically after the next backup runs.

jimr's picture
Submitted by jimr on Sun, 11/05/2017 - 18:19 Pro Licensee

I ran an automated backup @ 3:00 after I applied the patch ( I guess no bearing on file deletion from dropbox) and I got in the log 'Deleting backups from %d-%m-%y in Dropbox folder noideersoftware older than 3 days .. .. found 3 backups, but none were old enough' ... even though 1 backup was 4 days old ... is this a problem using %d-%m-%y rather than any other time format ?

jimr's picture
Submitted by jimr on Fri, 11/10/2017 - 20:11 Pro Licensee

I now have quite a few backups not deleted ... with the option backup->Schedule->Schedule and reporting->Command to run after backup I could code something that deletes whats needed until such time Virtualmin gets back on track within this module ?

The problem is likely that Virtualmin isn't parsing the backup date correctly from Dropbox.

If you SSH into your system as root and run :

virtualmin list-dropbox-files --path noideersoftware --multiline

does it show the correct timestamps?

jimr's picture
Submitted by jimr on Sun, 11/12/2017 - 15:12 Pro Licensee

The result from the above command is :-

ERROR: Error in call to API function "files/list_folder"
: request body: path: 'noideersoftware' did not match pattern '(/(.|[\r\n])*)?|id:.*|(ns:[0-9]+(/.*)?)'

but using this command :- virtualmin list-dropbox-files --path /noideersoftware --multiline works as expected ... note the / with the slash the following is returned:-

09-11-17
    Type: Directory
    Full path: /noideersoftware/09-11-17
10-11-17
    Type: Directory
    Full path: /noideersoftware/10-11-17
11-11-17
    Type: Directory
    Full path: /noideersoftware/11-11-17

Ok, it looks like there is a bug in the dropbox purging code for date-based directories like this. It will be fixed in the next release though ..

jimr's picture
Submitted by jimr on Sun, 11/12/2017 - 15:09 Pro Licensee

how far off is the next release ?

A few weeks, but if you need a fix sooner we can send you a beta.

jimr's picture
Submitted by jimr on Tue, 11/14/2017 - 14:31 Pro Licensee

well I would like to have it up and running as soon as ... if you point me to the patch I'll edit the local file(s) ... if it is a bit more far reaching just give me access to the full beta

I can send you a beta - just let me know which Linux distribution you're running there.

jimr's picture
Submitted by jimr on Wed, 11/15/2017 - 04:50 Pro Licensee

Hi Jamie I am using ubuntu 16.04 on one server and ubuntu 14.04 on the other

jimr's picture
Submitted by jimr on Wed, 12/06/2017 - 04:52 Pro Licensee

using webmin 6.02 I get

Deleting backups from %d-%m-%y in Dropbox folder noideersoftware older than 3 days ..
Deleting file 03-12-17, which is 3 days old ..
.. deletion failed : HTTP/1.1 409 Conflict
.. found 4 backups, but none were old enough

so still not working

What files are in that 03-12-17 folder?

jimr's picture
Submitted by jimr on Tue, 12/19/2017 - 15:47 Pro Licensee

Hi Jamie it looks like fixed but I am seeing

Deleting backups from %d-%m-%y in Dropbox folder noideersoftware older than 3 days ..
Deleting file 13-12-17, which is 6 days old ..
.. deleted 0 bytes.
Deleting file 14-12-17, which is 5 days old ..
.. deleted 0 bytes.

Deleting file 15-12-17, which is 4 days old ..
.. deleted 0 bytes.

Deleting file 16-12-17, which is 3 days old ..
.. deleted 0 bytes.

.. deleted 4 old backups, and skipped 3 that were not old enough

which may be down to the script I wrote to fix the issue ... I'll report back when my other server updates to the new version

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.