Backup fails

Howdy -- can you attach a screenshot of your scheduled backup screen that isn't working properly? Thanks!

Thanks for your image!

Last night, Virtualmin 3.97-2 was pushed out, which corrects some bugs that were discovered.

Can you try upgrading to that, and let us know if that helps with the problem you're having?

If not -- you mentioned that you were getting errors during the backup/delete, which errors are you seeing? We can use your screenshot and those errors to troubleshoot the problem you're having. Thanks!

Just to clarify, are you using the Virtualmin version that just went out, version 3.97-2?

You can determine that with this command:

dpkg -l 'webmin-virtual-server'

Same here, Virtualmin 3.97.gpl and 3.97.pro on Debian 6. Backup over SSH is broken, because target directory names will be escaped unnecessarily. That leads to "directory doesn't exist" while copying the file.

I do not see 3.97-2 available anywhere. Is it in the repositories? Regards,

Christian

@datenimperator - 3.97-2 should be available now, and fixes this issue (along with a couple of others).

Are you using the GPL or pro version, and on which Linux distribution?

Both GPL and Pro (on different systems), all on Debian 6. Target system for backup is Ubuntu 10 LTS.

I've just ran apt-get update && apt-get dist-upgrade but no update was available from neither download.webmin.com nor download.virtualmin.com

Are you still not seeing the 3.97-2 release? It looks like it is in our repositories to me ..

Alternately, the GPL Debian package can be downloaded and installed with dpkg from http://download.webmin.com/download/virtualmin/webmin-virtual-server_3.9...

Hm, strange. The deb package is installed, but Virtualmin still shows its version as 3.97.gpl instead of 3.97.gpl-2. Backup still fails.

… but

Virtualmin doesn't actually report it's version with the "-2" -- you can only determine that by looking at the package that's installed.

So, even though you have webmin-virtual-server version 3.97.gpl-2 installed, when looking at the version in Virtualmin, it will show just 3.97.

Are you continuing to see the same problem after upgrading to the new Virtualmin version?

@datenimperator - what is the exact path you are backing up to?

It backs up to a SSH server. This is the backup job that failed just some hours ago:

It's set to "create target directory", but that looks weird, too (notice the backslashes in the directory names):

The log contains:

I suspect that this might be triggered by using a destination path that doesn't start with /

If you change your backup destination to a full path, do you still see this problem?

By the way, I tested this with Virtualmin 3.97-2 and a relative path, but still couldn't re-produce the problem.

I changed it to /tmp/backup/… to try it and it would give me the same error. What puzzles me is the fact that 5 Virtualmin systems (both GPL and Pro) stopped their backup on the same day after working flawlessly for months. I did not change anything on the target system. What I did was upgrading webmin/virtualmin on the source systems when upgrade were available.

As a developer, I'd try to isolate the source problem with the escaped path. Looks as if a path string would have turned into a regex by escaping all dashes and slashes and whatnot. Did something change recently with regards to that?

Does the destination SSH user have an scp-only shell? That could impact how the destination directory is created ..

Yes, all destination users were scp-only, which worked before.

I've changed it to a full-featured shell but the error still persists. Can I increase the log detail by any means to help you understand what's going on?

All my SSH backups on all webmin-based systems stopped working , so this is getting more urgent with every day.

If possible, I'd be happy to login to your system myself to see what is going wrong here.

Source or destination? Please send your SSH key.

I'd need access to the source (Virtualmin) system, as that is where the bug most likely is.

Email me directly at jcameron@virtualmin.com and I'll reply with my SSH key.

I am having the exact same problem Virtualmin GPL 3.97. When I try to backup over SSH with SCP only user I get this error: Uploading archive to SSH server dev.hype.ro .. .. upload failed! webhosting@dev.hype.ro's password: scp: /backupservers/web.hype.ro_backup/16-01-2013/active.hype.ro.zip.info: Not a directory

I started to happen after the virtualmin upgrade no other info changed to the backup setup.

@daniel - does the destination SSH user have an scponly shell, or does he have a regular shell?

it is scponly shell on the destination. from filezilla or an other SCP client the upload of an file works without problems.

Unfortunately, the scponly shell would cause the problems you're seeing.

Although the scponly shell can work from filezilla or other clients designed for SCP, Virtualmin uses functionality that doesn't work properly with the scponly shell.

Try changing the shell to bash or similar to see if your backups begin working at that point.

I haven't tested this in awhile, but if you're interested in using a restricted shell for your backup user, I believe using lshell would work. You can find details of it here:

http://lshell.ghantoos.org/

I just found a bug in Virtualmin that is triggered when the scponly shell is in use, and you are backing up to a directory that doesn't exist yet. This will be fixed in the upcoming 3.98 release.

Ok I have changed the user shell to bash from scponly and the problem is that the error remains the same.

Make sure you delete the file that was incorrectly created instead of the destination directory.

I am on 3.97 and the backup fails when using ~/dom/theduck/%Y%m%d-%H%M%S because it won't create the directory. This started with 3.97.

If I switch back to "One file per server (old format)" then it works fine.

@cgdhosting - not sure if I asked this already, but does the user you are backing up to via SSH have an scponly shell? That will prevent the listing of directories needed to purge old backups.

Could you post your backup output, or email it to me at jcameron@virtualmin.com ?

I'm mainly interested in the section near the end where it mentions purging old backups.

Thanks. In cases where the backup failed, no purging of old backups will be done, so that you don't end up in a situation where all your valid backups have been lost.

However, for those that succeeded, purging should have worked.

If you run ls -l /home/curtis/backups on your remote server via SSH as the same user you use for backups, what does it output?

Was the directory actually deleted?

That incorrect size could be reported if Virtualmin couldn't run the command du -sk /home/curtis/backups/24-01-2013 on the remote host to get the full directory size..

Ok, let me know if that scheduled backup completes OK or not.

It looks like SSH commands to the backup destination by Virtualmin aren't actually working, causing the incorrect size to be displayed and deletion to fail.

Would it be possible for me to login to your system again to see what is going wrong?

You could send the log file /var/log/messages from the destination system - that should contact debugging information from the SSH server.

The Ubuntu version shouldn't matter.

Is there any log file under /var/log on the destination system that contains messages from the SSH server sshd ?

Another thing you could do is enable logging of commands run by Virtualmin, at Webmin -> Webmin Configuration -> Debugging Log File. Under "Events to log" check on "Commands executed ", and set "Debug log enabled? " to "Yes".

The kick off a backup in which the purging fails, and after it completes email the log file /var/webmin/webmin.debug to me at jcameron@virtualmin.com

Automatically closed -- issue fixed for 2 weeks with no activity.

Does it go back to failing again if you disable debugging?

Is the debugging still enabled on your server?