On ubuntu 16.04, may affect other platforms as well. Seeing an issue with Virtualmin installs run from a script. The install fails and we see the below output from the log.
INFO - 2016-08-17 05:55:59 - Started installation log in /root/virtualmin-install.log DEBUG - 2016-08-17 05:55:59 - Install mode: full DEBUG - 2016-08-17 05:55:59 - Product: Virtualmin GPL DEBUG - 2016-08-17 05:56:00 - Virtualmin Meta-Package list: virtualmin-base DEBUG - 2016-08-17 05:56:00 - install.sh version: 1.1.4 INFO - 2016-08-17 05:56:00 - Checking for fully qualified hostname... INFO - 2016-08-17 05:56:00 - Hostname OK: fully qualified as ults.test.build.rimuhosting.com INFO - 2016-08-17 05:56:00 - Installing serial number and license key into /etc/virtualmin-license INFO - 2016-08-17 05:56:00 - Loading OS selection library... INFO - 2016-08-17 05:56:00 - Download of http://software.virtualmin.com/lib/oschooser.pl Succeeded. INFO - 2016-08-17 05:56:01 - Loading OS list... INFO - 2016-08-17 05:56:01 - Download of http://software.virtualmin.com/lib/os_list.txt Succeeded. INFO - 2016-08-17 05:56:01 - Operating system name: Ubuntu Linux INFO - 2016-08-17 05:56:01 - Operating system version: 16.04.1 INFO - 2016-08-17 05:56:01 - Configuring package manager for Ubuntu Linux 16.04.1... INFO - 2016-08-17 05:56:01 - Enabling universe repositories, if not already available... INFO - 2016-08-17 05:56:02 - Disabling cdrom repositories... ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@
Additionally the /etc/apt/sources.list file ends up with junk in the last line and apt is broken until that is removed manually
root@trustedbit:~# apt-get update E: Malformed line 4 in source list /etc/apt/sources.list (type) E: The list of sources could not be read.
In our environment the sources files looks like this normally.
root@ults:~# cat /etc/apt/sources.list deb http://archive.ubuntu.com/ubuntu/ xenial main restricted universe multiverse deb http://archive.ubuntu.com/ubuntu/ xenial-updates main restricted universe multiverse deb http://security.ubuntu.com/ubuntu/ xenial-security main restricted universe multiverse
The installer is actually being run by cron using an @reboot command which then launches a small shell script to grab the installer and then clean up after it. Nothing special, just a wget of install.sh, run "/bin/bash install.sh", and then rm -f that file and the launching script. Does install.sh expect anything special in terms of the environment it runs in?
Comments
Submitted by andreychek on Wed, 08/17/2016 - 09:36 Comment #1
Howdy -- hmm, I've never seen anything like that!
It sounds like you're saying that is intermittent?
At the moment I'm not quite sure what could cause that. The installer adds a couple of lines of text to the sources.list file, and if the universe repository is present, it makes sure it's commented out.
Maybe this is a place to start --
Are you able to reproduce that issue, when running it manually on the command line, rather than from within cron?
If not, we may need to look into issues with it being run automatically, without a terminal/tty, or possibly path related issues of cron.
Submitted by support@rimuhos... on Wed, 08/17/2016 - 20:57 Pro Licensee Comment #2
Running manually from the command line 'just works'.
Running as described in the report repeatably fails with the reported issue on ubuntu 1604.. I can provide additional details if you need anything. Just let me know.
FWIW the installer used to work just fine. So this is likely related to a somewhat recent change.
Submitted by andreychek on Wed, 08/17/2016 - 21:43 Comment #3
Do you have a rough idea as to when it started occurring?
That is weird.
But, what happens if you use /bin/sh instead of /bin/bash? Maybe there's something going wrong there. We test against /bin/sh, which is not bash on Ubuntu (it is dash), and there are some differences between the two shells (but we test against bash on CentOS, so it does get tested with bash, as well...but, that's the only thing I could think of that could cause weirdness like this).
Submitted by andreychek on Thu, 08/18/2016 - 00:39 Comment #5
And to clarify, you can set the shell the script uses by editing the install.sh file, and at the top, change "#!/bin/sh" to "#!/bin/bash".
Submitted by support@rimuhos... on Thu, 08/18/2016 - 22:56 Pro Licensee Comment #6
From the calling script, doing this...
/bin/sh install.sh
... made no difference at all, still the same mess in the shell and a munged sources file. fwiw the login shell is indeed bash.
Dont see anything else touching the sources file. I wonder if there is some escaping in the sed commands that is misbehaving? I'll do some extra testing on those when I get time.
Submitted by andreychek on Thu, 08/18/2016 - 23:06 Comment #7
What Joe is suggesting, is to actually use bash, rather than sh.
On Ubuntu, /bin/sh can often point to another shell named "dash".
You could try using "/bin/bash install.sh", or just change the first line of the install.sh script to point to bash.
Submitted by support@rimuhos... on Tue, 08/23/2016 - 01:16 Pro Licensee Comment #8
I tried that, nothing seemed to help. I was also able to replicate the issue on ubuntu 1404. I note that centos6 works fine using this method (no systemd). I may have to try doing this with a systemd timer or similar instead.
Submitted by support@rimuhos... on Wed, 08/24/2016 - 00:14 Pro Licensee Comment #9
So I found a better workaround for now. Simply by adding a sleep 60 before the installer gets run, things work as expected. I know that the sleep length may still miss depending on how quickly the server comes up so it may still crop up, but its good enough for my env.
So the issue I reported above clearly seems to be caused by the job starting too early in the boot sequence. Not an installer issue. Feel free to close and please pardon the noise
Submitted by andreychek on Wed, 08/24/2016 - 00:31 Comment #10
That's still pretty odd! I'm not sure what about that would be causing it to break.
However, I'm thrilled that you found a solution, as we've been pretty stumped thus far :-)
Thanks for letting us know how you fixed it!