FTP Login error on New Ubuntu 20.04 Server

This is a bug report. This system was setup new and configured on 8/22/2020, Below is all the versions of everything:

Operating system Ubuntu Linux 20.04.1
Perl version 5.030000
Path to Perl /usr/bin/perl
BIND version 9.16
Postfix version 3.4.13
Mail injection command /usr/lib/sendmail -t
Apache version 2.4.41
PHP versions 7.2.33, 7.3.21, 7.4.9
Webalizer version 2.23-08
Logrotate version 3.14.0
MySQL version 8.0.21-0ubuntu0.20.04.4
ProFTPD version 1.36
SpamAssassin version 3.4.4
ClamAV version 0.102.4

What is happening, basic FTP (21) is not authenticating in FTP clients I get 530 Login incorrect in a web app I just get Cannot log in to FTP server

the same login details, the Admin user created for the specified Virtualmin Server for a specified domain, this seems to be happening with all domains on the server for all admin accounts.

I have included an error below from the auth.log. Please let me know if you could use any more information in order to fix this bug.

This is happening on all Virtual server on this instillation

Auth.log snip included below:

Aug 23 06:13:41 host4 proftpd: pam_unix(proftpd:auth): Couldn't open /etc/securetty: No such file or directory
Aug 23 06:13:41 host4 proftpd: pam_unix(proftpd:auth): Couldn't open /etc/securetty: No such file or directory
Aug 23 06:13:41 host4 proftpd: pam_unix(proftpd:session): session opened for user west123 by (uid=0)
Aug 23 06:13:41 host4 proftpd: pam_unix(proftpd:session): session closed for user west123

The below happened on 4 of 22 Virtualmin Servers created, not constant, back to back or any rhyme or reason, If there are any logs I can provide for you to look into this for your benifit, I will be more than happy to provide the necessary files/logs.

While some domains are working fine, I bumped into 1 so far that was missing the following code from the passwd file under the chroot jail

domainadminuser:x:1009:1006::/home/domainadminuser:

I have seen this issue before, but it was much more consistent in not including the /bin/bash line this ends up breaking SSH

domainadminuser:x:1009:1006::/home/domainadminuser:/bin/bash

Thanks,

P.S. I am trying to migrate to this server and will look for alternatives to complete the migration while you work on this issue.

Status: 
Active

Comments

What gets logged to /var/log/messages when a user tries to FTP in?

Hello Jamie,

The /var/log/messages log was disabled by default, so I enabled it and tried the ftp (21) again and nothing is written to the /var/log/messages file. I repeated the same test after rebooting the server as well.

The file did not exist before enabling the log in System Logs in Webmin

some other information that might be helpful, I will post 2 pictures of the System Logs page to the top post, but the pictures had me take a look at the /etc/rsyslog.conf and compare this (Ubuntu 20.04.1) to the (Ubuntu 18.04.5) install, they match.

1 other thing, after all these failures started happening, I rushed to look at the Fail2Ban wondering why I had not been banned... In my new install of Ubuntu 20.04, this is not set to start with the server, it gives the choice to start it and to enable it to start with the server, I have tested starting the fail2ban, which works, but not to start with the server yet.

And by the looks of the pictures I am about to post to the opening post, it does not seem like the log server is running either.... on Ubuntu 20.04 at least.

I had just gotten the skeleton in place, on this Ubuntu 20.04, previously I had gotten to this point on Ubuntu 18.04.5, but noticed you updated the Operating System A compatibility to include version 20.04 on it, so I created a new image trying for the most updated image.

A quick question, out of habit and out of compatibility for other control panels in the past, I had changed the default /bin/sh to be /bin/bash instead of the default /bin/dash. Would this affect anything?

Thank you

I added another screenshot, I have not seen this occur on previous installs. I hope this also helps you in your troubleshooting of this bug (FTP (21)) and the random /bin/bash not being in the /etc/passwd file in the chroot jail user and some other things that you might have discovered while looking at the pictures I have posted.

Thank you, good luck and please let me know if I can provide any other information.

Ilia's picture
Submitted by Ilia on Tue, 08/25/2020 - 13:34

Does this issue equally happen with domains being jailed and being not? It certainly works out of the box for regular domains (without jail) on my test Ubuntu 20.04.

Do you have steps to reproduce this issue. What makes you think it's a bug and not a local misconfiguration?

The install is on a clean Ubuntu 20.04.1 install, all updates installed, the only extra packages installed are ssh openssh-server for putty access to make configuration easier.

Other components installed: ntp nano vim-nox

insure all current updates are installed

Configure hosts and network

reconfigure /bin/sh to be /bin/bash instead of /bin/dash (and I have been asking if this is an issue, I have been doing this since way back in 16.04 with no issues)

run install.sh

After basic configuration, I add the additional versions of PHP I would like available

sudo apt-get update
sudo apt -y install software-properties-common
sudo add-apt-repository ppa:ondrej/php
sudo apt-get update

sudo apt-get install PHP7.2 PHP7.2-{calendar,cgi,ctype,curl,dom,exif,fileinfo,ftp,gd,gettext,iconv,imap,json,mbstring,mysqli,mysqlnd,PDO,mysql,Phar,posix,readline,shmop,SimpleXML,sockets,sodium,ssh2,sysvmsg,sysvsem,sysvshm,tokenizer,wddx,xml,xmlreader,xmlwriter,xsl,zip}

sudo apt-get install PHP7.3 PHP7.3-{calendar,cgi,ctype,curl,dom,exif,fileinfo,ftp,gd,gettext,iconv,imap,json,mbstring,mysqli,mysqlnd,PDO,mysql,Phar,posix,readline,shmop,SimpleXML,sockets,ssh2,sysvmsg,sysvsem,sysvshm,tokenizer,wddx,xml,xmlreader,xmlwriter,xsl,zip}

sudo apt-get install PHP7.4 PHP7.4-{calendar,cgi,ctype,curl,dom,exif,FFI,fileinfo,ftp,gd,gettext,iconv,imap,json,mbstring,mysqli,mysqlnd,PDO,mysql,Phar,posix,readline,shmop,SimpleXML,sockets,ssh2,sysvmsg,sysvsem,sysvshm,tokenizer,xml,xmlreader,xmlwriter,xsl,zip}

While some of the configuration files have changed and moved (ex the network settings), I am still performing the same basic install and configuration before I run the install.sh and I let that take care of any packages and dependencies.

I have used this setup means for this since way back as ubuntu 16.04 with no issues, some minor issues with 18.04 but they turned out being actual bugs that there are tickets here on.

I have been using the jail since you started offering it. And the FTP error if you take a look at it above, the /etc/securetty file was never created. the creation of this is not done by the end user.

Are you using a clean install of Ubuntu 20.04, or a upgraded version from 18.04? I specifically stated new system, I did not perform an upgrade, clean from the 20.04 iso file.

to answer your question on the FTP (21) works fine for non jailed users, but I am utilizing the Jail that you include and it is not working when jails are enabled so that should be considered a bug.

As for Miss-configuration, I have been utilizing the same configuration settings since back in 16.04 and had no issues back then or in 18.04 once the ssh jail issue was fixed and there was a problem with the php switcher that was corupting a configuration file that prevented apache from starting but that was fixed as well

I do NOT edit any files manually after running and setting up install.sh (beyond the php.ini global preferences that are edited through the dashboard)

The only files I ever edit by hand are: the netplan for networking before setup the hosts file to specify the local ip and FQDN and host name and like I said above the global php.ini files for each version of PHP on the system (through the control panel though)

The only reason I even know anything about the /etc/passwd file is because of the previous issue I had.

Thank you

I am in the middle of setting up another 20.04 server. The only difference so far is the media I used.

There are 2 different ISO's

ubuntu-20.04.1-live-server-amd64.iso - the new install interface

ubuntu-20.04-legacy-server-amd64.iso - previously know in older version as ubuntu-18.04.5-server-amd64.iso

I install all the updates install the dist-upate to get it to 20.04.1 configure network configure hosts

install the nano and vim-nox

I took a moment and researched the /bin/bash vs /bin/dash and on these forms here, I have seen the bash recommended, so I reconfigured the /bin/sh to be /bin/bash instead of /bin/dash

install ntp

then I run the install.sh setup

here is current things I still see, service --status-all shows that rsyslog is indeed running, as it was before on the last install of 20.04 as well, just the system logs page does not recognize it as running...for whatever reason

something I paid closer attention to this time since I was looking for it, was the Fail2Ban was running after initial install but it is still by default toggled to not start on boot up, and it will NOT let me change the toggle to set it to start at bootup, it keeps resetting itself to "no" but Fail2Ban CAN be manually started.

I am not seeing the errors that was in the login_screen_ubuntu_v20_04.jpg picture anymore

I have just been able to get far enough to verify this new install as of 8/25/2020 at 10:04 P.M. EST with the list of installed components and versions below, the FTP (21) still gave the same error as when I started this post from above

The error from the client 530 login incorrect, is the same thing as if the user did not even exist on the system.

At this point, I believe I only have this server set up to help you troubleshoot any existing bugs and report them to you. I will be more than willing to set up a user for you to be able to look around the server to help your efforts. but I do need to move foreword right now and I will be falling back to version 18.04 for the time being.

Please be aware that I did even less customization to this server than I normally do, I just changed some DNS record defaults, applied the PASV port range in FTP to match my firewall, I did not even install any extra versions of PHP this time. And of course enable the jails to be created in the template.

I have it set up so it can stay running, I may lower the resources since it will be just a test server for the time being. I am willing to help you as much as I am able. Good Luck and thank you

Operating system Ubuntu Linux 20.04.1
Perl version 5.030000
Path to Perl /usr/bin/perl
BIND version 9.16
Postfix version 3.4.13
Mail injection command /usr/lib/sendmail -t
Apache version 2.4.41
PHP versions 7.4.3
Webalizer version 2.23-08
Logrotate version 3.14.0
MySQL version 8.0.21-0ubuntu0.20.04.4
ProFTPD version 1.36
SpamAssassin version 3.4.4
ClamAV version 0.102.4

Hello,

This is a quick update on the comment of "Miss-configured" I just re-setup a 18.04 Server the same way.

Fail2Ban is able to be set to start with the server FTP(21) is working fine with jails enabled The System Logs page is showing Apply changes instead of the Start Syslog Server

Operating system Ubuntu Linux 18.04.5
Perl version 5.026001
Path to Perl /usr/bin/perl
BIND version 9.11
Postfix version 3.3.0
Mail injection command /usr/lib/sendmail -t
Apache version 2.4.29
PHP versions 7.2.33, 7.4.9
Webalizer version 2.23-08
Logrotate version 3.11.0
MySQL version 5.7.31-0ubuntu0.18.04.1
ProFTPD version 1.35
SpamAssassin version 3.4.2

This is to only point out that the way I set up and configure the server does not appear to be the issue.

The operating system you are developing this for is new, I do not fault you for there being issues still, I am just wanting to report them so they get fixed, and I am offering a test server for this purpose, that is already experiencing these issues. I have currently resigned myself to stay with 18.04 since it seems a great deal more stable at the moment. Thank you and I do appreciate all the hard work you do to make this such a wonderful product. Once I get free time, I plan to learn more to be able to actually help instead of just report bugs

Ilia's picture
Submitted by Ilia on Fri, 08/28/2020 - 06:38

The /var/log/messages log was disabled by default, so I enabled

You could always use -

journalctl -xe

23 06:13:41 host4 proftpd: pam_unix(proftpd:auth): Couldn't open /etc/securetty: No such file or directory

This can be safely ignored and affects nothing in regard of this issue.

I have seen this issue before, but it was much more consistent in not including the /bin/bash line this ends up breaking SSH

That would certainly break logins. I assume you intentionally changed login shell?

Aug 23 06:13:41 host4 proftpd: pam_unix(proftpd:session): session opened for user west123 by (uid=0)
Aug 23 06:13:41 host4 proftpd: pam_unix(proftpd:session): session closed for user west123

This indeed indicates a problem, which I could reproduce on Ubuntu 20.04 with Jailkit enabled but only when plain ftp connection is used (port 21). The simple and better solution for you would be to use sftp connection (port 22).

We deeply appreciate your findings and patience. While testing this, I also figured that we have a bug in Jailkit library in Virtualmin 6.12, which we will fix as soon as possible.

That would certainly break logins. I assume you intentionally changed login shell?

inside of the Virtualmin console I have not modified any shells that were setup by the system itself and whatever it defaults to for the jail's which it looks like /bin/bash, just the defaults are being used

The only thing I modified, which is system wide, before even starting the install.sh, is what /bin/sh is default to, I have /bin/sh defaulted to /bin/bash instead of /bin/dash

You could always use - journalctl -xe

Aug 28 18:11:52 host8 systemd-resolved[804]: Server returned error NXDOMAIN, mitigating potential DNS violation DVE-2018-0001, retrying transaction with reduced feature level UDP. Aug 28 18:11:52 host8 proftpd[865041]: pam_unix(proftpd:auth): Couldn't open /etc/securetty: No such file or directory Aug 28 18:11:52 host8 proftpd[865041]: pam_unix(proftpd:auth): Couldn't open /etc/securetty: No such file or directory Aug 28 18:11:52 host8 proftpd[865041]: pam_unix(proftpd:session): session opened for user west123 by (uid=0) Aug 28 18:11:52 host8 proftpd[865041]: pam_unix(proftpd:session): session closed for user west123

Thank you Ilia, would you mind posting back when this has been fixed (I guess 6.13?), I will give it another try once you get back with me and I will let you all know know if I find anything else, in a new bug report.

Thanks,

Ilia's picture
Submitted by Ilia on Fri, 08/28/2020 - 18:08

Thank you Ilia, would you mind posting back when this has been fixed (I guess 6.13?),

Yes, Virtualmin 6.13 should contain a fix for jails in general. I haven't checked what the real cause of ProFTPd issue is. I will update this ticket whenever it's fixed.

Thanks!