Setting up a VM on non-default custom http/ssl ports [#55192]

Submitted by bislinks on Tue, 01/16/2018 - 15:56 Pro Licensee

This is a Virtual Machine powered by Cloudmin.

I am having trouble setting up the server on custom http/SSL ports.

Things I have tried.

Restarted the Virtual Machine several times.
I have setup/changed default ports to custom ports ( http:8001 and SSL 4000) in System Settings -> Server Templates -> Default
Set custom ports for virtual server in Server Configuration -> Change IP Address

This is the error I get every time I restart httpd:

[root@ns874 ~]# service httpd restart
Redirecting to /bin/systemctl restart httpd.service
Job for httpd.service failed because the control process exited with error code. See "systemctl status httpd.service" and "journalctl -xe" for details.


[root@ns874 ~]# service httpd status -l
Redirecting to /bin/systemctl status  -l httpd.service
● httpd.service - The Apache HTTP Server
   Loaded: loaded (/usr/lib/systemd/system/httpd.service; disabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Tue 2018-01-16 16:41:11 EST; 4s ago
     Docs: man:httpd(8)
           man:apachectl(8)
  Process: 12229 ExecStop=/bin/kill -WINCH ${MAINPID} (code=exited, status=1/FAILURE)
  Process: 12226 ExecStart=/usr/sbin/httpd $OPTIONS -DFOREGROUND (code=exited, status=1/FAILURE)
 Main PID: 12226 (code=exited, status=1/FAILURE)

Jan 16 16:41:10 ns874.datalawn.com systemd[1]: Starting The Apache HTTP Server...
Jan 16 16:41:11 ns874.datalawn.com systemd[1]: httpd.service: main process exited, code=exited, status=1/FAILURE
Jan 16 16:41:11 ns874.datalawn.com kill[12229]: kill: cannot find process ""
Jan 16 16:41:11 ns874.datalawn.com systemd[1]: httpd.service: control process exited, code=exited status=1
Jan 16 16:41:11 ns874.datalawn.com systemd[1]: Failed to start The Apache HTTP Server.
Jan 16 16:41:11 ns874.datalawn.com systemd[1]: Unit httpd.service entered failed state.
Jan 16 16:41:11 ns874.datalawn.com systemd[1]: httpd.service failed.
[root@ns874 ~]#

Details that might be of help

[root@ns874 ~]# find /etc/httpd -type f | xargs grep -i listen
/etc/httpd/conf.d/ssl.conf:# When we also provide SSL we have to listen to the 
/etc/httpd/conf.d/ssl.conf:Listen 4000
/etc/httpd/conf.d/ssl.conf:#Listen [2600:1700:89e0:5ea0:54:ff:fe7d:d267]:4000
/etc/httpd/conf.d/ssl.conf:#Listen 76.209.12.9:8001
/etc/httpd/conf.d/ssl.conf:#Listen [2600:1700:89e0:5ea0:54:ff:fe7d:d267]:8001
/etc/httpd/conf/httpd.conf:# Listen: Allows you to bind Apache to specific IP addresses and/or
/etc/httpd/conf/httpd.conf:# Change this to Listen on specific IP addresses as shown below to 
/etc/httpd/conf/httpd.conf:#Listen 12.34.56.78:80
/etc/httpd/conf/httpd.conf:Listen 76.209.12.9:8001
/etc/httpd/conf/httpd.conf:#Listen 76.209.12.9:4000 https
[root@ns874 ~]# netstat -anlp | grep :4000
[root@ns874 ~]# netstat -anlp | grep :8001
[root@ns874 ~]# netstat -anlp | grep :80
tcp6       0      0 :::80                   :::*                    LISTEN      1755/docker-proxy   
[root@ns874 ~]# netstat -anlp | grep :443
tcp        0      0 76.209.12.9:43984       76.209.12.9:443         ESTABLISHED 794/gitlab-runner   
tcp6       0      0 :::443                  :::*                    LISTEN      1731/docker-proxy   
[root@ns874 ~]#

Appreciate help resolving this issue.

Status:

Closed (fixed)

Comments

Submitted by bislinks on Tue, 01/16/2018 - 16:00 Pro Licensee Comment #1

Body:

View changes

Submitted by JamieCameron on Wed, 01/17/2018 - 00:14 Comment #2

Does anything useful get logged to /var/log/httpd/error_log ?

Submitted by bislinks on Wed, 01/17/2018 - 13:52 Pro Licensee Comment #3

[root@ns874 ~]# cat /var/log/httpd/error_log |more (13)Permission denied: AH00091: httpd: could not open error log file /var/log/virtualmin/cicd87.ns87.datalawn.com_error_log. AH00015: Unable to open logs (13)Permission denied: AH00091: httpd: could not open error log file /var/log/virtualmin/cicd87.ns87.datalawn.com_error_log. AH00015: Unable to open logs (13)Permission denied: AH00091: httpd: could not open error log file /var/log/virtualmin/cicd87.ns87.datalawn.com_error_log. AH00015: Unable to open logs (13)Permission denied: AH00091: httpd: could not open error log file /var/log/virtualmin/cicd87.ns87.datalawn.com_error_log. AH00015: Unable to open logs (13)Permission denied: AH00091: httpd: could not open error log file /var/log/virtualmin/cicd87.ns87.datalawn.com_error_log. AH00015: Unable to open logs (13)Permission denied: AH00091: httpd: could not open error log file /var/log/virtualmin/cicd87.ns87.datalawn.com_error_log. AH00015: Unable to open logs (13)Permission denied: AH00091: httpd: could not open error log file /var/log/virtualmin/cicd87.ns87.datalawn.com_error_log. AH00015: Unable to open logs (13)Permission denied: AH00091: httpd: could not open error log file /var/log/virtualmin/cicd87.ns87.datalawn.com_error_log. AH00015: Unable to open logs (13)Permission denied: AH00091: httpd: could not open error log file /var/log/virtualmin/cicd87.ns87.datalawn.com_error_log. AH00015: Unable to open logs (13)Permission denied: AH00091: httpd: could not open error log file /var/log/virtualmin/cicd87.ns87.datalawn.com_error_log. AH00015: Unable to open logs (13)Permission denied: AH00091: httpd: could not open error log file /var/log/virtualmin/cicd87.ns87.datalawn.com_error_log. AH00015: Unable to open logs (13)Permission denied: AH00091: httpd: could not open error log file /var/log/virtualmin/cicd87.ns87.datalawn.com_error_log. AH00015: Unable to open logs (13)Permission denied: AH00091: httpd: could not open error log file /var/log/virtualmin/cicd87.ns87.datalawn.com_error_log. AH00015: Unable to open logs (13)Permission denied: AH00091: httpd: could not open error log file /var/log/virtualmin/cicd87.ns87.datalawn.com_error_log. AH00015: Unable to open logs (13)Permission denied: AH00091: httpd: could not open error log file /var/log/virtualmin/gitlab.cicd87.ns87.datalawn.com_error_log. AH00015: Unable to open logs (13)Permission denied: AH00091: httpd: could not open error log file /var/log/virtualmin/gitlab.cicd87.ns87.datalawn.com_error_log. AH00015: Unable to open logs (13)Permission denied: AH00091: httpd: could not open error log file /var/log/virtualmin/gitlabce.docker.datalawn.com_error_log. AH00015: Unable to open logs (13)Permission denied: AH00091: httpd: could not open error log file /var/log/virtualmin/datalawn.com_error_log. AH00015: Unable to open logs (13)Permission denied: AH00091: httpd: could not open error log file /var/log/virtualmin/datalawn.com_error_log. AH00015: Unable to open logs (13)Permission denied: AH00091: httpd: could not open error log file /var/log/virtualmin/datalawn.com_error_log. AH00015: Unable to open logs [root@ns874 ~]#

Submitted by JamieCameron on Wed, 01/17/2018 - 19:14 Comment #4

Are you sure Apache is being started as the root user?

Submitted by bislinks on Thu, 01/18/2018 - 13:00 Pro Licensee Comment #5

I have logged into SSH as root and into webmin as root. Able to update/install packages.

Submitted by bislinks on Thu, 01/18/2018 - 13:01 Pro Licensee Comment #6

[root@ns874 ~]# id uid=0(root) gid=0(root) groups=0(root) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 [root@ns874 ~]#

Submitted by andreychek on Thu, 01/18/2018 - 14:00 Comment #7

Yeah that's pretty bizarre.

What is the output of these commands:

rpm -qa | grep httpd 
ls -ld /var/log
ls -la  /var/log/virtualmin

Submitted by andreychek on Thu, 01/18/2018 - 14:02 Comment #8

Also, could you temporarily try disabling SELinux with "setenforce 0", and then see if that makes a difference? If it's an SELinux problem of some kind, that could explain the issues.

Submitted by bislinks on Thu, 01/18/2018 - 15:02 Pro Licensee Comment #9

[root@ns874 ~]# setenforce 0
[root@ns874 ~]# service httpd restart
Redirecting to /bin/systemctl restart httpd.service
Job for httpd.service failed because the control process exited with error code. See "systemctl status httpd.service" and "journalctl -xe" for details.
[root@ns874 ~]# 

[root@ns874 ~]# service httpd status -l
Redirecting to /bin/systemctl status  -l httpd.service
● httpd.service - The Apache HTTP Server
   Loaded: loaded (/usr/lib/systemd/system/httpd.service; disabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Thu 2018-01-18 15:59:32 EST; 46s ago
     Docs: man:httpd(8)
           man:apachectl(8)
  Process: 31765 ExecStop=/bin/kill -WINCH ${MAINPID} (code=exited, status=1/FAILURE)
  Process: 31762 ExecStart=/usr/sbin/httpd $OPTIONS -DFOREGROUND (code=exited, status=1/FAILURE)
 Main PID: 31762 (code=exited, status=1/FAILURE)

Jan 18 15:59:31 ns874.datalawn.com systemd[1]: Starting The Apache HTTP Server...
Jan 18 15:59:32 ns874.datalawn.com systemd[1]: httpd.service: main process exited, code=exited, status=1/FAILURE
Jan 18 15:59:32 ns874.datalawn.com kill[31765]: kill: cannot find process ""
Jan 18 15:59:32 ns874.datalawn.com systemd[1]: httpd.service: control process exited, code=exited status=1
Jan 18 15:59:32 ns874.datalawn.com systemd[1]: Failed to start The Apache HTTP Server.
Jan 18 15:59:32 ns874.datalawn.com systemd[1]: Unit httpd.service entered failed state.
Jan 18 15:59:32 ns874.datalawn.com systemd[1]: httpd.service failed.
[root@ns874 ~]# 

Same problem with sentenforce 0



[root@ns874 ~]# rpm -qa | grep httpd 
httpd-tools-2.4.6-67.el7.centos.2.vm.x86_64
httpd24-mod_ldap-2.4.27-8.el7.x86_64
httpd24-1.1-18.el7.x86_64
httpd-devel-2.4.6-67.el7.centos.2.vm.x86_64
httpd-debuginfo-2.4.6-67.el7.centos.2.vm.x86_64
httpd24-httpd-2.4.27-8.el7.x86_64
httpd24-mod_session-2.4.27-8.el7.x86_64
httpd24-mod_proxy_html-2.4.27-8.el7.x86_64
httpd24-libnghttp2-devel-1.7.1-6.el7.x86_64
httpd-manual-2.4.6-67.el7.centos.2.vm.noarch
httpd24-libcurl-7.47.1-4.el7.x86_64
httpd24-httpd-tools-2.4.27-8.el7.x86_64
httpd24-mod_auth_kerb-5.4-33.el7.x86_64
httpd24-httpd-devel-2.4.27-8.el7.x86_64
httpd24-curl-7.47.1-4.el7.x86_64
httpd-2.4.6-67.el7.centos.2.vm.x86_64
httpd24-libnghttp2-1.7.1-6.el7.x86_64
httpd24-build-1.1-18.el7.x86_64
httpd24-httpd-manual-2.4.27-8.el7.noarch
httpd24-libcurl-devel-7.47.1-4.el7.x86_64
httpd24-scldevel-1.1-18.el7.x86_64
httpd-itk-2.4.7.04-2.el7.x86_64
httpd24-runtime-1.1-18.el7.x86_64
httpd24-mod_ssl-2.4.27-8.el7.x86_64
httpd24-nghttp2-1.7.1-6.el7.x86_64
[root@ns874 ~]# ls -ld /var/log
drwxr-xr-x. 12 root root 4096 Jan 18 03:23 /var/log
[root@ns874 ~]# ls -la  /var/log/virtualmin
total 36
drwx--x--x.  2 root     root   4096 Jan 17 14:49 .
drwxr-xr-x. 12 root     root   4096 Jan 18 03:23 ..
-rw-rw----.  1 datalawn apache   20 Dec 19 13:25 cicd87.ns87.datalawn.com_access_log-20171224.gz
-rw-rw----.  1 datalawn apache   20 Dec 24 03:30 cicd87.ns87.datalawn.com_access_log-20171231.gz
-rw-rw----.  1 datalawn apache   20 Dec 31 03:32 cicd87.ns87.datalawn.com_access_log-20180114.gz
-rw-rw----.  1 datalawn apache   20 Dec 19 13:25 cicd87.ns87.datalawn.com_error_log-20171224.gz
-rw-rw----.  1 datalawn apache   20 Dec 24 03:30 cicd87.ns87.datalawn.com_error_log-20171231.gz
-rw-rw----.  1 datalawn apache   20 Dec 31 03:32 cicd87.ns87.datalawn.com_error_log-20180114.gz
-rw-rw----.  1 datalawn apache    0 Jan 14 18:36 datalawn.com_access_log
-rw-rw----.  1 datalawn apache  376 Jan 18 16:01 datalawn.com_error_log
-rw-r--r--.  1 root     root      0 Jan 17 14:49 gitlabce.docker.datalawn.com_error_log
-rw-r--r--.  1 root     root      0 Jan 17 14:48 gitlab.cicd87.ns87.datalawn.com_error_log
[root@ns874 ~]#

Submitted by andreychek on Thu, 01/18/2018 - 15:35 Comment #10

This is kind of a shot in the dark, but I'd like to rule out some other unusual issues.

Could you try changing the ports back to the default values?

Also, what does "sestatus" show now?

Submitted by bislinks on Thu, 01/18/2018 - 15:39 Pro Licensee Comment #11

[root@ns874 ~]# sestatus SELinux status: enabled SELinuxfs mount: /sys/fs/selinux SELinux root directory: /etc/selinux Loaded policy name: targeted Current mode: permissive Mode from config file: enforcing Policy MLS status: enabled Policy deny_unknown status: allowed Max kernel policy version: 28 [root@ns874 ~]#

Submitted by bislinks on Thu, 01/18/2018 - 15:43 Pro Licensee Comment #12

Running a docker giblab-ce on port 443.

Changing it's port will be some work....

[root@ns874 ~]# docker container ls
CONTAINER ID        IMAGE                     COMMAND             CREATED             STATUS                  PORTS                                                            NAMES
75d40e7fe150        gitlab/gitlab-ce:latest   "/assets/wrapper"   2 days ago          Up 26 hours (healthy)   0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp, 0.0.0.0:2202->22/tcp   gitlab_docker_18jan15_1826
[root@ns874 ~]#

Do you want me to try anyway?

Submitted by bislinks on Thu, 01/18/2018 - 17:05 Pro Licensee Comment #13

Well, got it working!!!!!!!

Changed Protocol from any to http:8001 / https:4000 for IP in Apache Web server -> Global Configuration -> Network and Addresses
Removed ip6 from SSL configuration in httpd.conf for domain
Enabled/set certs path ( had disabled/commented)

Redirecting to /bin/systemctl status  -l httpd.service
● httpd.service - The Apache HTTP Server
   Loaded: loaded (/usr/lib/systemd/system/httpd.service; disabled; vendor preset: disabled)
   Active: active (running) since Thu 2018-01-18 17:49:45 EST; 15s ago
     Docs: man:httpd(8)
           man:apachectl(8)
  Process: 23105 ExecStop=/bin/kill -WINCH ${MAINPID} (code=exited, status=1/FAILURE)
 Main PID: 25040 (httpd)
   Status: "Total requests: 0; Current requests/sec: 0; Current traffic:   0 B/sec"
   Memory: 10.8M
   CGroup: /system.slice/httpd.service
           ├─25040 /usr/sbin/httpd -DFOREGROUND
           ├─25049 /usr/sbin/httpd -DFOREGROUND
           ├─25051 /usr/sbin/httpd -DFOREGROUND
           ├─25052 /usr/sbin/httpd -DFOREGROUND
           ├─25053 /usr/sbin/httpd -DFOREGROUND
           ├─25054 /usr/sbin/httpd -DFOREGROUND
           └─25056 /usr/sbin/httpd -DFOREGROUND

Jan 18 17:49:44 ns874.datalawn.com systemd[1]: Starting The Apache HTTP Server...
Jan 18 17:49:45 ns874.datalawn.com systemd[1]: Started The Apache HTTP Server.
[root@ns874 ~]#

Can't believe spent so many hours trying to figure what was wrong.

Thanks for your help/efforts too.

Submitted by tpnsolutions on Thu, 01/18/2018 - 18:12 Comment #14

Glad to hear you've got it working!

Submitted by tpnsolutions on Thu, 01/18/2018 - 20:01 Comment #15