Fronting Virtualmin/Apache with Nginx (a la Plesk, but better)

Hi,

I'd like to implement an Nginx front to Virtualmin/Apache, Plesk style so I can use Nginx caching etc and leave the PHP/htaccess and other heavy-lifting to Apache.

I have made progress, but I need help.

What I've done so far is:

  1. Set Apache to listen on 1080 and 1443 respectively

  2. Installed Nginx and set it to include /etc/nginx/vhosts/*.conf

  3. Set Virtualmin to run a pre/post script which sets up Nginx vhosts (see below)

Where I got stuck - at server aliases, which in Nginx are just extra hostnames on the server_name line. I am not sure how to determine if a new server is an alias, which is the parent and how to actually keep the server_name line updated with aliases when they are added/deleted/modified, without duplicates and so on...

Any help most welcome. I'll make a tutorial when/if I ever finish this.

#!/bin/bash
if [[ "$VIRTUALSERVER_ACTION" = "MODIFY_DOMAIN" || "$VIRTUALSERVER_ACTION" = "CREATE_DOMAIN" || "$VIRTUALSERVER_ACTION" = "RESTORE_DOMAIN" ]]; then

# START VHOST
echo "
server {
        server_name $VIRTUALSERVER_DOM www.$VIRTUALSERVER_DOM;
        listen  $VIRTUALSERVER_IP;
        root $VIRTUALSERVER_PUBLIC_HTML_PATH;
        access_log /var/log/virtualmin/"$VIRTUALSERVER_DOM"_nginx_access_log
        error_log /var/log/virtualmin/"$VIRTUALSERVER_DOM"_nginx_error_log
        fastcgi_param GATEWAY_INTERFACE CGI/1.1;
        fastcgi_param SERVER_SOFTWARE nginx;
        fastcgi_param QUERY_STRING \$query_string;
        fastcgi_param REQUEST_METHOD \$request_method;
        fastcgi_param CONTENT_TYPE \$content_type;
        fastcgi_param CONTENT_LENGTH \$content_length;
        fastcgi_param SCRIPT_FILENAME $VIRTUALSERVER_PUBLIC_HTML_PATH\$fastcgi_script_name;
        fastcgi_param SCRIPT_NAME \$fastcgi_script_name;
        fastcgi_param REQUEST_URI \$request_uri;
        fastcgi_param DOCUMENT_URI \$document_uri;
        fastcgi_param DOCUMENT_ROOT $VIRTUALSERVER_PUBLIC_HTML_PATH;
        fastcgi_param SERVER_PROTOCOL \$server_protocol;
        fastcgi_param REMOTE_ADDR \$remote_addr;
        fastcgi_param REMOTE_PORT \$remote_port;
        fastcgi_param SERVER_ADDR \$server_addr;
        fastcgi_param SERVER_PORT \$server_port;
        fastcgi_param SERVER_NAME \$server_name;

"       > /etc/nginx/vhosts/$VIRTUALSERVER_DOM.conf

# PROXY PHP to Apache and protect htaccess
echo "
        location ~ \.php$ {
       
        proxy_set_header X-Real-IP  \$remote_addr;
        proxy_set_header X-Forwarded-For \$remote_addr;
        proxy_set_header Host \$host;
        proxy_pass http://127.0.0.1:1080;
        }

        location ~ /\.ht {
                deny all;
        } " >> /etc/nginx/vhosts/$VIRTUALSERVER_DOM.conf

# END PROXY PHP

# SSL section
        if [[ $VIRTUALSERVER_SSL == 1 ]]; then
        echo "
        listen $VIRTUALSERVER_IP:443 ssl;
        ssl_certificate $VIRTUALSERVER_SSL_CERT;
        ssl_certificate_key $VIRTUALSERVER_SSL_KEY;" >> /etc/nginx/vhosts/$VIRTUALSERVER_DOM.conf
                if [ -z "$VIRTUALSERVER_SSL_CA" ]; then
        echo "\
        ssl_client_certificate $VIRTUALSERVER_SSL_CHAIN;" >> /etc/nginx/vhosts/$VIRTUALSERVER_DOM.conf
                fi
        fi
# END SSL

echo "}" >> /etc/nginx/vhosts/$VIRTUALSERVER_DOM.conf
# END VHOST

elif    [[ "$VIRTUALSERVER_ACTION" = "DELETE_DOMAIN"  ]]; then
        rm -fv /etc/nginx/vhosts/$VIRTUALSERVER_DOM.conf
fi
Status: 
Needs review

Comments

Thanks, but people have been asking me for Nginx, hence my efforts...

The Varnish tutorial is very basic though, does not cover multiple backend IPs nor HTTPS which is what most of the web traffic is nowadays.

Varnish doesn't actually natively offer support SSL:

https://www.varnish-cache.org/docs/3.0/phk/ssl.html

We don't have any formal support for Nginx as a front end, though if you have questions regarding things like how to use the command line tools we can do that.

If you run a command such as "virtualmin list-domains --domain DOMAIN.TLD --simple-multiline", that will generate a bunch of info regarding that domain.

If you look at the "Type" parameter, you can see if it's an alias.

Mostafa's picture
Submitted by Mostafa on Mon, 12/26/2016 - 11:14

You can check to see if the variable $PARENT_DOMAIN_DOM is set or not. When the variable is set, the domain is either a sub-server or an alias. You can then check if this variable does not end with $DOMAIN_DOM to ensure that it is not a sub-server, and at last it is possible to use sed to update the server_name line! I guess the syntax would be something like this (Not tested):

sed -i /etc/nginx/vhosts/$VIRTUALSERVER_DOM.conf -e "s/server_name\(.*\)\s$PARENT_DOMAIN_DOM\swww\.$PARENT_DOMAIN_DOM\(.*\)/server_name\s\1$PARENT_DOMAIN_DOM\swww\.$PARENT_DOMAIN_DOM\s$DOMAIN_DOM\swww\.$DOMAIN_DOM\s\2/"

Thanks, I was trying to rely on ALIAS_ACTION=CREATE_ALIAS, alas this is declared even when creating top-level vhosts. ..

The best way to check if a domain being created is by using the $VIRTUALSERVER_ALIAS variable - if it is non-empty, it contains the ID of the alias's target domain.

Mostafa's picture
Submitted by Mostafa on Tue, 12/27/2016 - 01:08

@Lucian please update your first post with the final version of your script once you've finished it! I'm interested in this!

Im currently using nginx+apache on my servers however it doesnt support SSL as nginx is forwarding all requests to apache:

server {
        listen 80 default;
        server_name _;
        root /var/www;
        client_max_body_size 20M;

        location / {
                access_log      off;
                proxy_pass http://PUBLIC-IP:81;
                proxy_set_header X-Real-IP  $remote_addr;
                proxy_set_header Host $host;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_read_timeout 120;
                proxy_connect_timeout 120;
        }
}

Hello,

I got something that sort of seems to be working. The "code" might require some proper indentation here and there or some quotes, but "it works for me", including SSL and aliases. I've used the variable Jamie suggested.

This could be easily extended using custom global variables, such as:

if "NGINX_STATIC_CACHING" variable exists, then include in nginx vhost conf some snippet to cache css/jpg/etc or

if "MAGENTO_SHOP" is defined, then include whatever fastcgi/proxy cache snippet voodoo you do there. Sky's the limit ..

Any testing and feedback welcome.

You need to include /etc/nginx/vhosts/*.conf in the main nginx config file for the below to work.

#!/bin/bash

# if the action is positive (ie not deletion); then we need to do virtualhost stuff

if [[ "$VIRTUALSERVER_ACTION" = "MODIFY_DOMAIN" || "$VIRTUALSERVER_ACTION" = "CREATE_DOMAIN" || "$VIRTUALSERVER_ACTION" = "RESTORE_DOMAIN" ]]; then

# if it's just an alias, we simply add it to a separate aliases file we include in the main vhost
if [ -n "$VIRTUALSERVER_ALIAS" ]; then
echo "server_name $VIRTUALSERVER_DOM www.$VIRTUALSERVER_DOM;" >> /etc/nginx/vhosts/"$ALIAS_VIRTUALSERVER_DOM".aliases
service nginx reload

else

# IF NOT ALIAS THEN ADD PROPER VHOST

# prepare aliases file for any eventual aliases
touch /etc/nginx/vhosts/"$VIRTUALSERVER_DOM".aliases

# actual vhost config
echo "
server {
        server_name $VIRTUALSERVER_DOM www.$VIRTUALSERVER_DOM;
        include /etc/nginx/vhosts/$VIRTUALSERVER_DOM.aliases;
        listen  $VIRTUALSERVER_IP;
        root $VIRTUALSERVER_PUBLIC_HTML_PATH;
        access_log /var/log/virtualmin/"$VIRTUALSERVER_DOM"_nginx_access_log;
        error_log /var/log/virtualmin/"$VIRTUALSERVER_DOM"_nginx_error_log;
"       > /etc/nginx/vhosts/$VIRTUALSERVER_DOM.conf

# PROXY PHP to Apache and protect htaccess and htpassword
echo " 
        location ~ \.php$ {
        proxy_set_header Host \$host;
        proxy_set_header X-Real-IP \$remote_addr;
        proxy_set_header X-Forwarded-For \$proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto \$scheme;
        proxy_pass http://$VIRTUALSERVER_IP:1080;
        }

        location ~ /\.ht {
                deny all;
        } " >> /etc/nginx/vhosts/$VIRTUALSERVER_DOM.conf

# END PROXY PHP

# SSL section
        if [[ $VIRTUALSERVER_SSL == 1 ]]; then
        echo "
        listen $VIRTUALSERVER_IP:443 ssl;
        ssl_certificate $VIRTUALSERVER_SSL_CERT;
        ssl_certificate_key $VIRTUALSERVER_SSL_KEY;" >> /etc/nginx/vhosts/$VIRTUALSERVER_DOM.conf
                if [ -n "$VIRTUALSERVER_SSL_CHAIN" ]; then
        echo "\
        ssl_client_certificate $VIRTUALSERVER_SSL_CHAIN;" >> /etc/nginx/vhosts/$VIRTUALSERVER_DOM.conf
                fi
        fi
# END SSL

echo "}" >> /etc/nginx/vhosts/$VIRTUALSERVER_DOM.conf
# END VHOST
service nginx reload
fi

# if the action is to delete a domain, we need to remove the vhost or the alias
elif    [[ "$VIRTUALSERVER_ACTION" = "DELETE_DOMAIN"  ]]; then

if [ -n "$VIRTUALSERVER_ALIAS" ]; then
        sed -i "/^server_name "$VIRTUALSERVER_DOM" www."$VIRTUALSERVER_DOM";/d" /etc/nginx/vhosts/"$ALIAS_VIRTUALSERVER_DOM".aliases
        service nginx reload
else
        rm -fv /etc/nginx/vhosts/"$VIRTUALSERVER_DOM".conf /etc/nginx/vhosts/"$VIRTUALSERVER_DOM".aliases
        service nginx reload
fi
fi
Status: Needs work » Needs review
Mostafa's picture
Submitted by Mostafa on Wed, 12/28/2016 - 11:16

It worked for me as well!

You should replace: proxy_set_header X-Forwarded-For \$remote_addr; with proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; to make it correct.

I used the following proxy pass instead

        location / {

        proxy_set_header X-Real-IP  \$remote_addr;
        proxy_set_header X-Forwarded-For \$proxy_add_x_forwarded_for;
        proxy_set_header Host \$host;
        proxy_pass http://$VIRTUALSERVER_IP:81;
        }

to forward all requests to apache so that .htaccess files work fine! As you see, I have used $VIRTUALSERVER_IP:81 instead of 127.0.0.1:1080 because virtualmin creates apache virtual servers with the same IP as the dns IP of the domain. You can easily modify the server template and update apache port to listen on the port 81 though.

Jamie are we going to implement this nice feature? Plesk and Directadmin both have this feature with mod_aclr2! This can go far with the new virtualmin PHP-FPM feature to directly serve php files as well! (Plesk has done this!)

Thank you for the testing. I've edited my post to be more "correct", let me know if you spot any other issues.

As long as variable functionality remains solid in Vmin and my script doesn't break, I am happy with how things are now. I don't need that aclr2. I am looking forward to having some variables related bug fixed so I can insert custom instructions, such as static file caching and so on. (opened https://virtualmin.com/node/45060 )

Personally I think nginx is a bit overrated, apache mpm-event + php-fpm should be plenty good for most needs.

Mostafa's picture
Submitted by Mostafa on Wed, 12/28/2016 - 11:48

Sorry I just edited my post. $proxy_add_x_forwarded_for is different from $remote_ip when a X_FORWARDED_FOR header is set by the client (e.g. when using cloudflare for your domains). See this and this

There is another thing here, you should replace

elif    [[ "$VIRTUALSERVER_ACTION" = "DELETE_DOMAIN"  ]]; then

with

elif [[ "$VIRTUALSERVER_ACTION" = "DELETE_DOMAIN" ]] || [[ "$VIRTUALSERVER_ACTION" = "MODIFY_DOMAIN" && "$VIRTUALSERVER_WEB" = "0" ]] || [ "$VIRTUALSERVER_ACTION" = "DISABLE_DOMAIN" ]; then

since the apache virtual host is removed when web feature or the virtual server are disabled

Will note that in my final version.

Thanks, mr Finch

I was searching for similar requirement for Ubantu 16.04 LTS. currently in this post and in github repository it says only for Cent OS. I would like to know is all the steps are same for Ubantu server or do i need to do any additional action??.

I am new to Virtualmin and webhosting, I dont want to mess existing installation.

also would like to know, is this feature already integrated to the new version of virtualmin 5.x or yet to integrate it??

Many thanks