Rebuilding from the ground up - looking for advice.

Sorry, I missed this ticket initially.

So if you want to do webhosting, Virtualmin is the package you want. However, we do have a free GPL version that meets most needs.

I'm a little confused as to what kind of webhosting setup you want to do though. Do you want to have multiple sites on the same IP, but served by different machines?

In that case, what you'd want to do is install Virtualmin on the system with the external IP, create your domains, and set each up to proxy traffic through to the appropriate backend VMs.

Yeah like Jamie said, this is what you would do:

• Create all the different VMs you want, setting up all of their various services and hostnames. I.e. one might host "mail.blah.com:80" and another might host "git.blah.com:80" and so on.

• Create one VM, or use the cloudmin master, to install Varnish or Nginx, the two highest-performance proxy servers in the world. I would probably pick varnish because it's natively capable of doing memcaching (nginx requires the memcached module for that) of static resources like javascript and images and such, and will only forward requests for dynamic resources (like PHP) over to the backend servers, saving you a lot of CPU time.

• In the chosen proxy config, set it up so that i.e. requests for "mail.blah.com:80" are proxied to the internal "10.0.1.1:80" machine, "git.blah.com:80" is proxied to the internal "10.0.1.2:80" machine etc.

• This setup is known as reverse proxying, so now you know what to search for.

• Bonus: coincidentally, this is the same kind of setup you would do if you wanted to do load balancing. In that case, you would set up a central SQL server which stores all data + all PHP sessions (many apps already use SQL for their session storage by default, i.e. Roundcube and piwik and will work out of the box), then set up the same set of websites on every VM (you can do this by creating 1 VM template and then launching multiple copies of it), so that they all contain the website contents for mail.blah.com, git.blah.com, etc. Then, put your load balancing proxy server in front of them and tell it to send the request to a random internal server. The servers in turn will be talking to the central SQL backend, ensuring that all of the servers have the same "state" and are equally suited to handling the request.

Good luck and have fun! :)

Correct, you make your DNS point to the Varnish machine, and the Varnish machine in turn forwards the request to the internal machine (and sends some headers containing the IP of the actual person that wanted the page, so that your site knows who accessed it from outside the proxy).

Look for Varnish load balancing / reverse proxy guides. I suggest the Load balancing setup, in which case you would have the same sites on every VM. It ensures that load is spread evenly rather than all mail users hammering the mail machine and all git users hammering the git machine, etc. The only thing you need to do is take care that all sites are configured to read/write data from a central MySQL database so that they are all equally capable of handling any request.

There will be some extra initial work to set up all systems so that every website speaks to the central SQL server, but after that you're in heaven because all of your machines can be based on 1 single VM template that contains all software, and you can effortlessly spawn new VMs from the template if you need more processing power for your sites. When updating software, you simply need to update that single template, and you then re-spawn new VMs from it, killing off the old and outdated ones.

This of course depends entirely on how busy your projects are, but for heavy sites it's absolutely worth doing load balancing and not just reverse proxying.

You're welcome.

Also, you don't need two NICs, as long as you put the Varnish proxy behind a router:

             Internet
                |
              Router ____
              /      \    \
         Varnish    App1   App2

Requests would go into the Router, which passes it on to the Varnish server, which speaks to the App VMs over the local network.

Oh, so every VM is hosted on one machine? That's awfully inefficient in a proxying situation, especially in your example graph where traffic would come in on the physical adapter, get translated and enter the Varnish VM, which then goes back out to the virtual network adapter and then back into itself on a different VM, which then returns the request out to the virtual network adapter again, and then from there back into the Varnish VM, and then from the Varnish VM back out onto the real network. Luckily, some of these steps can be taken out of the equation (I'll show you how).

I assume you're only running multiple VMs for a sense of security and isolation between sites, then? Or perhaps you're offering people their own VMs all running on your machine, and you want a way to direct traffic to the correct machine? Those are the reasons I can think of why you'd be running different VMs per-site, instead of simply having every site on a single VM. If it's only your own sites, then I suggest that you run every site inside one VM and avoid all the headaches.

Anyway, whatever your reasons may be, here's a setup that would work for you and is as efficient as possible given the circumstances:

             Internet
                |
       Varnish + Cloudmin (host)
           /         \
        VM1           VM2

Configure the VMs to use Host-only Networking, which means that the host and its VMs can speak to each other but the VMs don't have any presence of their own on the physical network adapter/network. From there, your host machine runs Varnish which looks at the HTTP/1.1 "Host" header to determine which internal VM to send the traffic to.

Basically, what you do is define a network bridge adapter that isn't tied to any physical network card, then tell your VMs to use that. This will allow your host and VMs to speak to each other via their own little virtual network, without exposing the VMs to the physical network whatsoever. Just make sure to assign an unused IP range to the virtual network, and a netmask that won't clash with your physical network. That's easily done by choosing a start-address that's higher than what you use on your own network. I.e. your physical network may use 10.0.0.0 to 10.0.0.255, so you would simply assign the virtual network on the 10.0.1.0 to 10.0.1.255 range, or any other private, unused range.

The other issue is that your VMs will have to be able to make outgoing connections to the internet for various features like DNS lookups and whatever. For that, your host machine will have to act as their NAT/Gateway.

You're getting into advanced and quite inefficient routing territories here, and should also be asking yourself if you could simply move all sites to a single VM, with unix User permissions handling the user isolation and security.

Oh so you do have multiple machines, and a very good reason for that as well. Good, then let's proceed!

Since they are actually physically separate machines, you are definitely going to need a router so that you can have all machines and VMs in a local network address space, capable of speaking to each other. The router will be responsible for sending incoming port 80 (and other) traffic to the Varnish machine.

This is the setup for you:

              Internet
                 |
      ____ Router (1.2.3.4) ____
    /            |               \
   VM1     Cloudmin+Varnish      VM3
                 |
                VM2

Each VM is bridged to the local network, with its own internal address, such as VM1=10.0.1.1, VM2=10.0.1.2, VM3=10.0.1.3

The Cloudmin hostmachine speaks to all of them via those IPs for control purposes.

You run Varnish directly on the Cloudmin host (not in a VM), to cut out a few routing steps.

The DNS for all websites point to the router's IP (1.2.3.4).

The Router directs incoming Port 80 (web) traffic to the Varnish host.

Varnish looks at the "Host: example.com" header and decides which VM (10.0.1.1, 10.0.1.2 or 10.0.1.3) to send that request to.

You only need 1 physical network adapter per machine, since the router handles the issue of where to send the traffic.

For your needs, even a $50 home-grade router with a big routing table would suffice, like a DLink DIR-655 gigabit router. Just any router with enough memory to keep track of all the open connections and their sources/destinations. You only need enterprise-grade routers if you want to have multiple VLANs (meaning different LANs for different ports), if you have more than a gigabit connection, or if you need more than 4 physical ports. Otherwise the DIR-655 will suit your needs as it has enough memory to handle all the connections to your sites.

I hope you have a nice trip! :-)