How to use only subdomain in virtualmin

Virtualmin doesn't care what you name your domains. Really, does not differentiate between "sub-domain" and "domain" in any way; those are just names.

Just create a new virtual server with the subdomain name. It'll Just Work, assuming you've delegated DNS appropriately. If you want to host DNS for both the domain and the subdomain on your Virtualmin host, you can do that, too. Just create the zone for the domain in the BIND module, and you can still create the virtual server account in Virtualmin for the sub-domain.