migrating virtualmin from system logins to LDAP in order to free email addresses

Moving to LDAP won't help here .. it has the same limits on name length.

What is the error message you are getting exactly, and what alias name are you trying to create?

OK, so you are hitting the 16 character username limit on FreeBSD .. which is non-trivial to change.

One solution though is to have Virtualmin use a shorter suffix for usernames, instead of the full domain. On the domain creation form, this can be set at Advanced options -> Prefix for mail usernames.

Virtualmin currently doesn't support Postfix virtual users, as it needs mailboxes to be real Unix accounts so they can have quotas.

A few FreeBSD users have rebuilt world with longer username support, and have talked about it in the forums. I think I also have notes about how to do it, written up by a third party, somewhere in the docs, but it may not have made it over from the old website, depending on whether it was actually linked from any of the indexes.

Anyway, on Linux systems this limit is not an issue, out of the box, so it rarely comes up; FreeBSD only represents about one of out of 1000 users of our userbase. Moving to virtual users loses you a lot of other nice capabilities, so I'd tend to recommend avoiding it (which is why Virtualmin doesn't support them in Postfix; it does have virtual user support in qmail, but that's a huge bunch of tradeoffs).

So, I kinda think rebuilding with longer username support is the best solution here.

Unfortuantely, there's no built-in support in Virtualmin at the moment to change the prefix for an existing domain - unless it doesn't have any users yet, in which case you can do it on the Edit Virtual Server page.

Are you sure you would want to do this though? Renaming existing mailboxes would confuse users, as it would break their SMTP and IMAP settings..

Wow this is a pretty big issue when considering cloudmin/virtualmin for a sizable hosting environment deployment. I've been spending some time trying to figure out how a cloudmin setup could replace an HSphere deployment and virtual mail is always at the top of my list when evaluating new platforms as we really struggled with this back in my ISP days of the mid- to late 90's. Then eventually built our own system to handle our 250,000 customers spread across various domains from acquired companies.

I understand the challenges of migrating from a single server solution to a distributed system design where mail is likely to be serviced by a system other than the web server, where spam/virus filtering may occur on a proxy, etc., and yet we still want to support quotas and user-configured filter settings. But surely there should have been more progress at this point.

Instead of making up new unique usernames with a combination of various letters attached, couldn't we at least move to a system that supports "USER@DOMAIN.TLD" as the account name? I understand Linux doesn't support use of the @ symbol in usernames, but it does seem to allow the underscore character (as well as the dot) so you could do "USER.DOMAIN.TLD" and then possibly provide a migration path to real virtual users down the road (since the domain info is already attached to the username). A system like this would provide more natural login usernames across all services (since they very closely mirror the user's email address) vs the current use of username plus parts of the domain name or some other prefix.

Instead of making up new unique usernames with a combination of various letters attached, couldn't we at least move to a system that supports "USER@DOMAIN.TLD" as the account name? I understand Linux doesn't support use of the @ symbol in usernames, but it does seem to allow the underscore character (as well as the dot) so you could do "USER.DOMAIN.TLD" and then possibly provide a migration path to real virtual users down the road (since the domain info is already attached to the username).

That's a misunderstanding. Users with @ in them are supported by Linux and Virtualmin. Postfix does have an issue with them, but Virtualmin works around the issue by creating a second user just for Postfix with the same UID/GID but using a different separator.

There's a FAQ about it here:

http://www.virtualmin.com/documentation/email/faq

It's the second item titled "What's the deal with @ in mailbox usernames?"

Ah. A misunderstanding. Venema is absolutely correct that "@" shouldn't appear in usernames. In otherwords, what we don't want is:

firstname@lastname@DOMAIN.TLD

In uucp days one might have had...

localuser!user@DOMAIN.TLD

I'm not proposing either of these methods be used. The problem at hand is trying to figure out how to support virtual users (at least for mail) without having to re-invent the existing quota mechanisms and ideas of user storage space for sieve filters or whatever--these are functions you WILL have to reinvent if you are going to (for instance) support virtual users all running under one userid (the typical virtual user setup).

So at the moment you have the existing setup where userid overlap is prevented through the use of "Prefix for mail usernames". That results in system-wide unique names but it also creates a "weird" naming scheme that doesn't map to anything because it is random.

In contrast, what I was suggesting is providing the capability (perhaps as an option) to create UNIX usernames of the format:

user.domain.tld

...instead of the current "Prefix for mail usernames" setup.

NOTE, the are NOT EMAIL ADDRESSES, but UNIX usernames. "user.domain.tld" would map to the email address of "user@domain.tld" which means we now have a unique UNIX username that can be easily translated into an email address without having to do any other lookups.

So the MTA would have to be able to support this mapping. It would have to be able to convert the email address "user@domain.tld" to the username of "user.domain.tld" for purposes of mail storage, procmail filtering, or whatever (I haven't looked that far into the mail capabilities of this system yet so I don't know how vacation messages and the like are handled yet).

I know this type of conversion can be easily handled in Exim (which I've been using for a billion years now) and I suppose it can be accomplished with other MTAs. I'm not sure if the POP3/IMAP4 servers are going to care because the user will login with "user.domain.tld" and that is the username so that is where those servers will look for mail storage.

As to how this relates to LDAP, well, once you can support that username format, I don't think LDAP would have any problem with UIDs in that format (we currently use firstname.lastname as UIDs in LDAP and it works swell with the Courier IMAP/POP daemons).

Finally, if Cloudmin later supports greater mail user virtualization -- perhaps a system that DOES run under a single userid -- then it should be a trivial task to convert all of those user.domain.tld usernames to "user@domain.tld" if a user is able to later login with "user@domain.tld" OR "user.domain.tld."

Hope that clears things up.

Oh, and Linux most definitely doesn't support UNIX usernames with "@" in them:

[root@dhcp210 ~]# useradd -m admin@fuzzy.com useradd: invalid user name 'admin@fuzzy.com'

[root@dhcp210 ~]# useradd -m admin\@fuzzy.com useradd: invalid user name 'admin@fuzzy.com'

[root@dhcp210 ~]# useradd -m admin.fuzzy.com [root@dhcp210 ~]# id admin.fuzzy.com uid=504(admin.fuzzy.com) gid=505(admin.fuzzy.com) groups=505(admin.fuzzy.com)

-mark

The useradd command may not support usernames with an @, but you can add them to the /etc/passwd file directly with no problems. Try this out in the Users and Groups module in Webmin, which adds to files in /etc directly without using the "useradd" command. We have plenty of Virtualmin customers using usernames in this format..

I believe you. But I thought that one of the wonderful things about Webmin is that it preserves compatibility with the standard *nix toolchain. If the "@" sign isn't supported by the useradd command, then what other tools don't support it?

I'm just trying to theorize on a way to enable the notion of "traditional" virtual user paradigms within the existing "every user has an account on each system where they have services" system. But after having re-read the previous replies (above) it's clear that even the "user.domain.tld" format won't solve anything if the underlying system can't handle usernames of less than say 255 chars (and as noted above, FreeBSD only supports 16 chars).

Perhaps the only solution really is to implement a real virtual user architecture backed by either LDAP or SQL.

I believe you. But I thought that one of the wonderful things about Webmin is that it preserves compatibility with the standard *nix toolchain. If the "@" sign isn't supported by the useradd command, then what other tools don't support it?

We agree with you, which is why we don't recommend it.

Perhaps the only solution really is to implement a real virtual user architecture backed by either LDAP or SQL.

Interesting logical leap you've made there. ;-)

There are many username formats supported by Virtualmin, which do not impose the additional complexity and reduced feature set of virtual users. (And I deeply hate MySQL-backed virtual mailbox architectures; they're so wrong-headed it hurts my brain to think about it. LDAP is great, but I still don't like virtual users.)

I haven't actually run into any issues with @ usernames in other Unix tools, and plenty of Virtualmin users use them - the only real issues come from Postfix, which cannot differentiate them from email addresses in map files. Things like quotas, procmail, spamassassin and so on all seem to work fine..

And I deeply hate MySQL-backed virtual mailbox architectures...

Surely you jest. How else could one deploy and manage a very large number of users across a distributed system? The user account system was never meant to handle the concept of virtual users (aka "groups of unrelated users"). Now that you're all headed down the "Cloudmin" path, I would suggest thinking about the scalability of the Cloudmin/Virtualmin/Webmin architecture with regard to domain spaces and cluster size.

The original ticket owner is concerned about growth prospects and subsequently thought LDAP might help. Ideally, any directory service (whether its LDAP or sql based) should solve this problem provided that the backend is designed correctly. IMHO, a correctly-architected system (that is, one that was designed from the ground up to support virtual users) would completely treat the idea of a user account as an abstraction. Once that's been done, moving a user from one system to another or from one system to a load-balanced cluster should become a lot easier if you also leverage the concept of virtualized storage. "A lot easier" in the sense that there would be no potential for account conflicts and data would not have to be copied from one node to another, thereby making the relocation practically instantaneous (this is a big deal for users that tie up a lot of storage).

At the very least, if you are going to continue to support the allocation of real user accounts, mathematically you should strive to allocate user names like (aaaaaaaaaaaaaaaa, aaaaaaaaaaaaaaa1...z9999999999999999) to maximize the number of accounts that could be configured across the entire cluster (this is for 16 char user names). But the problem here is that now people will have to configure their mail clients (for example) to login with these awkward user names. Even more awkward than mangled user+domain name combinations.

True virtual user systems, on the other hand, support user@domain.tld logins which in themselves are necessarily unique in the entire known namespace. You might not like it, but it's the only way to support true virtualization in a hosting environment. Come to think of it, it's the only sane way to support any large deployment at all (think of a large enterprise deployment for example).

My comments to this issue are strictly related to the possibilities of Cloudmin deployed as the control panel for a large distributed hosting panel where we'd want to be able to easily migrate customers (internal and external) between traditional virtual servers (e.g. virtualmin shared servers) and virtualized servers (Xen containers), while centralizing all mail, dns, and sql on dedicated clusters for each service. We can do all of this now with our HSphere deployment but are looking to migrate to a better-maintained and possibly EVEN MORE flexible platform (as are many HSphere operators); if Cloudmin could do all of these things AND also offer support for migration between traditional and Xen-based (or similar) instances, this could be really cool to say the least.

These will be my last comments attached to this issue as I'm taking it far off track. I think it's pretty clear how Virtualmin is currently limited with regard to providing support for a large number of users. The only thing I'll add is stop looking at this as developers and look at it from the standpoint from your customers and what their needs are. These types of products always get screwed up when they're guided by developers who believe they know best (despite industry trends) vs customers who actually deploy systems in real environments.

In short: system user accounts made sense up until the late 90's when the concept of virtual users really gained traction; user accounts only make sense now for very small deployments and the only reason we still see them so much is because there are a tremendous amount of legacy deployments out there...even the mighty HSphere doesn't support true virtualization when it comes to allocating user space (while it doesn't allocate user accounts for mail, it does allocate accounts for web space)...but then, that platform is now ancient anyway.

Virtualmin does support virtual user accounts partially - you can use vpopmail and Qmail+LDAP already, neither of which use real Unix users for mailboxes. However, they aren't very commonly used, as most customers are more into web hosting than email..

But support for something like Postfix+MySQL or Postfix+LDAP could certainly be added. I'd be interested to hear which particular combination has worked out best for users who manage a large number of mailboxes.

I currently use Postfix+MySQL for a large client of mine and most of it I had to build manually because VM did not support it the MySQL end of it.

I liked the MySQL part because I could assign that DB to the user and any backups would ALWAYS have the database as opposed to LDAP which potentially could be lost if a system burned.

I would love to see Postfix+MySQL support in VM Pro and would use that as the default setup for all my clients.