dlostboy (at) lostinfo (dot) com 

Home | Journal | Multimedia | Files | Other | Links | About

      FreeBSD 4.4 HOWTO for CyrusIMAP, a IMAP/POP3 server

UPDATED 2/20/03

So the time comes when you may need a mail server that is scalable, reliable and not the biggest pain to manage. I spent a good bit of time looking at some options, and wound up with Cyrus from Carnegie Mellon University. This is the mail server implementation they use for themselves (from what I've read, around 6,000 mailboxes). The real beauty of this is that you do not need user accounts on the machine in order to deliver to them. This allows you to have a much more secure machine since you don't have 1000's of user accounts laying around. Setups like that usually are called "black box" setups.

For now we'll try getting this to work with CyrusIMAP, Postfix and mySQL as the authentication back end. I chose a mySQL back end so that we can web-manage the user accounts easily (via PHP) and since we can usually get other products to authenticate using mySQL as well (centralization!).

First thing to do is to install Cyrus:

  • cd /usr/ports/mail/cyrus-imapd2; make install
    When that is done, add the user "cyrus" to the daemon group in /etc/group. Now the port won't install all the directorys you need because you may want to change them. Once the directories are set (the defaults should work for you), then
  • /usr/local/cyrus/bin/mkimap /usr/local/etc/imapd.conf
    Doing this will create the subdirectories under /usr/local/etc/imap/user that cyrus will expect to be able to find.

    The next thing we'll need to do is to add the mySQL capability. Cyrus supports PAM (pluggable authentication modules) for building bridges to other systems. So we'll install the pam-mysql port.
  • cd /usr/ports/security/pam-mysql; make install clean
    Once this is installed, we can edit the /etc/pam.conf file that 4.X machines have. We'll do that right after we create a mySQL database for our needs. We'll call it "mailserver" and add 3 tables to it.
    
    	# Our addresses table contains the username@DomainID information we're looking for
    	CREATE TABLE Address (
    	  AddressID bigint(64) unsigned NOT NULL auto_increment,
    	  Target varchar(255) NOT NULL default '',
    	  DomainID int(16) unsigned NOT NULL default '0',
    	  AddressTypeID int(8) unsigned NOT NULL default '0',
    	  DataField text NOT NULL,
    	  PRIMARY KEY  (Target,DomainID),
    	  UNIQUE KEY AddressID (AddressID)
    	) TYPE=MyISAM;
    	# --------------------------------------------------------
    
    	#
    	# Table structure for table `AddressType`
    	#
    
    	CREATE TABLE AddressType (
    	  AddressTypeID int(8) unsigned NOT NULL auto_increment,
    	  AddressType varchar(255) NOT NULL default '',
    	  PRIMARY KEY  (AddressTypeID),
    	  UNIQUE KEY AddressTypeID (AddressTypeID),
    	  KEY AddressTypeID_2 (AddressTypeID)
    	) TYPE=MyISAM;
    	# --------------------------------------------------------
    
    	#
    	# Table structure for table `Domain`
    	#
    
    	CREATE TABLE Domain (
    	  DomainID int(32) unsigned NOT NULL auto_increment,
    	  Domain varchar(255) NOT NULL default '',
    	  UsernamePrefix varchar(255) NOT NULL default '',
    	  PRIMARY KEY  (DomainID),
    	  UNIQUE KEY Domain (Domain),
    	  KEY Domain_2 (Domain)
    	) TYPE=MyISAM;
    	# --------------------------------------------------------
    
    	#
    	# Table structure for table `UserData`
    	#
    
    	CREATE TABLE UserData (
    	  UserID bigint(64) unsigned NOT NULL auto_increment,
    	  Username varchar(64) NOT NULL default '',
    	  Password varchar(64) NOT NULL default '',
    	  DomainID int(32) unsigned NOT NULL default '0',
    	  Enabled tinyint(1) unsigned NOT NULL default '0',
    	  DomainMgr tinyint(1) unsigned NOT NULL default '0',
    	  Comment text NOT NULL,
    	  PRIMARY KEY  (UserID),
    	  UNIQUE KEY Username (Username),
    	  KEY Username_2 (Username),
    	  KEY DomainID (DomainID)
    	) TYPE=MyISAM;
    
    	INSERT INTO AddressType VALUES (1, 'Alias');
    	INSERT INTO AddressType VALUES (2, 'Forwarder');
    	INSERT INTO AddressType VALUES (3, 'CatchAll');
    	INSERT INTO AddressType VALUES (4, 'Multi Recipient');
    	INSERT INTO AddressType VALUES (5, 'Bounce');
    	INSERT INTO AddressType VALUES (6, 'DevNull');
    
    	
    When that is done, you'll need to create a mySQL user named "mailserver". You can give this account a password but in actuality, unless your machine is completely hidden from any user accounts that aren't "trusted", you'll probably be better off without one at all (because you'll have to put it in cleartext in the pam.conf if you put a password). Give it "localhost" only access, with only SELECT privliges to the "mailserver" database. You only need it to be able to pull user/pass combinations. Now add the "cyrus" user to your userinfo table. Be sure to PASSWORD() the password that you put in.

    I mentioned that PAM allows you to bridge systems. What I mean by this in example is that you could install mod_auth_pam to apache and have it poll the same table in order to allow the email customer to have access to a control panel to do things like change their password. Although, I personally would probably use mod_auth_mysql for such a task, it was an easy example.

    To test this system, add a domain (that has it's MX record pointed to your server) to the domains table. Write down the id number that you get. This is what the other tables will reference. Add a username/password (don't forget to use the PASSWORD() function in mySQL to encrypt the password) and set the domain to the number you got from the other table. Then set the enabled flag to 1. Lastly set the domainmgr flag to 1. We'll use this flag to tell us (via web-management) who has access to change the domain and who doesn't. You'll want to add an entry to the addresses table as well so we can test the recieving of mail. "Target" will be whatever is left of the @ sign of an email address. So maybe we can use "test", then set the domain to the number you have from the domain table. Set the addresstype to 1. We'll be using 1 to indicate a local email address, 2 to indicate an forwarder, 3 to indicate a "catchall", 4 to indicate an address we'd like to have "bounce", and 5 for one that just silently gets discarded. The datafield information will change based on the addresstype. For a type 1, it'll be the local username. For type 2 it will be the email address we intend to forward to, and for type 3 it'll be the address we want mail to goto when the left side doesn't match any users we have set up (thus the name "catchall"). The right side of the email address of course is generated from the domainID you put in. Type 4 & 5 require no datafield.

    Back to the PAM configuration: We're going to comment out the line that is already in there that begins with "imap". That line is telling the machine to have any imap server use the passwd file for authentication. We'll replace it with
    
    	imap    auth    optional        pam_mysql.so    db=mailserver table=userdata \
    		usercolumn=username passwdcolumn=password crypt=2 user=mailserver where=enabled=1
    	imap    account required        pam_mysql.so    db=mailserver table=userdata \
    		usercolumn=username passwdcolumn=password crypt=2 user=mailserver where=enabled=1
    	
    Now do the exact same thing with the rows that start with pop. Alot of walkthroughs mention creating a directory to house files with these directives in them. The only problem with that is that the 'man pam' page tells us that if we have such a directory, that /etc/pam.conf gets ignored after that, which means we'll have a helluva time doing things like sshd after that unless we want to create pam files for each service on our machine that uses it.

    While the README to pam-mysql should be adequate in describing these lines, the important parts are the "crypt=2" which tells PAM that we want to use mySQL PASSWORD() type fields in the database, and the line that says "where=enabled=1" which says that we only want positive results for those records that not only match our username/password combination we pass to it, but also have the enabled tick set to 1. This will allow us to control access later in the game without having to delete the account.

  • cp /usr/local/lib/pam-mysql.so /usr/lib/pam-mysql.so
    The file gets put in the right place but pam doesn't look there. *shrug* maybe ldconfig?

    What we need to do now is set the default password for the administrative account.

  • saslpasswd cyrus
    It will ask for a password. Set it to something you won't forget. Now we'll have to add the cyrus user as an administrator in the configuration file. Open up the /usr/local/etc/imapd.conf file and find the line that says:
          admins:
    And change "" to read "cyrus".

    We're getting close now.

  • cyradm -u cyrus -auth IMAP localhost
    You will prompted for that administrative password, then given a "localhost>" prompt. This is where you add email accounts. By simply typing
  • localhost>cm user.test
    You'll create a mailbox for that test user we added earlier.

    As a side note, to delete said user, you will set the "c" flag to their account

  • localhost>setacl user.test cyrus c
  • localhost>sq user.test 1000 (sets a 1MB quota)
  • localhost>dm user.test

    Now edit your /etc/rc.conf file. Edit/Replace the lines with these corresponding entries.

    	daily_status_mail_rejects_enable="NO"
    	daily_status_include_submit_mailq="NO"
    	daily_submit_queuerun="NO"
    	sendmail_enable="YES"
    	sendmail_flags="-bd"
    	sendmail_outbound_enable="NO"
    	sendmail_submit_enable="NO"
    	sendmail_msp_queue_enable="NO"
    	

    Now we'll move on to the postfix setup.

  • cd /usr/ports/mail/postfix; make install clean

    Allow it to "replace sendmail in /etc/mailer.conf"
    With the mailbox created, we can now generate the mysql rows that'll tell postfix what to do with the mail. We'll have to make a virtuals file for postfix using that same data.

    Put a domain name into the domains table. Generate a "test" username record in the username table and specify that it's enabled and that it's domain is the ID number you got from the domains table. Then goto the address table and generate a type 1 email address of info with a datafield of "test". All this will cause email for info@yourdomain.com to goto the user "test" 's mailbox. Run the utility to write out the file.

    Now it is possible to compile in mySQL support into postfix and have it generate virtuals tables from that, but I didn't see where those plugins allowed for quite as detailed a virtual-hosting as my method, nor did it allow for as many types of email addresses as we'll be able to support (regular, forwarders, multi-recipients, catch-alls).

    Next we need to set up the Bounce and /dev/null addresses. Open /usr/local/etc/postfix/aliases and put these lines in it
    	InvalidAddress	|"exit 67"
    	DEVNULL		/dev/null
    	
    Now when you specify addresses with those attributes, they should be written out to the virtuals file like this:
    	my+old_email_addy @ lostinfo.com     InvalidAddress
    	throwaway @ lostinfo.com        DEVNULL
    	
    Which will cause them to perform the requested action. Please note, the spaces are in there to fool email harvesting programs. They shouldn't be there in your virtual file.

    If you use the utility I made or one of your own in order to generate a virtual-users file. Postfix does not need to be restarted to see the changes.

    Now go into /usr/local/etc/postfix/master.cf and find the line after the one that defines "cyrus". It should have an "argv=/cyrus/bin/deliver". Well, clearly cyrus is not in the root partition. Change this to say :

    	user=cyrus argv=/usr/local/cyrus/bin/deliver -e -r ${sender} -m ${extension} ${user}
    	

    Now open main.cf and uncomment the line that says "mynetworks_sytle = host". Now find a line that mentions "virtual_maps" and add a line under it that says "virtual_maps = hash:/usr/local/etc/postfix/virtual". This will let us use the virtual users file we'll create. Search for "alias_maps =" and add this line- "alias_maps = hash:/usr/local/etc/postfix/aliases". Scroll down a bit and find alias_database and use this line- "alias_database = hash:/usr/local/etc/postfix/aliases". Now go uncomment the line that says "#mailbox_transport = cyrus".

  • #/usr/local/etc/rc.d/postfix.sh start
  • #/usr/local/etc/rc.d/imapd.sh start
    Now configure your email client to use your machine's IP and to use IMAP. You should be able to send yourself an email and then check the mail. Congratulations!

  •  
     

      ©2000, ©2001 LostInformation