I want to acknowledge the huge help that Andre LeBranche of Apple has been with this issue. You rock!

Here’s what we were trying to do-
in Lion Server v10.7
Get iCal server to send and receive invitations via email
to people not on the local network
from a server on a private network (no FQDN)
where the clients email host does not support + (plus) addressing

I’ve been working on this for an embarrassingly long time, and now I think it’s worthwhile to write down the conditions required to make it work. First of all, there’s been a large change since 10.6 Snow Leopard- you no longer require Open Directory, DNS working in forward and reverse and a mail server running on your server. In theory this makes it much easier, because the Server.app now has the ability to use an email address that you set without a lot of other conditions.

But I just couldn’t make it work. So what specifically was wrong? Digging into the log files gave this-

2013-07-17 13:14:07+1000 [-] [caldav-0]  [PooledMemCacheProtocol,client] [twistedcaldav.scheduling.imip#error] Could not do server-to-imip request : <twistedcaldav.scheduling.imip.ScheduleViaIMip object at 0x10213e910> iMIP VFREEBUSY REQUESTs not supported.

However Andre says this is a message saying that the invitees address is not recognised, and thus the server cannot set free/ busy status for them.

Plus Addressing
The next hump we covered was that when creating an event, the iCal server was not sending any emails to the invitees. It turns out that we had set up an email address for the iCal server at our host, but the host does not support + addressing. Plus addressing works like this-

test@email.com    is equivalent to      test+5A349789-E3DF-447C-B6A7-1F6B3E078DEF@email.com

ie. Everything after the plus sign and before the @ is ignored. The server treats those emails in exactly the same way, but they contain extra information and it does not need to be added as an attachment. This trick is useful because the iCal Server can send an email with a unique identifier in the email address, receive it back and make decisions on it’s content.

It works like this- in the above example the iCal server has created an event with a unique identifier of 5A349789-E3DF-447C-B6A7-1F6B3E078DEF
So it can send an outgoing email with the from address test+5A349789-E3DF-447C-B6A7-1F6B3E078DEF@email.com with a .ics (calendar) file attached
When the person at the other end adds the .ics to their Calendar program and marks ‘Accept’ ‘Decline’ or ‘Maybe’ that triggers an email back with your answer to that long email address (if the client is IMIP compatible). The iCal server picks it up, figures out that it’s for an event it has in it’s database, then scours the email for acceptance details and organiser details so it can update the event- ie. reads the .ics file.

So what did we learn?
Firstly- that this may work without setting up Open Directory, but don’t bother. I was able to replicate some of the issues on a machine without setting up OD, but can’t guarantee that this will work without it. I’m not going to demote the server to test. Lot’s of services like a properly setup OD so you may as well do it.

Second- there is some information around on the internet about using a GMail account for your calendar. This is good advice because GMail supports + addressing, but I never got this to work properly. This could be because of the type of send authentication required (no one was able to answer that definitively, when you set up email invitations in iCal Server it offers you a bunch of authentication types which don’t match what Mac Mail offers) or possibly because these odd emails were triggering a defensive firewall at Google. One person replying sent me to a GMail unblocking site, which said it was ‘now unblocked’ but who knows if it was previously blocked? Use an iCloud address- they support + addressing and they are free and easy to set up. You will need to do this from the System Preferences/ iCloud pane.

Third- DO NOT use a personal email address. Set up a special address only to be used by iCal server. The iCal server is designed to keep the inbox clear by deleting everything it finds in your inbox. You have been warned!

Fourth- in Server.app when you set up a user, there is a space for an email address- DON’T add one! Strange as this sounds, if you add an email address in here the iCal server will use it as the ‘from’ address when speaking to the mail server to send you invitation, and the server will reject it because the name does not match the sending email address (your special server email). This is probably a bug, because here’s a huge gotcha- if you leave out this address, the iCal server will ignore emails from your invitees with their acceptance details on them. So you’ll never get a reply. And this is where we are currently stuck.