Replacing MS Exchange.
Role: Project Manager
Starting Situation
In 2013, the client operated an e-mail system for the education sector which was comprised of an on-premise Microsoft Exchange cluster (approx. 12.000 mailboxes for staff) and an open source mail system using mainly Horde and Cyrus (45.000 mailboxes for students). Perdition was used as „glue“ between the two systems, so they would appear as a single system under a single domain.
The Problem
Correct licensing was a major problem, since at that time client access licences (CAL) were required for the end users (in addition to Exchange server licenses), and those CALs would have to have been provided by different entities, depending on the end user’s employer which could belong to the federal, stare or municipal level, or even private entities. There was no system in place which allowed an accurate overview in near real-time.
Another problem was that the system lacked groupware features like shared address books and calendars (across subsystems) which hindered effective collaboration.
Furthermore, at the time, both mail systems were nearing their end-of-life.
So, it was decided that a new mail system was needed which could house all existing mailboxes, provide groupware features and could accommodate an expected growth in users over the next 6 years. Furthermore, taking inspiration from projects like LiMux, another goal was to eliminate the technological lock-in (concerning Exchange and MAPI) and to save on costs.
The Challenges
This project had a very diverse set of stakeholders across various entities, each with their own interests and technical preferences. Therefore, tactful communication and consensus building – key success factors in any project – were of paramount importance here.
The biggest technical challenge was the question of how to replace Exchange’s MAPI protocol that was used by approx. 1.000 power users were using. Due to the licensing issues, moving all users to an Exchange environment was not an option. In 2013, the list of production-ready MAPI replacements was rather short. There were some solutions that used „connectors“ that needed to be installed on the clients, but this was not a viable option due to limited capacity in the client’s support structure.
Another challenge was designing the migration process such that there were minimal down times and service interruptions for the end users. A „big bang“ migration was out of the question due to the size of the user base and the operational risks involved. So, a co-existence of new system and the legacy systems was necessary for the duration of the migration.
Finally, actually building the new system was a challenge. It would have to be redundant such that the loss of a data center would not impact operations, and it would have to scale to a 100.000 users. Thankfully, the client was able to staff the project with excellent people: sysadmins, network experts, software engineers and customer support agents, who could get the job done.
The Solution
After surveying possible Exchange alternatives available at the time, we settled on three systems that we set up and tested under near-real conditions. From those, the one that best satisfied all requirements (licensing, pricing, scalability, features, etc.) was selected.
Then, we set up a test stage system and developed migration tools that were capable of migrating Exchange user’s mails, calendars, contacts and to-dos. Stakeholders and power users were invited to test and give feedback.
The production system was set up alongside Exchange and Horde as third subsystem behind the Perdition IMAP Proxy, thus enabling smooth co-existence. The client’s central identity management system was used to keep track of whether a user’s mailbox lived on subsystem one, two or three. At this point, we were able to move users seamlessly across the subsystems.
The migration tool would update the identity management system after migrating a mailbox, so that the user would subsequently be routed to the new system. Thus, the migration was transparent for POP3/IMAP+SMTP — the majority of users. MAPI users required a more creative approach: their clients were re-configured to use ActiveSync instead.
Next, we defined a set of pilot users to be migrated in production. After further quality assurance, it was time to move the remaining users, which was done in chunks of several thousand users each during predefined maintenance slots.
At the time of writing (Mar. 2021) the system’s user base has grown to more than 80.000 active users.