The Regional Government of Andalusia in Spain has been using mail systems based on open source software since 2001. The main driver was cost: Exchange for many tens of thousands of users is breathtakingly expensive.
Our work here follows the decades old Unix way: you make small things that perform simple tasks, and then you assemble them as needed. The economic gains relative to proprietary solutions must be huge. Vendor proposals ended prematurely when we mentioned that our present costs were around 4 euro per user per year.
Policy Context
The Regional Government of Andalusia in Spain has been using mail systems based on open source software since 2001. In that year, 'IT Coordination' became a General Directorate, explains Juan Conde, Chief of Staff for the Promotion of Free Software at Junta de Andalucía. and started setting up corporate services for communications, web and e-mail.
Until then, each regional ministry and public agency in Junta de Andalucía had had its own subdomain for e-mail, and each one ran its own mail server based on a mix of proprietary and open source products — mostly Exchange and Sendmail. The new corporate service provided mail functionality to all public employees — about 120,000 at that time — under a single domain: juntadeandalucia.es. Public agencies had the choice of using the corporate mail servers, or keeping their own, using the corporate servers as gateways to deliver and receive messages.
Cost
Mobile integration was not a problem, says Conde. We even had Blackberry systems integrated with our e-mail platform. The main problems with our mail system were shared calendars and, of course, inertia. The massive need for mobility solutions only became relevant over the past few years. The main driver, however, was cost; Exchange for many tens of thousands of users is breathtakingly expensive.
Junta de Andalucía officially prefers open source, a position it first set out in 2003. We announced, but never published, an open standards decree in 2006, four years before Spain published its national decree for interoperability. But the main reason was that it was absolutely out of our budget to provide e-mail services to all public employees — currently about 250,000 people — the way we were doing it at the time.
Description of the way to implement the initiative
The first corporate deployment was based on Oracle eMail Server (OES), Conde continues. That system had so many issues, fortunately, that it didn't last long. Messages were kept in a database, creating an unjustifiable bottleneck; addresses were limited to 24 characters, excluding the dots; the webmail initially was set up from sample code provided by Oracle; and so on.
The need to manage message routing and to reduce costs made us switch to an LDAP-based routing system, initially using Sendmail, the OpenLDAP directory server and the SquirrelMail webmail interface. We used this configuration for several years, until performance issues made us consider other open source servers. That is when we chose Exim, in particular for its efficient integration with LDAP.
Over the years, public organisations have progressively moved from their particular solutions — whether proprietary or open source — to the corporate mail service. Presently, proprietary mail products are in a minority, mostly used where shared calendars and other integrations are complex or expensive to migrate.
Technology solution
The services currently offered are:
- e-mail,
- contacts,
- instant messaging,
- calendaring,
- task management,
- file transfer, and
- collaboration tools (Red Professional).
Since 2012, the mail/messaging infrastructure has been based on these components:
- most servers run CentOS Linux;
- service load balancing: proprietary hardware;
- SMTP server: Exim v4;
- IMAP/POP server: Dovecot v2;
- webmail: Roundcube and SquirrelMail;
- search engine: Solr v3;
- mailing lists: Sympa v6;
- administrators' interface (per organisation): developed in Java, published as open source;
- automated account creation: developed a specific connector to link to the human resources system;
- anti-spam: SpamAssassin 3.x;
- anti-virus: Kaspersky 5.5;
- directory server: OpenLDAP 2.3;
- address book, calendaring and tasks: Agata, our own open source development, provides a web interface and integrates with desktop clients like Thunderbird and Outlook using *DAV or SyncML;
- instant messaging: ejabberd (XMPP-based); authentication through OpenLDAP; integration with Roundcube webmail through a plugin; developed a simple Android/iOS app based on Xabber;
- file transfer: a service to link large files instead of sending these by e-mail, based on an open source software previously developed by the University of Seville;
- collaboration: We have set up a corporate social network called 'Red Professional', based on Elgg, Etherpad and BigBlueButton. It is being used as a 'self-service' social network: anyone can create a group — open or closed — and invite other public employees or external staff. Currently it serves as an intranet platform for two organisations. It provides blogs, fora, messaging, surveys, brainstorming, shared pages (like wiki), online shared documents, online meetings (webinars, i.e. web conferences), files, bookmarks, news, and group activities. It is being used by more than 5,300 people.
As you can see, we work in the decades-old Unix way, says Conde. You make small things that perform simple tasks, and then you assemble them as needed. We have no written high-level strategy in this regard. I think we were lucky to have more hackers than mouse-clickers when we designed the system. Moreover, when we started there were no monolithic solutions available.
Technology choice: Standards-based technology, Open source software
Return on investment
Service levels
The minimum service levels for the messaging infrastructure set by the Regional Government of Andalusia are:
- availability:
- SMTP/POP/IMAP: >99%
- webmail: >95%
- problem resolution times:
- high severity: <6h (24x7) >95%
- medium severity: <14h (24x7) >80%
- low severity: <26h (8x5) >50%
- monitored services:
- SMTP/POP/IMAP/webmail response time: <5 seconds
These levels are easily met, as last year's (Oct 2014 — Sep 2015) availability statistics show:
- SMTP/POP/IMAP: >99,99%
- webmail: >99,99%
These numbers relate to the Service Level Agreement (SLA) between Consejería de Hacianda y Administración Pública (the regional ministry responsible for the corporate IT services) and Sandetel (the IT company owned by the regional government). They are measured at the central messaging infrastructure, not at the service recipients' premises, that way isolating the service measurements from possible networking malfunctions.
According to Manuel R. Saez, ICT specialist at the Cabinet of Innovative Projects in Junta de Andalucía, the public agencies are not charged for the service.
Huge economic gains
We have no figures to compare, says Conde, but the economic gains relative to proprietary solutions must be huge. Vendor proposals ended prematurely when we mentioned that our present costs were around 4 euro per user per year!
In 2012, the usage numbers of the corporate infrastructure were:
- e-mail accounts: 234,000 (164.000 centralised, 70.000 distributed);
- messages: ~1,000,000 messages per day (Monday-Friday);
- traffic: 425 Gbyte per day (Monday-Friday);
- POP connections: 825,000 per day (Monday-Friday);
- IMAP connections: 320,000 per day (Monday-Friday);
- webmail: 1,350,000 connections per day.
And the costs involved:
- development and maintenance: €176,000 per year;
- housing and system administration: €690,000 per year;
- hardware: €100,000 per year;
- technical support: €17,500 per year, excluding end-user support;
This yields a total cost of €983,500 per year, or €4.21 per user per year.
Lessons learnt
Scope: Regional (sub-national)