Android 4.4 now *can* sync multiple calendars via ActiveSync

Android_robotWith the release of Android 4.4 called KitKat Google made some interesting changes to their ActiveSync implementation: the code is now set up to sync more than one calender, and the first KitKat user already confirmed that new feature.

In February I described in a blogpost why Android cannot sync multiple calendars via ActiveSync. The problem was that Google did not implement the necessary parts of the ActiveSync specification in Android.

However, that seems to have changed: if you look at the current ActiveSync implementation of Android 4.4 KitKat, the source code (tag 4.4rc1) does list support for multiple calendars – and also for multiple address books:

        MAILBOX_TYPE_MAP.put(Eas.MAILBOX_TYPE_USER_CALENDAR, Mailbox.TYPE_CALENDAR);
        MAILBOX_TYPE_MAP.put(Eas.MAILBOX_TYPE_USER_CONTACTS, Mailbox.TYPE_CONTACTS);

I had no chance yet to test that on my own, but there are reports that it is indeed working:

Today i flashed a Android 4.4 Rom on my smartphone. After adding the Exchange Profile all my Calendars are there […]
I’ve uploaded a screenshot here:
http://postimg.org/image/5d4u364ub/

Looks like Google actually listened to…erm, corporate users? At least to someone, though 😉

But: Since I have no first-hand-experience in this regard I would like to ask all of my nine readers out there if anyone has a stock KitKat running and if the could check this feature. Please test this and leave a report about your experiences in the comments. I will include it in the article.

By the way, the above mentioned source code snippet also tells quite exactly which other ActiveSync functions are not yet supported in Android:

        //MAILBOX_TYPE_MAP.put(Eas.MAILBOX_TYPE_TASKS,  Mailbox.TYPE_TASKS);
        //MAILBOX_TYPE_MAP.put(Eas.MAILBOX_TYPE_NOTES, Mailbox.TYPE_NONE);
        //MAILBOX_TYPE_MAP.put(Eas.MAILBOX_TYPE_JOURNAL, Mailbox.TYPE_NONE);
        //MAILBOX_TYPE_MAP.put(Eas.MAILBOX_TYPE_USER_TASKS, Mailbox.TYPE_TASKS);
        //MAILBOX_TYPE_MAP.put(Eas.MAILBOX_TYPE_USER_JOURNAL, Mailbox.TYPE_NONE);
        //MAILBOX_TYPE_MAP.put(Eas.MAILBOX_TYPE_USER_NOTES, Mailbox.TYPE_NONE);
        //MAILBOX_TYPE_MAP.put(Eas.MAILBOX_TYPE_UNKNOWN, Mailbox.TYPE_NONE);
        //MAILBOX_TYPE_MAP.put(MAILBOX_TYPE_RECIPIENT_INFORMATION_CACHE, Mailbox.TYPE_NONE);

I guess syncing tasks could come in handy in corporate environments. Combined with support for multiple task folders you could even design your own Kanban “board” that way.

Nevertheless I’d like to add that ActiveSync is no big deal for me anymore because I am very happy with a – albeit 3rd party and not yet Open Source – CalDav implementation, which can even sync multiple task folders.

Advertisements

Google continues CalDav support for everyone, now also adds CardDav

Android_robotYesterday Google announced that it will not restrict the CalDav access to their calendars to registered partners only, but that they will continue to provide it for everyone. Additionally, Google now offers CardDav support.

A couple of weeks ago Google announced that they would restrict CalDav access to their calendars to registered developers only. That resulted in a huge uproar among developers, users and open standards advocates and made many people wondering if Google will become a closed standards/software company in the future.

However, the pressure (and most likely the bad press and reputation) Google got worked, and they announced that the CalDav API will be continued as an API open for everyone:

In response to those requests, we are keeping the CalDAV API public.

And it becomes even better: CardDav support is added as well, meaning the address data can be accessed via open protocols as well:

And in the spirit of openness, today we’re also making CardDAV – an open standard for accessing contact information across the web – available to everyone for the first time.

This way CalDav and CardDav have an even better chance to become THE royalty free and open alternative to Microsoft’s ActiveSync protocol. Additionally, application developers don’t have to worry to add special code to support Google calendars and address books: they just add CalDav and CardDav support and they automatically support almost all groupware servers and services available.

This is good news and gives me back some trust in Google’s policies and priorities. There is still no CalDav or CardDav support in Android, yes – but at least the server side is better now.

[Howto] Sort spam mails to the junk folder in Zarafa

Tux
Groupware setups like Zarafa often leave the spam scanning to external tools – but still has to sort the tagged mails. This article shows two ways of sorting and filtering tagged spam mails in Zarafa.

Imagine the pretty common setup where the actual spam filtering is done on a separate MTA. In such cases the spam is scanned and tagged, but the groupware system, here Zarafa, has to sort the mail to the appropriate folders. For example, an e-mail header spam tag created by Spamassassin looks like:

X-Spam-Flag: YES
X-Spam-Score: 64.448
X-Spam-Level: ****************************************************************
X-Spam-Status: Yes, score=64.448 tagged_above=-999 required=6.2
	tests=[ADVANCE_FEE_2_NEW_MONEY=0.992, ADVANCE_FEE_3_NEW=2.734,
	ADVANCE_FEE_3_NEW_MONEY=0.001, ADVANCE_FEE_4_NEW=0.001,
	ADVANCE_FEE_4_NEW [...]

The question is how the groupware can auto-sort such mails – Zarafa knows (at least) two ways to accomplish that aim.

The first is to use Procmail, Maildrop or similar tools to check the mails for such tags: all mails are delivered from the separate MTA to Procmail, which forwards all mails to the zarafa-dagent. However, while normal mails are just forwarded to the normal zarafa-dagent without any options, spam mails are forwarded with the option zarafa-dagent -j. That way, zarafa-dagent knows that the mails are spam, and sorts them into the Junk folders. The Procmail configuration for that is for example:

:0 w
* ^X-Spam-Flag: YES
| /usr/bin/zarafa-dagent -j $1
EXITCODE=$?

:0 w
| /usr/bin/zarafa-dagent $1
EXITCODE=$?

The second way – which I prefer – is to tell zarafa-dagent to look for the spam tag itself. The Dagent can be configured to search for a spam tag flag sorting these mails into the Junk folder automatically. The configuration is done in /etc/zarafa/dagent.cfg:

spam_header_name = X-Spam-Flag
spam_header_value = yes

Please note that the values are case-insensitive.

Which method you use highly depends on your actual groupware setup: in case you already have a Procmail/Maildrop/Whatnot setup you might want to use the first method since it can easily be integrated with the existing setup. In case you do not have or do not want to introduce another layer like Procmail, you should go with the latter method. Also, the latter method is independent of the rest of the setup – so even with a working Procmail setup you might want to use the latter method to have it out of your mind.

Google & ActiveSync, Microsoft & CalDav: Pure irony

Android_robotToday Microsoft announced plans to implement CalDav and CardDav support in Windows Phone. That will enable users to still sync with Google services once these shut down their ActiveSync support in Summer. That is highly ironic and almost ridiculous, since Google itself does not support CalDav and CardDav in Android.

It all started with Google’s Winter cleaning: Google announced a couple of weeks ago that their services will soon be no longer offer an ActiveSync interface. That means: all client devices accessing Google’s services via ActiveSync need to switch to some other way of synching. Btw., read carefully: this has nothing to do with Android. Not at all! Also, iPhones don’t have to bother because they can simply switch to CalDav and CardDav which is natively supported in iOS. However, id does affect users of Microsoft’s Windows Phone. They only had ActiveSync as an option.

Now Microsoft announced they are going to implement CardDav and CalDav support in their Windows Phone. So that users can happily sync their Windows Phones with Google services.

And here comes the irony: Google itself does not support CalDav nor CardDav on client side. Google’s Android operating system does not offer it, not at all! Google only supports its own, proprietary sync way used in the Google apps, and has support for ActiveSync, albeit pretty limited support.

So, to summarize: Google forces others to use open standards which they do not support themselves.

While it is good that Microsoft is forced to implement open standards, Google’s acting nevertheless looks ridiculous, that is just sad. I wish Google would have the guts to just add CardDav and CalDav support and have a party with the people fighting for open standards. I mean, how bad would it look like if a Microsoft operating system would support open standards better than a Google operating system?

Howto: Extracting entire mails in mbox format from Zarafa [link]

Tux
A Zarafa groupware does workin complex setup you most likely have spam filtering software like Spamassassin running on the main MTA in front of Zarafa. Modern spam filters however need to be trained with ham and spam examples – and Zarafa has no simple way to export stored mails in a txt or mbox format, and the database does not store mails but mapi objects.

Luckily, a coworker of mine here at credativ has written a script to extract mails from a Zarafa to mbox as long as some header information are known: [Howto] Zarafa mail extraction.

[Howto] Syncing multiple calendars between Android and Zarafa

Android_robotSyncing multiple calendards between Zarafa (or any other groupware) and Android over ActiveSync is not possible due to limitation in Android. However, Zarafa can export calendars via CalDav, and there is an Android apps which adds CalDav calendars to the native calendar system.

Background: ActiveSync and CalDav

ActiveSync is the Microsoft way of syncing data, and is well established in the business ecosystems and thus also in groupware sync solutions like z-push. However, sharing multiple calendars via ActiveSync is not possible with Android without any special hacks. Additionally, ActiveSync is patented and copyrighted and as as result for each device which is able to sync via ActiveSync a fee is payed to Microsoft.

CalDav on the other hand is an open standard for syncing data, available to everyone for free. Unfortunately, it is not natively supported by Android although many groupware solutions provide support for it. But there are 3rd party apps to add CalDav support to Android.

Zarafa

The zarafa support for CalDav is quickly added by installing the zarafa-ical package. Here is for example the package description on a CentOS/Fedora system:

$ rpm -qi zarafa-ical
[...]
The zarafa-ical package includes the Zarafa iCal/CalDAV gateway service
to enable users to access their calendar using iCalendar (RFC 2445/5545)
or CalDAV (RFC 4791) compliant clients. The iCal/CalDAV gateway service
can be configured to listen for HTTP and HTTPS requests.

The configuration is done in /etc/zarafa/ical.cfg. The only really interesting part is if you want to enable ical over TLS or not. After everything is set up, try to reach the calendars of your system via web browser, the address should look similar to https://www.example.net:8443/caldav/testuser/Calendar. Afterwards, create some more calendars to verify later on that everything worked.

Many other groupware solutions offer CalDav support as well, the setup should be equal similar. The beauty in CalDav is that it does not contain any special magic.

Android

Once Zarafa is set up, you can configure the Android client. As mentioned before, Android does not provide native CalDav support, thus a 3rd party app is required. I made quite good experiences with the app CalDav sync beta. While the app does cost 2,55 €, the author does promise to open source the app once it has matured enough.

After the app was installed, you just enter user credentials and server URL and are ready to go:
Screenshot_2013-02-23-01-12-31

The synced calendars show up in the Android calendar overview natively, and can be re-used in any calendar app out there which accesses the default Android calendar store:
Screenshot_2013-02-23-01-13-18

That’s it, you can now sync all calendars you want, even carious task lists, to your Android mobile phone. It works pretty well for my own Zarafa setup, but we’ve also tested it at credativ with dedicated calendar server in a productive environment.

Conclusion

As a result, the sync between multiple calendars in Zarafa and Android does work now flawlessly. An additional bonus is that you are free to choose the colors of the calendars, in contrast to the ActiveSync implementation where you are stuck with a random color. 🙂

Besides, CalDav is also implemented in groupware fat clients like Thunderbird, KDE’s Kmail and Gnome’s Evolution, and you can now access all data via the same interface.

Why Android cannot sync multiple calendars via ActiveSync

Android_robotIf you use ActiveSync on your Android device you are not able to sync more than one calendar. The reason is the missing support in the ActiveSync implementation of Android.

Using Android multiple calendars is not a problem at all – as long as you use Google Calendars. However, in business environments – or if you want to keep your data private – it might happen that you want to use your own calendar server. In such cases the sync is most often done via ActiveSync – and there multiple calendars cannot be synced, see for example Google code issue #36797. Of course, there are also other protocols like CalDav, but unfortunately Android does not support these natively.

There are lot of discussions out there why this does not work, and the situation is not simplified by the fact that there are various ActiveSync implementations on server side and even on mobile side (Samsung ActiveSync vs Google ActiveSync, etc.). But for plain Android, the situation is clear: the code lacks the ability.

The Exchange ActiveSync protocol specifies types of folders – like one type for the default mailbox, one for user created mail folders, etc. And while Android does know the type “12, User-created Mail folder”, it does not know the type “13, User-created Calendar folder”. It does not know “14, User-created Contacts folder” either, by the way. It’s simply not implemented in the class “FolderSyncParser”. Just check the list in line 60-75, and compare it to the above given type numbers.

Thus you are not able to natively sync multiple calendars with plain Android and ActiveSync. If you really need it, you have to use one of the many, many hacks: export to Google calendars, create one user for each calendar on the server side, etc. – or use another protocol like CalDav which is not natively supported in Android but can be added by 3rd party tools.

I do hope that Google implements multi calendar sync via ActiveSync (or CalDav, speaking about) at some point in the future. I find the feature of multiple calendars extremely helpful in the daily office routine. But then again, there would be one reason less to use Google calendars on Android phones, so it might be that this is a political decision.