[Short Tip] Debug Spamassassin within Amavisd

920839987_135ba34fff
Filtering e-mail for spam and viruses can be done efficiently with Amavisd-New. Besides its own technologies to identify and filter out Spam it can also make use of Spamassassin and its results. However, since Amavisd starts Spamassassin itself, it sometimes is hard to debug when something is not working.

For example in a recent case I saw that the Bayes database was not used when checking for spam characteristics, although the database was properly trained with ham and spam.

Thus first I checked Spamassassin itself:

$ su -s /bin/bash mailuser -c "spamassassin -D -t < ExampleSpam.eml 2>&1"  | tee sa.out

That worked well, the Bayes database was queried, results were shown.

Next, I added $sa_debug = '1,all'; to the Amavisd configuration and run Amavisd in debug mode:

$ amavisd -c /etc/amavisd/amavisd.conf debug

And that showed the problem: one of the Bayes files had wrong permissions. After fixing those, the filter run again properly.

Advertisements

[Howto] LDAP schema for Postfix

Postfix LogoThe official Postfix documentation to use LDAP for user and alias lookup mentions certain LDAP attributes which are not part of the default OpenLDAP. In this article I will shortly explain a basic theme providing these attributes and the corresponding object class.

Postfix can easily be connected to LDAP to lookup addresses and aliases. The Postfix LDAP documentation covers all the details. As mentioned there the default configuration of Postfix expects two LDAP attributes in the LDAP schema: mailacceptinggeneralid and maildrop. This also shows in the code in src/global/dict_ldap.c:

dict_ldap->query =
    cfg_get_str(dict_ldap->parser, "query_filter",
        "(mailacceptinggeneralid=%s)", 0, 0);

However, these attributes are not part of the default OpenLDAP installation, and the Postfix documentation does not mention how exactly that has to look like and where to get it. For that reason we at my employer credativ provide such a schema at Github: github.com/credativ/postfix-ldap-schema. The github repository contains the schema, the corresponding licence and a short documentation. A German introduction to the schema can also be found at credativ’s blog: LDAP-Schema für Postfix-Abfragen

The provided schema defines the necessary attribute types mailacceptinggeneralid and maildrop as well as the object class postfixUser. Please note that in this schema the used OIDs are of the type Experimental OpenLDAP, see also the OID database.

To use the schema it must be used by OpenLDAP, for example by including in in slapd.conf. A corresponding LDAP entry could look like:

dn: uid=mmu,ou=accounts,dc=example,dc=net
objectclass: top
objectclass: person
objectclass: posixAccount
objectclass: postfixUser
cn: Max Mustermann
sn: Mustermann
uid: mmu
uidNumber: 5001
gidNumber: 5000
homeDirectory: /home/vmail
mailacceptinggeneralid: mmu
mailacceptinggeneralid: max.mustermann
mailacceptinggeneralid: m.mustermann
mailacceptinggeneralid: bugs
maildrop: mmu

As you see the example covers multiple aliases. Also, the final mailbox is a domain less entity: maildrop: mmu does not mention any domain name. This only works if your mail boxes actually do not require (or even allow) domain names – in this case this was true since the mail is finally transported to a Dovecot server which does not know about the various domains.

Please note that this schema can only be the foundation for a more sophisticated, more complex schema which need to be tailored to fit the individual needs of the corresponding setup.

Postfix Architecture Overview

Postfix LogoPostfix consists of numerous services, commandos and queues. The documentation describes them in detail, but a general overview was missing – so I gave it a try.

The Postfix documentation is very detailed and of high quality. One example is postfix.org/OVERVIEW.html which explains the various services, queues and commandos and how they interact with each other. However, despite the included ASCII diagrams a “big picture” is missing. Since we do have quite some customers using Postfix at credativ my company gave me some time to spend at creating such an overview. Here you go:

Postfix Architecture Overview
Postfix Architecture Overview

The image links to the company website where you can download the PDF, but there is also a Postfix Architecture Overview Github repository including the odg, a pdf and a png.

Of course Postfix is far too complex to highlight all services and all commandos and all ways of communication, so the picture had to be simplified. For example, the process of flushing mails does only cover the deferred and incoming queue and not the processes which actually trigger the flushing. Also, policyd’s are missing, and I would actually like to include where Milter filters can be attached. Maybe I’ll update the graphic in the future. The license will be CC-BY-SA, so if you have ideas to extend the picture further you will be able to send me pull requests. Speaking about ideas, thanks to Patrick Ben Koetter who gave me the idea to create such an image.

Android 4.4 now *can* sync multiple calendars via ActiveSync

Android_robotWith the release of Android 4.4 called KitKat Google made some interesting changes to their ActiveSync implementation: the code is now set up to sync more than one calender, and the first KitKat user already confirmed that new feature.

In February I described in a blogpost why Android cannot sync multiple calendars via ActiveSync. The problem was that Google did not implement the necessary parts of the ActiveSync specification in Android.

However, that seems to have changed: if you look at the current ActiveSync implementation of Android 4.4 KitKat, the source code (tag 4.4rc1) does list support for multiple calendars – and also for multiple address books:

        MAILBOX_TYPE_MAP.put(Eas.MAILBOX_TYPE_USER_CALENDAR, Mailbox.TYPE_CALENDAR);
        MAILBOX_TYPE_MAP.put(Eas.MAILBOX_TYPE_USER_CONTACTS, Mailbox.TYPE_CONTACTS);

I had no chance yet to test that on my own, but there are reports that it is indeed working:

Today i flashed a Android 4.4 Rom on my smartphone. After adding the Exchange Profile all my Calendars are there […]
I’ve uploaded a screenshot here:
http://postimg.org/image/5d4u364ub/

Looks like Google actually listened to…erm, corporate users? At least to someone, though 😉

But: Since I have no first-hand-experience in this regard I would like to ask all of my nine readers out there if anyone has a stock KitKat running and if the could check this feature. Please test this and leave a report about your experiences in the comments. I will include it in the article.

By the way, the above mentioned source code snippet also tells quite exactly which other ActiveSync functions are not yet supported in Android:

        //MAILBOX_TYPE_MAP.put(Eas.MAILBOX_TYPE_TASKS,  Mailbox.TYPE_TASKS);
        //MAILBOX_TYPE_MAP.put(Eas.MAILBOX_TYPE_NOTES, Mailbox.TYPE_NONE);
        //MAILBOX_TYPE_MAP.put(Eas.MAILBOX_TYPE_JOURNAL, Mailbox.TYPE_NONE);
        //MAILBOX_TYPE_MAP.put(Eas.MAILBOX_TYPE_USER_TASKS, Mailbox.TYPE_TASKS);
        //MAILBOX_TYPE_MAP.put(Eas.MAILBOX_TYPE_USER_JOURNAL, Mailbox.TYPE_NONE);
        //MAILBOX_TYPE_MAP.put(Eas.MAILBOX_TYPE_USER_NOTES, Mailbox.TYPE_NONE);
        //MAILBOX_TYPE_MAP.put(Eas.MAILBOX_TYPE_UNKNOWN, Mailbox.TYPE_NONE);
        //MAILBOX_TYPE_MAP.put(MAILBOX_TYPE_RECIPIENT_INFORMATION_CACHE, Mailbox.TYPE_NONE);

I guess syncing tasks could come in handy in corporate environments. Combined with support for multiple task folders you could even design your own Kanban “board” that way.

Nevertheless I’d like to add that ActiveSync is no big deal for me anymore because I am very happy with a – albeit 3rd party and not yet Open Source – CalDav implementation, which can even sync multiple task folders.

Thoughts on crypted communication

network-63770_150Due to the recently published information about mass surveillance on a yet not known level right now the question remains how to encrypt communication. I had some thoughts regarding that topic involving a GPG like web of trust combined with user friendliness which I’d like to share here.

Given everything which was published so far, un-encrypted communication is not save at all. The same is unfortunately true for encryption methods which rely on encryption provided by the servers of some organization. If there is a centralized organization storing the keys for you, or just providing you with the encryption technology, you are screwed, because the intelligence agencies will force them to cooperate. For that reason, the encryption must take place on the end users system already (and the software must be Open Source).

However, if you have end-user encryption, you have the problem of the key exchange – if two people want to communicate securely, they need to exchange the keys or at least securely verify that a public key indeed belongs to their private key. That only works if they meet in person – or if there is a web of trust.

A common example of such a web of trust is the GPG web of trust: people who have properly verified that person A belongs to key M sign this key. If person B trusts person A, it can just use key M since it is already verified by person A. However, in case of GPG the web of trust never reached mainstream. It is mainly used by technical minded people. Most users never got used to it.

So, from what I can tell the only chance to establish a web of trust is to hide the technical details as much as possible from the users. The same is true for the actual key exchange – it needs to be as simple as possible so that each normal user can use it.

Given this background I would suggest the following solution at least for mobile phones. You download the app, and it asks the user for a password. In the background, a key pair is generated and secured with the password, and all data stored on the device are encrypted using the public key. If user A meets user B all they need to do is pressing a button in the app, and a QR code is shown. The other user scans that QR code, and its done. The user shows up in the contact list, and they can chat. In the background, the app extracts the key ID and fingerprint from the QR code, downloads the public key, signs it and uploads the signature automatically.

The biggest problem comes up when user C comes into play, wants to communicate with user A, but they both have no common connection in their web of trust. They would have to meet – or use some other way of exchanging the data securely. A simple way would still be to talk on the phone, but that never worked for GPG. So some kind of web service to host their QR code for a short time only would probably a solution, although it would be pretty risky.

To lower the danger of a man in the middle attack in the above given web example the key servers must only accept one key pair for each identity, which is different to the way GPG works. That would in fact mean that you can have each login only once – if you loose your key, your are screwed.

One question though remains: how many steps in the web of trust are still trustworthy? I guess that could be left as a configuration option if, and only if, a user wants to modify that.

To summarize: I guess that the current cryptography technologies we have could really help to establish secure communication. But to really bring that communication to the masses we need easy-to-use (read: your grandma!) applications doing everything in the background.

Short Tip: Extract attachments from multipart messages

920839987_135ba34fff

Sometimes e-mails are stored as a plain text file. This might be due to backup solutions or as in my case, sometimes I have to save e-mails in text format because the client has issues with more complicated S/MIME mails with attachments.

In such cases the text file itself contains the multipart message body of the e-mail, so the attachments are provided as base64 streams:

--------------060903090608060502040600
Content-Type: application/x-gzip;
 name="liquidat.tar.gz"
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
 filename="liquidat.tar.gz"

qrsIAstukk4AA+17Wa+jSpZuPfMrrK6Hvi1qJ4OxsfuodQU2YLDBUJDGrvqBeR7MaPj1N7D3
OEmSxO8Wq7+3Y48dTWvXi8XvvKj8od6vPf9vKjWIv1v7nt3G/d8rEX5D/FdrDIxj2IrUPeE/
j5Dv4g9+fPnTRcX006T++LdYYw7w+i...

These data can be extracted manually, but that’s a wearisome task. It’s easier to use munpack, which can easily extract attachment(s) out of such text files and write them into a proper named files.

$ munpack -f mail.txt
liquidat.tar.gz (application/x-gzip)

[Howto] Installing Owncloud News, a self hosted RSS reader

two_glossy_cloudsOwncloud News, a RSS news reader for the self hosting cloud service Owncloud, is available in an Alpha version. That comes right at the time Google Reader is bound to see its end soon.

I must admit that I do not understand why Google decided to shut down the Google Reader service. Social media with their unstructured news areas are nice, but no match to a well structured news feed full of read and unread news. But, there are replacements, and one pretty wise choice would be to not depend on yet another web service, but to host it yourself.

In comes Owncloud: it can already host your addresses, calendars, files and musik and can be integrated with your desktop as well. Now a RSS reader app, Owncloud News was released as an Alpha version, and indeed already looks promising:

Owncloud-Reader-General

The installation is pretty smooth as well. The requirements are a running Owncloud 5 version, so 4.5 won’t do it. The installation itself basically consists of two steps: installing and activating the so called App Framework, which is supposed to be the foundation for other Owncloud apps in the future, and afterwards installing the news app itself:

# cd /var/www
# git clone https://github.com/owncloud/appframework.git
Cloning into 'appframework'...
[...]
# git clone https://github.com/owncloud/news.git
Cloning into 'news'...
[...]

I choose /var/www here because it is recommended in the manual and because there the appropriate user has the necessary access rights. But it could be any dir, since you only link the plugins anyway:

# ln -s /var/www/appframework /var/www/owncloud/apps
# ln -s /var/www/news /var/www/owncloud/apps

Speaking about rights, make sure the web server can write cache files:

# sudo chown -R www-data:www-data /var/www/news/cache

Afterwards, login to your owncloud, and active the plugins: first the framework, followed by the actual application. Add feeds, play around, as you will see it works pretty nice.

What is still missing right now is an Android news reader which could sync with the server. When that is available as well, Owncloud News might become *the* Google Reader descendant.