Google continues CalDav support for everyone, now also adds CardDav

Yesterday Google announced that it will not restrict the CalDav access to their calendars to registered partners only, but that they will continue to provide it for everyone. Additionally, Google now offers CardDav support.

A couple of weeks ago Google announced that they would restrict CalDav access to their calendars to registered developers only. That resulted in a huge uproar among developers, users and open standards advocates and made many people wondering if Google will become a closed standards/software company in the future.

However, the pressure (and most likely the bad press and reputation) Google got worked, and they announced that the CalDav API will be continued as an API open for everyone:

In response to those requests, we are keeping the CalDAV API public.

And it becomes even better: CardDav support is added as well, meaning the address data can be accessed via open protocols as well:

And in the spirit of openness, today we’re also making CardDAV – an open standard for accessing contact information across the web – available to everyone for the first time.

This way CalDav and CardDav have an even better chance to become THE royalty free and open alternative to Microsoft’s ActiveSync protocol. Additionally, application developers don’t have to worry to add special code to support Google calendars and address books: they just add CalDav and CardDav support and they automatically support almost all groupware servers and services available.

This is good news and gives me back some trust in Google’s policies and priorities. There is still no CalDav or CardDav support in Android, yes – but at least the server side is better now.

Howto: Managing dotfiles with dfm

Most system administrators have a set of personalized dotfiles like .vimrc and .bashrc. Taking these files with you from host to host and keeping them up2date everywhere can be a quite wearisome task. There are various tools to ease the pain, and I like to shed some light on one of them: dfm – the dotfile manager.

My background

On my machines I usually keep a set of personalized dotfiles which I don’t want to miss on any other server I have to administrate:

.screenrc
.bashrc
.inputrc
.vimrc
.vim/colors/jellybeans.vim
.vim/colors/desert256.vim

I need these files on all machines which I regularly work on – and since there are quite some customer machines I have access to regularly I wrote my own, git backed Python script years ago to keep these files synced and up2date on each machine. While it was fun to write the script, I always knew that it did not cover all my use cases regarding dotfiles, and it was not really flexible in terms of complex directory structures and so on. Also, I knew there must be other people with the same problem out there – and thus I was sure better solutions already existed.

And boy, there are so many of them!

Some interesting solutions for dotfile management

Many people have looked at this problem before – and solved it in their own ways. Most often the basic principle is that the files are stored and tracked via git in a hidden directory, and the tool of your choise manages symlinks between the files in the store and in $HOME.

For example, a very interesting idea is to use GNU Stow to manage dotfiles. It tracks the necessary files in subdirectories and of course links the files from there to the ‘real’ places in $HOME. I like reusing existing tools, so the idea of using GNU Stow appealed immediately. Also, the ‘packages’ or ‘group’ support it offers is tempting. Unfortunately, on most systems GNU Stow is not installed by default, and I cannot install new software on customer machines.

The problem of necessary software installation is also relevant for another often mentioned solution: Homesick. Homesick is Ruby based, and works similar to the GNU Stow solution mentioned above: files are stored in a hidden subdirectory, tracked with git, and linked in $HOME. The main feature here is that it can keep the configuration files in various git repositories, called ‘castles’, so you can integrate the work of projects like oh-my-zsh.
While Homesick does offer quite some features, it is Ruby based – and I cannot expect a working Ruby environment on each system, so it is out of question. I can go with Perl or Python, but that’s about it.

Other people had the same Ruby problem and created Homeshick – a Homesick clone spelled with an additional ‘h’ and besides written in Bash. It is quite straight forward and offers all necessary features like listing and tracking various git repositories as source for dotfiles, linking the actual dotfiles to your home, and so on. This one is almost my favorite! I wouldn’t be surprised if it is the favorite for most of the users out there.

But Homeshick is only almost my favorite – meet dfm – a Utility to Manage Dotfiles! It is written in Perl and mainly does the same as mentioned above, even minus the support for more than one repository. But on the plus side it has the capability of ensuring file rights via chmod. I haven’t seen that in any other solution. Additionally it supports arbitrary scripts executed during the update process for example for host specific commands. And last but not least, using a three letter program feels, somehow, right

Starting with dfm

So, first of course you have to get dfm. If you are hosting your dotfiles on github anyway, just fork the dfm starter repo and clone it. Otherwise, if you later want to host it yourself, clone the main dfm repo and change the remote URL. My choice was the second way:

$ git clone git@github.com:justone/dotfiles.git .dotfiles
Cloning into '.dotfiles'...
remote: Counting objects: 3212, done.
remote: Compressing objects: 100% (1531/1531), done.
remote: Total 3212 (delta 1413), reused 3096 (delta 1397)
Receiving objects: 100% (3212/3212), 4.22 MiB | 202 KiB/s, done.
Resolving deltas: 100% (1413/1413), done.

Next I configured the just cloned repository to use my own URL since my dotfiles are not on github:

$ cd .dotfiles/
$ git remote -v
origin  git@github.com:justone/dotfiles.git (fetch)
origin  git@github.com:justone/dotfiles.git (push)
$ git remote set-url origin git@git.example.net:dotfiles
$ git push origin master
Counting objects: 402, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (139/139), done.
Writing objects: 100% (402/402), 58.03 KiB, done.
Total 402 (delta 207), reused 389 (delta 195)
To git@git.example.net:dotfiles
 * [new branch]      master -> master

You now have the repository up and ready. So let’s install dfm as a tool available in $PATH, meaning creating a symlink between ~/bin and ~/.dotfiles/bin and also extending the $PATH variable in .bashrc.load, which is added to .bashrc:

$ ./.dotfiles/bin/dfm
INFO: Installing dotfiles...
INFO:   Symlinking bin (.dotfiles/bin).
INFO:   Symlinking .bashrc.load (.dotfiles/.bashrc.load).
INFO: Appending loader to .bashrc

The .bashrc is hardly modified:

$ tail -n 1 .bashrc
. $HOME/.bashrc.load

As a side node, I am not sure if I really want to drop all my customizations on the bashrc loader, but the reasoning behind that move from the dfm author is rationale:

Why .bashrc.load instead of .bashrc?

Each OS or distribution generally has its own way of populating a default .bashrc in each new user’s home directory. This file works with the rest of the OS to load in special things like bash completion scripts or aliases. The idea behind using .bashrc.load is that dotfiles should add new things to a system rather than overwriting built-in funcitonality.

For instance, if a system sources bash completion files for you, and your dotfiles overwrites the system-provided .bashrc, then you would have to replicate that functionality on your own.

But no matter if you agree with it or not, the next step is to add further files to your dfm repository, which is quite easy because dfm comes along with an import function:

$ dfm import .vimrc
INFO: Importing .vimrc from /home/liquidat into /home/liquidat/.dotfiles
INFO:   Symlinking .vimrc (.dotfiles/.vimrc).
INFO: Committing with message 'importing .vimrc'
[master d7de67a] importing .vimrc
 1 file changed, 29 insertions(+)
 create mode 100644 .vimrc

The usage is pretty straightforward, and supports directories as well:

$ dfm import .vim
INFO: Importing .vim from /home/liquidat into /home/liquidat/.dotfiles
INFO:   Symlinking .vim (.dotfiles/.vim).
INFO: Committing with message 'importing .vim'
[master e9bd60a] importing .vim
 3 files changed, 875 insertions(+)
 create mode 100644 .vim/colors/desert256.vim
 create mode 100644 .vim/colors/jellybeans.vim

Using dfm on a new system

Using dfm on a new system is straightforward as well: clone the repo, invocate dfm, and you are done:

$ git clone git@git.example.com:dotfiles .dotfiles
Cloning into '.dotfiles'...
remote: Counting objects: 418, done.
remote: Compressing objects: 100% (142/142), done.
remote: Total 418 (delta 211), reused 401 (delta 207)
Receiving objects: 100% (418/418), 66.83 KiB, done.
Resolving deltas: 100% (211/211), done.
$ ./.dotfiles/bin/dfm
INFO: Installing dotfiles...
INFO:   Backing up .vimrc.
INFO:   Symlinking .vimrc (.dotfiles/.vimrc).
INFO:   Backing up bin.
INFO:   Symlinking bin (.dotfiles/bin).
INFO:   Symlinking .bashrc.load (.dotfiles/.bashrc.load).
INFO:   Backing up .inputrc.
INFO:   Symlinking .inputrc (.dotfiles/.inputrc).
INFO:   Backing up .vim.
INFO:   Symlinking .vim (.dotfiles/.vim).
INFO: Appending loader to .bashrc

As you see quite some files are backed up, that just means they are moved to .backup, so in worst case you know where to look.

Now lets see what happens when you change something.

$ cd ~/bin
$ ln -s /usr/bin/gnome-terminal gt
$ dfm add bin/gt
$ dfm commit -m "Added gt symlink for gnome-terminal."
[master 441c067] Added gt symlink for gnome-terminal.
 1 file changed, 1 insertion(+)
 create mode 120000 bin/gt
$ dfm push
Counting objects: 6, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (4/4), 363 bytes, done.
Total 4 (delta 1), reused 0 (delta 0)
To git@sinoda:dotfiles
   b28dc11..441c067  master -> master

As you see, dfm supports git pass through: git commands are directly handed over to git. The changes where added to the git repository, and the repository was pushed to the remote URL.

So, to get the changes onto the other system you just have to ask dfm to update the files via dfm umi. In this case I called it after I made changes to .screenrc:

$ dfm umi
[...]
INFO: re-installing dotfiles
INFO: Installing dotfiles...
INFO:   Symlinking .screenrc (.dotfiles/.screenrc).

dfm special features

As mentioned above, the strongest feature of dfm is to be able to ensure file system rights and to start scripts after an update. The first option comes in handy when you are sharing files in your ssh config directory. The second is useful whenever you have to alter files or do anything based for example on host names. Imagine that you have various build machines to build rpm files, but you have to use different packages names on each build environment (think of customer specific e-mail addresses here).

It should be possible to create a script that would fill in the necessary details in the rpmmacros file based on IP or hostname. I haven’t given that a try, but it should be worth it…

Keeping dfm up2date

Last but not least, it is of course desirable to keep dfm itself up2date. The dfm wiki proposes the following workflow for that:

$ dfm remote add upstream git://github.com/justone/dotfiles.git
$ dfm checkout master
$ dfm fetch upstream 
$ dfm merge upstream/master

It is a pretty neat way, using git tools as they should be used, and is still easy enough to handle.

Summary

So, summarizing I can say dfm offers a quite neat and easily understandable solution for managing dotfiles while not relying on languages or tools you probably cannot install on the systems you are working on. However, Homeshick comes in as a close second, and I might give that one a try at some other point in the future. In the end, both solutions are much better than self written solutions – or no solution at all.

Skype is following your links – that’s proprietary for you

Yesterday it was reported that Skype, owned by Microsoft these days, seems to automatically follow each exchanged https link. Besides the fact that this is a huge security and personal rights problem in its own it again shows how important it is to not trust a proprietary system.

The problem, skin deep

Heise reported yesterday that Skype follows https links which have been exchanged in chats on a regular basis. First and foremost, this is a privacy issue: it looks like Skype, and thus Microsoft, scans your chat history and acts based on these findings on a regular base. That cannot be explained by “security measures” or anything like it and is not acceptable. My personal data are mine, and Microsoft should not have anything to do with as long as there is no need!

Second, there is the security problem: imagine you are exchanging private links, or even links containing passwords and usernames for direct access (you shouldn’t, but sometimes you have to). Microsoft does follows these links -and therefore gains full access to all data hidden there. Imagine these are sensitive data (private or business), you have no idea what Microsoft is going to do with them.

Third, there is the disturbing part: Microsoft only follows the https links, only the encrypted URLs. If this action would be a security thing, they would surely follow the http links as well. So there must be another explanation – but which one? It is disturbing to know that Microsoft has a motivation to regularly follow links to specifically secured content.

The problem, profound

While these news are shocking, the root problem is not Skype or the behavior of Microsoft – I am pretty sure that their Licence Agreement will cover such actions. And it is most likely that others like WhatsApp, Facebook Chat or whatnot do behave in similar ways. So the actual problem is handing over all your data to a company which you have no inside to. You have no idea what they are doing, you have no control about it, and you cannot even be sure that nothing bad is done with it. Also, most vendors try to lock you in with your service, so that switching away from them is painfully due to used workflows, tools and social networks.

The solution

From my point of view, my personal perfect solution is hosting such sensitive services on my own. However, that cannot be a solution for everyone, and I for myself cannot provide for example the SLAs others need.

Thus I guess the best solution is to be conscious about what you do – and what the consequences are. Try to avoid proprietary solutions where possible. For example for chats, try to use open protocols like XMPP. Google Talk is a good example here: company based, but still using open protocols, they even push the development forward (Jingle, …). Or, if you upload files to web services, make sure you have local backup. Also, try not to upload sensitive data – if you have to, encrypt it beforehand. And if you use social networks, try to not depend on one of them too much, use cross posts for various services at the same time if possible.

And, last but not least: ask your service providers to establish transparency and rules for a responsible and acceptable usage of your data. After all, they depend on the users trust, and if enough users are requesting such changes, they will have to follow.

Howto: Installing Owncloud News, a self hosted RSS reader

Owncloud News, a RSS news reader for the self hosting cloud service Owncloud, is available in an Alpha version. That comes right at the time Google Reader is bound to see its end soon.

I must admit that I do not understand why Google decided to shut down the Google Reader service. Social media with their unstructured news areas are nice, but no match to a well structured news feed full of read and unread news. But, there are replacements, and one pretty wise choice would be to not depend on yet another web service, but to host it yourself.

In comes Owncloud: it can already host your addresses, calendars, files and musik and can be integrated with your desktop as well. Now a RSS reader app, Owncloud News was released as an Alpha version, and indeed already looks promising:

The installation is pretty smooth as well. The requirements are a running Owncloud 5 version, so 4.5 won’t do it. The installation itself basically consists of two steps: installing and activating the so called App Framework, which is supposed to be the foundation for other Owncloud apps in the future, and afterwards installing the news app itself:

# cd /var/www
# git clone https://github.com/owncloud/appframework.git
Cloning into 'appframework'...
[...]
# git clone https://github.com/owncloud/news.git
Cloning into 'news'...
[...]

I choose /var/www here because it is recommended in the manual and because there the appropriate user has the necessary access rights. But it could be any dir, since you only link the plugins anyway:

# ln -s /var/www/appframework /var/www/owncloud/apps
# ln -s /var/www/news /var/www/owncloud/apps

Speaking about rights, make sure the web server can write cache files:

# sudo chown -R www-data:www-data /var/www/news/cache

Afterwards, login to your owncloud, and active the plugins: first the framework, followed by the actual application. Add feeds, play around, as you will see it works pretty nice.

What is still missing right now is an Android news reader which could sync with the server. When that is available as well, Owncloud News might become *the* Google Reader descendant.

Ask people for their best project

While reading through a project management book I came across a very simple, but also very efficient trick regarding changing people’s behavior for the better: asking them what their best project was.

I was reading “How to Change the World” from Jurgen Appelo when I came across an interesting story how to get people to tell their best practices. Appelo himself their mentions a discussion with a Scrum coach who tells him to ask people for their best project, to let them tell stories how everything went, what was good, and so on.

First of all, I like the idea because it puts the people in story telling mode. In such a mode they do not try to fit their experiences in any pre-structured form and they don’t have a pressure to fulfill any expectations, a problem that can come up if they have to answer questions. Second, they just tell their own experiences from their own point of view – and together with it, as Appelo outlines, their best practices. That way you can get a pretty good idea of how the people think, what they expect and what actually works for them. In other words, this way you will learn about working practices and approaches they experienced themselves.

The next step is naturally: if you are entering a running project, or planning a new one, ask them what has to change in the project so that it becomes as successful as the one they have just told you about. It is a very simple, but also pretty efficient step towards more modern and flexible project management, which is very, very important in IT projects.