Last call for ….

I haven’t posted anything in a while on this blog, and now I made the decision that this will not change: it is unlikely that this blog will be updated anytime soon. The reason is actually twofold:

Job

I’m working full time as an Open Source/Linux consultant these days – and after work I do not really have the time nor the energy to invest even more time into Open Source (besides the Fedora packaging).

Company’s Blog

I was successful in convincing enough people in our company to start a blog – and I blog there since then, so when I get home I usually already have blogged about whatever comes to my mind.

That means effectively that you will not receive any more new posts here. It hurts my heart and kills kittens, but you can remove the blogfeed. @planets where I might still be listed at: please remove this blog feed as well.

However, if you *do* want to keep up with my thoughts: credativ’s company blog is working quite nice these days. Btw., in case you don’t know, credativ is an Open Source/Linux company and the one behind the Open Source Support Center (OSSC) and the Open Source Support Card (yeah, “catchy” names, I know). They are focussed on Open Source support (Linux-Support, PostgreSQL-Support, etc.) and have offices in DE, UK, US, etc. So the general topics are pretty close to this blog. If you look close you will recognize my style: short italic introduction, eye catcher on the upper right side, special headline markings for Howtos and Short Tips, and so on. Also, the categories are quite the same, and it is actually available in German and English. Also, I am not the only person writing there – one very active PostgreSQL developer keeps blogging there, if I want it or not.

However – it is a company blog, so you will (!) find information regarding the company itself, or newest marketing things. You are warned!

So this is it: the last post. Thanks everyone for wonderful years full of blogging, discussions, news, Howtos and good tips. So long, and thanks for the fish!

Playing the numbers game 2008: number of Linux installations world wide

The number of Linux users and installations is impossible to determine. But there are several different statistical information available which can be used to at least get a rough idea of the number of Linux installations world wide.

Merging different statistical data into one number is a tricky exercise and the result is questionable at best. Keep that in mind when you read the following information. The idea is not to get exact numbers but to get a rough idea of the dimension, nothing else.

Source #1: Fedora

One of the best statistical sources regarding Linux usage are the Fedora statistics. There the number of downloaded images as well as the number of unique IPs getting software updates is counted.
The data are difficult to interpret: no one knows if a downloaded image was only used to test the new system or to bun it onto a CD and distribute it to thousands of magazine readers or thousands of company computers. The second number is problematic because one new IP can mean a big NAT network or just a dial-in user who re-connected. So flaws everywhere, but it is a interesting coincidence that the IP numbers and the downloads are rather close.

There are also the smolt data. It tracks the users who opted in to a tracking system. Currently the smolt web server seems to be lacking behind. But there are current data available for older Fedora releases: these informations say that every month Fedora still gets more than 10k new Fedora 7 users – although there are already Fedora 8 and Fedora 9 available.

So the question is how to read all the data. One way was recently suggested by Paul Frields, Fedora’s project leader: he sums up the data to be around 11.5 million. Together with currently 2.5 million Red Hat subscriptions this would result in 13 million users. Focussing on Fedora alone and leaving the Fedora 6 users Fedora would still have a user base of 9.5 million users.

Source #2: relative statistics

There are hardly any other trustfully data from other Linux distributions available. Therefore, the Fedora number does say a lot about Fedora, but not that much about Linux in general. However, there are other statistics which measure the relative acceptance of Linux distributions.

One such source is the 2007 Linux Desktop Survey done by DesktopLinux.com. There the relative importance of Fedora/Red Hat is measured with 9%. Unfortunately there is no more recent survey available. I wonder why no one has picked up such a survey in 2008. Maybe I should start one on my own?

The result

Given that Fedora/Red Hat has roughly 10% and also roughly 10 million users together (which in fact seems like a at least slightly realistic data base, given the facts), the total number of Linux users world wide would sum up to 100 million Linux users. Nice.

That would leave Mac OS far behind, which is however not that surprising: Mac OS is hardly used in Offices or the government outside the US, and it is far easier to give Linux a try and keep it as a dual boot option besides a Windows installation. Also, the EU governments are pushing Linux quite a lot, and many companies and governments indeed switch to Linux right now or already switched over in the client space for some of the day-to-day workstations.

Still, last year I played the “numbers game” already (unfortunately with the same relativity source, btw.) and the result said something about 20 to 30 million users. I doubt that the number of Linux users spiked that much in the last year, but think that we can safely say the number of Linux users world wide is somewhere in the middle two digit million area, somewhere around 50 million installations worldwide.

Keep in mind that this counts mainly workstations – not traffic lights, shop information terminals or any other specialized hardware. Including all these devices would result in much, much larger numbers.

The problem: discrepancy

Most numbers available guessing the number of Linux users world wide say that there are not that many Linux installations out there, not at all. Most often it is said that, in percentage, Apple has a low one-digit number, while Linux has a dot before its first non-zero number.
The statistical backup for such numbers is most often created by browser strings aggregated from Web pages. This procedure has the flaw that these strings are often faked to make it easier to access specific pages. Also, the monitored pages are only a subset of the entire web, and surprisingly often focus on the US only which is not representative for the world at any rate.

Still, I do often wonder why such numbers and my estimates are different in the order of magnitudes. I welcome any comment on that issue!

The Open Source Year 2009

Every year Open Source technology is improved and extended. This post sheds some light on new technologies which might arrive in 2009.

Btrfs

With ext4 a new file just left the developer corner. However, ext4 is an old-style file system and does not offer “hot” features like on-line snap shots, versioning and so on. ZFS does, but it is not an option for Linux due to licence reasons. Here comes Btrfs into play: it is in development for quite some time now and many Kernel developers already asked to include Btrfs in the Kernel to speed up the development process. Additionally, several Kernel developers already mentioned that they expect Btrfs to be the next generation default file system for Linux in the mid-term.

In 2009 Btrfs will most likely stabilize its file system format and publish a beta version for testing purposes.

oVirt

oVirt is a small host image that provides libvirt services and hosts virtual machines. Additionally it also has a well designed web based management system. The aim is to provide an enterprise ready VM management console capable of managing large sever clusters hosting large numbers of virtual machines, but is also supposed for single users.

In 2009 oVirt will hopefully see its first beta release ready for first real-world-like tests. Additionally, with some luck, it might be bundled with openfiler to ease the storage management. Last but not least it could include support for Xen in a future version.

OpenGL 3.0

The release of OpenGL 3.0 this year was rather surprising: it was delayed for almost a year without any notice at all, which is usually a clear sign that a project is dead. However, left aside the question if the OpenGL 3.0 release is the beginning of a new era or or just a last breath of the project, OpenGL 3.0 is now out in the wild and the Free Software community will adopt it sooner or later.

While Nvidia has already released a first version of an OpenGL 3.0 capable driver the FLOSS OpenGL implementation Mesa hasn’t released anything yet. But Mesa is alive and vibrant again since 2007 and a new release can be expected in the near future. Also likely is that AMD/ATI will release a new version of their OpenGL stack featuring the newest OpenGL spec. I would like to see AMD/TI team up with Mesa on that one but that’s just a wish, I’m afraid.

So in 2009 we will see OpenGL 3.0 coming to the masses – in proprietary as well as in Free drivers. This way newest graphics card technology will come to Linux and application developers can built upon that.

Gallium

Simply said, Gallium3D is an attempt to make graphics card driver development on Linux much easier: it abstracts the driver development from the underlying graphics standard implementation (for example OpenGL). Due to that abstraction, switching to another graphics standard should also be fairly easy. That way it should be easier to write one single graphics card driver for different devices (which do often need something else than OpenGL). And in case OpenGL is really dead, it could be a way to more or less painlessly replace it with something new.
Right now Gallium3D is in heavy development and we yet have to see it in the wild. There are only few drivers ported to it and I haven’t seen any distribution shipping it yet.

In 2009 this could switch: a first testing release for the broader masses is likely, and it could speed up the development of drivers for Gallium3D.

Gem and KMS

Speaking about graphics, there are other things which are in development and which are already surfacing here and there: the new graphics memory manager GEM. Using GEM the graphics cards does not have to be re-initialized as soon as you switch to another application. Also, everything will be written to the memory and the composition manager can simply access it there, avoiding some problems current drivers have when for example running videos on AIGLX.
Besides, Kernel Mode Setting (KMS) will move other tasks of the graphic subsystem away from X towards the kernel. As a result it will be much easier and flicker-free to switch from X to a tty console, and the graphical system will be able to show kernel oops. Linux will get its own blue screen capability, finally!

These features have partially found their way into newer Fedora releases, but only for specific hardware and under certain conditions. In 2009 it can be expected that the current FLOSS/Nvidia/AMD drivers will switch over to GEM and KMS to provide a much saner graphics experience to the user.

KDE 4.3: Pimp your PIM

Curently KDE’s PIM is in a difficult situation: Kontact is one of the best free groupware clients out there, but it was never designed to be one, and using it as such today can be an adventure. To fix that Akonadi was created. It was shipped with KDE 4.1 to back up Mailody, KDE 4.2 will see it the first time together with Kontact. This will give the developers quite some time to sanitize and improve the Akonadi service as well as to add new plugins to provide something revolutionary right in time for KDE 4.3.

In 2009 we will finally see a FLOSS groupware client which is working with a broad range of groupware servers, has a maintainable code base – and is perfectly integrated on all major platforms.

Qt on the mobile mass market

This year almost started with the news that Nokia acquired Trolltech. Recently it was announced that Qt now runs on Symbian S60. Also, with the iPhone, Google’s G1 and even a new Blackberry Nokia seriously needs a cool new device with fancy graphics and an appealing software platform.

Now put two and two together. With a bit of luck we will see the first Qt-Nokia devices with multi touch screen in 2009. With even a bit more of luck, it will be shipped in a way that Qt developers can use the tools they are used to to develop software for the new platform. Think of running KDE on these devices.

Gnome 3.0 development

In summer this year the Gnome developers started planning their next big release – Gnome 3.0. Currently not too many information have surfaced, but such breaks need their time. A state tracker for the Gtk+ changes is online and shows that indeed some work is underway already.

In 2009 first Alpha release could surface to show in which direction Gtk+ and Gnome are heading, and how the transition progress works out. That will definitely be an interesting time – the transition was a major task for KDE, and the Gnome team better takes a close look at that to learn from KDE’s experience.

Conclusion

While I already called 2007 the Year Of Open Source Graphics, 2009 can become a good candidate for it as well. In this post it got three paragraphs, and if everything comes true, 2009 will revolutionize the world of Linux graphics. This will, however, happen mostly under the hood. The users will not notice several fixes, but not the large underlying changes, which is different to 2007.

But in general 2009 will be exciting in almost all FLOSS areas. Keep in mind that this list is not and cannot be complete! So I ask every reader to drop a comment here containing his or her tip for revolutionary changes or news in the FLOSS world in 2009!

Kontact and Citadel – experiences

While there are many FLOSS Groupware servers out there, hardly any one them is reliable working with any FLOSS client. Recently my company had to bring Citadel and Kontact together, and we used the opportunity to patch while we go.

Background

Recently my company, the credativ GmbH, was faced with the task to sync the usual groupware data (calender, tasks, addresses) between a mobile phone and a PIM client – using FLOSS and via Internet. The natural solution of course is Funambol to get the data of the phone. The next step is to connect that one to a groupware server and access the server via a well known KDE 3 groupware client.

However, having a closer look at the currently available FLOSS groupware solutions was quite disillusioning. While there are many nice servers, there are hardly any solutions where a server works correctly with a client. The best working example is maybe Kolab-Kontact, but Kolab doesn’t work easily together with Funambol. Actually, the current groupware-Funambol connector only supports three groupware servers: Citadel, OpenGroupware and eGroupware. For various reasons we settled on Citadel, and now the task was to check how well this works together with Kontact.

Citadel

Citadel is rather unique in the groupware world. It is one of the oldest projects, and started off as something different than it is today. For that reason there are many historically grown things – like the room concept. However, once you get used to that the server is quite usable, and the developer community was very helpful.

Regarding the described setup there were only smaller things to fix, and the Citadel guys did that for us, so thanks to them. Citadel still desperately needs a theme which is pleasing the eyes – the current one looks horrible. But that is a task for a web designer and can come after the code basically works.

Kontact – solved problems

Kontact has quite some problems on many levels. The main reason behind all these problems is the protocol: there is no single open standard for doing groupware stuff, there are many different. There is though Groupdav, an attempt to create such a common standard, but until now Groupdav is still a proposal, and besides some missing main functions (no notifications when the content was changed, for example) it wasn’t accepted by everyone. As a result, the current Groupdav support in Kontact is mixed, and we tried to check what worked, what didn’t, and what was fixable for us.

The first problem was that Kontact of KDE 3 has problems with special characters like German umlauts. That is described in Bug #159795, and the patches necessary to fix that behavior are added to the bug report. This is fixed in KDE 4 anyway, so the patch is only interesting for people who are working with Kontact of KDE 3.x for some more time.

The second problem was that tasks flagged as complete were not synced properly – at least, the complete flag part wasn’t synced. When in Kontact a task was flagged as completed it didn’t show up as such on Citadel and vice versa. The problem seems to be that Kontact looks at the percentage value of the completion while Citadel looks at the parameter ‘COMPLETED’. See Bug #171905 for more details – and a patch.

The base for the third problem lies more in the groupdav protocol than in Kontact: there is no mechanism defined to inform the client that the content on the server has been updated. That is sad – and requires the client to pull for changes. This is not properly implemented in Kontact. Therefore we settled on a hack: whenever a resource in Kontact (addressbook or organizer) is deactivated and again activated, the content is reloaded. We decided to trigger exact that action with a refresh button.
Immediately the question comes up if that should not be triggered by time, but the usual work flow in an office is: “Hey, Joe, it’s me, John, have you seen the new appointment I just added to your calendar?” – a time base pulling would confuse the user, a button to pull would not. The bug to this problem is Bug #175409 – including the patches.

Kontact – problems still valid

While we were able to fix some of the problems (hurray Open Source) there are still some things left: we were not able to properly implement a working Free/Busy system with Kontact and Citadel. This is definitely due to some rather strange bus in Kontact, and will require quite some rewrite of Kontact’s resource plugins. But since Akonadi will be out soon the question is if that should be fixed now or if we should wait until Akonadi is out and then have another look at it.

Final words

Kontact and Citadel do work together somehow. And the bits which are missing can be fixed. However, it is clear that Kontact developed into something which was not planned originally, and that a revolutionary change is needed.
Akonadi will hopefully bring this change, but currently it is not there yet.

But even if Akonadi arrives right in time – and with plugins for all the needs we have – the underlying problem of the broken groupdav protocol remains. It is sad somehow that there are so many fine FLOSS Groupware servers out there but that there is hardly any protocol which is supported by all of them and delivers all the features the users want. No wonder that Outlook/Exchange has such a strong position right now.

A first look at Fedora 10

Recently Fedora 10 beta was released. I took the opportunity to update my rather old Fedora 8 to a more shiny and new system – with KDE > KDE 3.5.

Fedora 10 Beta was released more than a week ago. Among other things it features:

• New NetworkManager with connection sharing
• Improved printer handling
• Remote virtualization and easier virt storage
• Sectool, an auditing and security testing framework
• RPM 4.6, the first big RPM change in several years

Especially the first and the third part are pretty important to me since prefer to have networking “made easy” and also plan to get more experience regarding virtual machines. However, for me all the other goodies from Fedora 9 are also new since I never came around using Fedora 9 after some very bad experiences. So I decided to take the beta and give it a try with all goodies together: KDE 4.1, encrypted main partitions, kvm, new NetworkManager, PackageKit, etc.

And so far I must say I’m very pleased. The system pretty much works and KDE 4.1.x is really amazing. Of course, there are still quite some glitches, but after all, Fedora 10 is still in development, and many glitches seem to be X related and and my Nvidia card with the binary drivers is certainly a good candidate to be the source of some problems.

Some glitches

The main glitch in the system is the same reason why I haven’t yet filled bug reports against the glitches: I have no realiable browser. Firefox is crashing all the time with segmentation faults (unusable for more than maybe two minutes, but very hard to exactly reproduce), Konqueror sometimes seems to kill the DNS/internet connection somehow (!), and Arora often takes seconds to start actually loading a page – and additionally cannot log me in to my wordpress.com account. The fact that the NetworkManager applet (Gnome one) dies occasionally isn’t helping.
Another problem right at the start was that the GDM login manager somehow didn’t realize that I switched the keyboard layout to German. It took me quite some time to figure that out. Also, if I deactivate quiet boot and rhgb (and I always do that) the password dialog for my encrypted file system is lost in the kernel output.

The good things

Besides these annoying glitches (well, Beta is Beta) the system is stable – and promising: I’m looking forward to test the network connection sharing, which seems to be nicely integrated with the rest of the system. Also, the encryption is of course very important – and since Luks is used it would make sense to provide a GUI to easily add and remove other keys for the decryption. Last but not least KDE 4.1 is important for me – maybe the most important reaon of all. Even with Nvidia drivers it is working surprisingly well, but more about that later.

Conclusion

To summarize, Fedora 10 is shaping up quite nicely, and the Beta already runs much better for me than the Fedora 9 release. However, I wouldn’t advise normal users to start using Fedora 10 because it is still in heavy development – too much is still changing at the moment. I for example got an update right before the weekend which made it impossible to run the binary Nvidia driver on the system and had strange side effects when I connected an external monitor – sad if you want to watch DVD in the evening and show off KDE 4′s bling the next morning.