Linus Torvalds included patches into the mainline tree which implement a stable userspace driver API into the Linux kernel.
The stable driver API was already announced a year ago by Greg Kroah-Hartman. Now the last patches where uploaded and the API was included in Linus’ tree. The idea of the API is to make life easier for driver developers:
This interface allows the ability to write the majority of a driver in userspace with only a very small shell of a driver in the kernel itself. It uses a char device and sysfs to interact with a userspace process to process interrupts and control memory accesses.
Since future drivers using this API will run mainly in userspace there is no need to open up the source code for these parts. Also, drivers can be re-used even after kernel changes because the API will remain stable.
The background motivation for the inclusion of such a stable API comes form the embedded world: there embedded drivers are often closed source and just developed for a single kernel version and are not maintained over time. Using the new API these drivers could be used for a longer time.
And in fact such an API has already been developed by people from the embedded industry to make their life easier. But it was just developed for a single device and not as a generic interface. Such developments are now unnecessary.
However, DMA transfer between userspace and kernelspace is not yet implemented. This means essentially that drivers which involve high traffic are not an option yet. So graphic drivers as well as file system drivers and similar cannot use this API at the moment.
Some people might ask now why not simply allow closed source kernel drivers directly in the kernel, since they are allowed in userspace now anyway. But there is a huge difference between these two types of drivers – while userspace drivers can be controlled in a certain way and cannot trash the kernel kernelspace drivers could do that. A quite good explanation about the topic can be read at LWN.
The other way around one could ask if this will mean the end for Linux (for example according to the LWN article). But again, kernelspace and userspace is very different – an d with the new API the important part (the kernelaspace part) still has to be Open Source! Besides, this API does not introduce anything which was not possible before: as already mentioned such an API was already developed quite some time ago, and was actively used.
But I must also admit that while I see the need for such an API I still prefer hardware vendors who simply make the drivers Open Source and bring them upstream.
Thanks to German heise.de for a detailed article about the topic.
/home/liquidat 
Nice!
I hope for a Linux micro-kernel in the future, with all this cool features!
I think this stable API is a huge chance for Linux. Of course it would be much better if the hardware vendors published free Linux drivers. However, in my opinion, it is better to increase the Linux market share by making it easier to provide proprietary drivers instead of inhibiting people to use Linux by not supporting their hardware. Maybe, in the future Linux will be strong enough to force hardware vendors to publish Open Source drivers, but this point is IMO not yet reached, so we should – as I already said – accept unfree drivers at the expense of not (yet) having a completely free system.
“”it is better to increase the Linux market share by making it easier to provide proprietary drivers instead of inhibiting people to use Linux by not supporting their hardware.””
Besides the ‘moral’ ‘Free software’ gnu-style arguements against proprietary software, there are serious technical and practical issues against proprietary drivers.
Mainly they revolve around the fact that if there is something wrong with them, nobody can fix them. Nobody knows how they work, nobody knows how the hardware works, so if the driver breaks for you nobody from Ubuntu or Redhat or Kernel.org will be able to help you. Unless you can get a hold of the original (most of time nameless) developer working at broadcom or ATI then your up shit’s creek without a paddle.
Ever try to get a hold of a developer from ATI to fix a driver issue that only affects you and your paticular hardware/software combination? Either in Windows or Linux? Or nvidia? Or Texas Instruments? Ya, email their customer service and just see how little they care about you. Your not worth their time. Unless it affects a significant number of users they aren’t going to fix it. It’s not worth their time.
In comparision with open source drivers I’ve gotten attention from the people that wrote them. I filed bugs, they fixed them. Every day people working on Ubuntu or Debian submit patches and different fixes to various drivers.
This is why Redhat refuses to support hardware configurations that require proprietary drivers. This is why Dell will refuse to ship server hardware that uses proprietary drivers. They know and understand this in the server room.. Linux with open source drivers is significantly cheaper for both the end user and the OEM to support then Linux with closed source drivers.
The Linux developers do require the assistance of the hardware manufactures to create full featured drivers in a timely fasion. If not then it requires reverse engineering and the drivers are going to lag behind.. like Open source 3d drivers for Nvidia and ATI. So it creates the illusion that Linux devs aren’t able to keep up with and create quality drivers sometimes.. But this is, indeed, a illusion.
For both the end user, the hardware manufacturer, and the people that need to do commercial support Linux with OSS drivers has a significant advantage in stability, ease of use, and often performance. This advantage is one of things that will make Linux much more appealing for end users versus Windows.
All of this is purely technical, it still doesn’t take into effect any sort of moral, ethical, or legal reasons for not using proprietary drivers.
So saying that Linux needs to support proprietary drivers for success is a red herring. If end users have no choice, then yes, support from proprietary drivers is better then no support at all. But this is hardly a good situation.
The ultimate solution, the real solution, is that if you want to run Linux you should have hardware that is well supported by OSS drivers. This is real setup that makes Linux appealing.
Right now the only time you NEED to have proprietary drivers for Linux desktop is high-end 3d acceleration. That’s it.
For Wifi, for ACPI, motherboards, NIC cards, RAID drivers, etc etc etc. For almost any sort of hardware you want to use there exists good and high quality hardware with OSS support. But for 3D the only choices for high-end graphics are ATI and Nvidai, and they both refuse to support Linux with OSS drivers.
(for regular 3D graphics, for beryl/compiz and simple games then onboard Intel will work acceptively and it has OSS drivers)
Having userspace API isn’t going to help that any.
Realy, this API is designed for embedded developers anyways. It has very little to do with proprietary drivers. In fact it has not much to do with it at all. There is certain times with embedded development were userspace drivers is require for stability, control, and certain performance reasons. It does not matter at all if the source code is closed or open, there are technical reasons why you want userspace.
If you don’t beleive me.. then understand this:
In Linux, video drivers are ALREADY USERSPACE DRIVERS. Right now Video cards, for both 2D and 3D performance in X Windows are userspace drivers. They’ve always have been, and probably always will be.
Suprised?
Using the DRI/X/Linux driver model it works like this… I’ll use Intel drivers for a example.
Then xf86-video-intel video drivers is the 2-D X driver. It’s file is i915_drv.so and is provided by your X.org release. It’s completely userspace and newer versions support both XAA and EXA acceleration models.
The i915_dri.so file provides for 3D acceleration. They call it the ‘DRI driver’. DRI drivers are based on the open source Mesa OpenGL stack.. they take Mesa, accelerate what they can on the video card, and do the rest with software rendering. (OpenGL is a huge API, no video card accelerates all of it, only the portions that matter to performance in games and applications)
Now, having closed source drivers for those poses no legal problem at all. X.org and Mesa licenses allow for their use in proprietary software.
The only portion of the driver that needs to be in the kernel is called the DRM driver. This driver is what allows those userspace drivers to access the hardware and get their acceleration. It’s kept to a more-or-less minimum.
So the Linux driver model for video cards do allow for userspace drivers.
And guess what?
ATI’s driver still sucks and still is hugely complicated to install. It’s a total crap shoot if it works or not… And Nvidia still refuses to fix bugs and problems, some of them that cause crashes and security issues, because these problems are only noticed by a minority of users. And in both cases those companies still shovel large amounts of buggy, closed source code into your kernel were it can cause all sorts of problems from kernel panics to FS corruption (if your very unlucky).
So don’t think that userspace drivers are going to solve anything….
> In Linux, video drivers are ALREADY USERSPACE DRIVERS.
> Right now Video cards, for both 2D and 3D performance in X > Windows are userspace drivers. They’ve always have been,
> and probably always will be.
>
> Suprised?
>
> Using the DRI/X/Linux driver model it works like this… I’ll use
> Intel drivers for a example.
The NVIDIA drivers are not userspace drivers, part of them runs in userspace, but a _huge_ part runs in the kernel. Certainly _not_ only a DRI driver.
And I vaguely remember having to load a fglrx.ko module for some friends of mine using ATI cards, so I assume it’s the same for ATI.
Yes, exactly my point.
The XGI company makes a proprietary driver for Linux. They have open source DRM driver for the kernel, open source 2-D driver, and a closed source DRI driver (due to code obtained from another company).
Nvidia and ATI have always had the option of doing userspace video drivers, but they don’t. Do you know why? I don’t. There isn’t any sort of performance advantage to having kernel code for these sort of hardware.
They do it because they feel like it, I guess. It’s certainly much more likely to cause a crash and corrupt data the way they are doing it now. This is why Microsoft has moved from in-kernel video drivers to userspace video drivers for Vista.
The open source video drivers for X Windows have always been mostly userspace. In Linux, lots of USB drivers are userspace. Also, via fuse, you have file systems that are userspace.
Having drivers in userspace, with a fairly stable API, is nothing realy remarkable. _This_ paticular patch, for making generic API for embedded developers, is rather new, but userspace driver concept for Linux isn’t. For drivers that are in the kernel, there is a very good reason for it.
Once the hardware manufacturers and open source driver developers get together and work together then that produces the highest quality driver possible. Better then Windows or any sort of proprietary driver for Linux, generally. This is what is most desirable and is what everybody needs to aim for.
For selecting hardware, unless your realy a Linux geek, the best thing you can do is simply purchase a laptop or desktop from some place like System76 or Dell that already has all of it’s hardware supported by OSS drivers and pre-installs Linux. The only thing that remains closed source, is those nvidia drivers, because for high performance 3d Nvidia is the only game in town. (for normal 3d desktop and such Intel onboard works fine and is supported by OSS) That and the modem driver in Dell’s laptops is closed source for whatever reason, but most people don’t realy care about that.
This way it will ‘Just Work’. Just like when you go to a Apple store and buy a Apple computer all you have to do is open the box and plug it in.
It’s funny that Microsoft have moved the video driver from the kernel space to the userspace, given that it was a design decision to do this early on in the days of Windows 2000.
nate, about the proprietary video drivers:
For them building upon Mesa was not an option at all. Mesa was outdated and several years behind development for quite some time therefore relying on Mesa would have been crazy.
So they used their own OpenGL implementation – and this was closed.
“”It’s funny that Microsoft have moved the video driver from the kernel space to the userspace, given that it was a design decision to do this early on in the days of Windows 2000.””
For some drivers, yes. For video drivers, no.
Video drivers as of XP were in-kernel. In fact it’s pre-Windows 2000. The desire for userspace drivers comes from earlier versions of NT. In fact very early of versions NT was a real Microkernel, but Microsoft dropped that when they realised that microkernels and high performance are two things that will never meet.
For Vista WDDM is the new driver model.
“”nate, about the proprietary video drivers:
For them building upon Mesa was not an option at all. Mesa was outdated and several years behind development for quite some time therefore relying on Mesa would have been crazy.
So they used their own OpenGL implementation – and this was closed.””
That’s completely irrelevent to anything I said.
My point is that for video drivers in Linux, which (for modern high-end 3d) is the only major class of hardware were you can’t find good OSS drivers for, it’s always been possible for Nvidia and ATI to do the vast majority of the driver in userspace. Which they do not; they put a lot of code in the kernel. Whose OpenGL stack they choose to use is pretty much irrelevent to this.
Having this API for userspace drivers isn’t going to accomplish anything in terms of making Linux easier for end users, besides a small amount of embedded developers. For them there is certain classes of hardware and tasks were userspace drivers are technically desirable. Licensing isn’t realy a issue. If licensing or ‘stable api’ or anything like that was the issue, then this patch wouldn’t exist.
Sorry I don’t want to be a hard-ass or anything. I understand what your saying and such. I want to be friends. 🙂
I’ll try another way to say it…
Userspace drivers aren’t going to make Linux any easier for folks because..
* Closed source drivers are, generally, technically inferior to OSS drivers with all else being equal. For Linux to be the most successfull it can be requires openess on the part of the drivers and hardware manufacturers supporting Linux.
* For graphical drivers, it’s always been possible to do most of it in userspace. (closed vs open doesn’t mater for this). XGI has closed source 3d graphics driver for Linux and it’s userspace. Nvidia and ATI could do much more of their drivers in userspace, but they do not. This patch isn’t going to affect them one way or another.
* For this patch in paticular it’s designed for specific circumstances that don’t apply much to desktop anyways.
* For the most part OSS drivers, weither in or out of the kernel, exist for almost all classes of hardware besides high end 3D graphics. Weither wifi, sata, sound, or anything else you can always find good and modern hardware with decent OSS drivers.
They are all seperate points. I am sorry for not being clearer with them before, but communication through writing is not my strong point. I’ll leave this blog alone now. 🙂
And on a side note; for the most part the simple act of installing a OS, any OS, on random hardware is a singificant enough a barrier to prevent Linux adoption on the desktop. The real answer is to have Linux pre-installed and supported on the OEM level, then it will have much more of a chance for higher popularity.
Ok, I lied. Almost.
This post on LKML will help to illistrate the point behind this paticular patch:
http://lkml.org/lkml/2007/7/19/557
So far it’s only for industrial I/O cards. Very simple devices, often only a few made, just flipping switches. Nothing like your typical consumer device that is hugely complicated and requires lots of other things in the kernel to work properly.
nate: don’t worry, I don’t take such things personal. I’m just a bit surprised because it looks like you read my post like I would expect things massively to change now.
But I don’t. I’m very well aware that this API is mainly for embedded. And that userspace drivers have been possible before.
So thanks for the detailed comments. 🙂
nate,
Having a small well-defined interface for userspace drivers will make it very easy to snoop on the driver’s interaction with kernel space and easier to reverse-engineer in the long run.
@nate:
I can’t believe you wrote an entire essay to reply to a comment. You have too much time on your hands.
Listen and learn …
The pros –technical and in terms of reliability– of implementing drivers in user space are covered in http://en.sevenload.com/videos/DqzIRi2/Andrew-Tanenbaum-Design-of-microkernel-OS
As whether they should be binary, of course not! … for various reasons including for reliability –for those of you that are pragmatists– and security –for those of that are paranoid. Not that this thinning out of linux will not help facilitating the easy development of binary drivers and that this might help consolidate, promote, and gain corporate popularity for linux.
nate: I don’t think one can go very far in a shit creek – paddle or not 🙂
what do you think about this?
greetings from sofia
Because I’m interested in the technical background. Besides the technical interesting part Linux is about the Freedom of choice – and that is very important.
Thats great isnt it!!! Being a developer on the hardware end of things…its a great boon for a person like me to seamlessly write device drivers for Linux without much kernel compiling and stuff…
Soham, the API is targeted at embedded device developers.
Also, the compile time will not be really influenced by this API. The only thing which matters is that you now don’t need to care about different kernel versions.
One point that is often overlooked about the desire to have user-space drivers is not that it necessarily allows *system* vendors to release closed source drivers, but that it allows *hardware* vendors to release bare-board drivers without the limitations of GPL. Ie. nothing documents and enables the hardware you’re selling better than well-documented driver code that runs and makes the h/w flex the muscles it claims to have! For both internal verification as well as customer evaluation, being able to provide a single reference “driver” that can be used as a starting point on a variety of systems is a big win. This presumes that inefficiencies or inelegances that can be corrected by adapting for the native OS would then be done by whoever feels like doing it, based on the knowledge gleaned from the user-space driver. For this to be the case, the driver needs to be liberally licensed so that GPL linux drivers as well as non-GPL non-linux drivers can be built using the reference, without license contamination issues for anyone.
On the other hand, h/w vendors that take the time/effort to get a linux kernel driver working reliably can find the exercise frustrating (I speak from experience) – you can’t hand the resulting driver to BSD, Solaris, QNX, […] developers for all the obvious GPL reasons. I know full well that authors can release their code under non-GPL as well as GPL conditions as a way of getting around this problem, but that frequently doesn’t wash with corporate legal opinion, I’ve seen it happen – if it is a GPL linux driver using linux interfaces, they won’t touch it no matter what GPL-free assurances you concoct for them. So … does the vendor repeat the entire exercise for non-GPL systems as well?
Note also that there should be nothing stopping other kernels (BSDs et al) from exporting compatible user-space interfaces so that this class of drivers be truly portable among *nix flavours.
Now whether that’s what vendors use this for is another question entirely, I totally agree that it would be deflating if it merely becomes a means to release binary-only drivers, DRM, and myriad other abominations of the sort …
splitenz, I never heard of any hardware developer who would loved to release their drivers as anythings even more open than the GPL. Do you have any example there?
Also, if they would be ok with GPL they could simply dual licence it or put it under Public Domain.
Btw.: “nothing documents and enables the hardware you’re selling better than well-documented driver code”
For most of the developers out there (BSD, Linux) technical documentation and specifications would be enough already. But even that is missing.
liquidat: w.r.t. examples, while I can’t point to those that most directly concern me, good examples can be found around the board support for any of the linux archs, ie.support for SoCs, different boards, controllers, etc. The vendors in those cases inevitably have to provide functioning code for “bring up” on whichever systems their prospective customers are running, and that cost is not really their bread and butter so much as a marketing overhead (if you see what I mean). You will see that a lot of the arch-specific churn and maintenance in linux is coming from contributors working within the h/w (rather than the s/w) food-chain.
As for dual-licensing and what-not, as I said – I know this is true to the spirit and letter of the law, but corporate legal types are a peculiar breed at the best of times. I can’t claim to fully grok the line of thinking, but I believe it goes something like this: if the code you give me under license X (where X != GPL) is also available “out there” under GPL, and it also uses linux kernel interfaces (and code-style, and … [etc]) then we are more vulnerable to accusations of impropriety than we would choose to be. Be it chronic paranoia or just some subtlety of risk management that I’m too crude to follow, it doesn’t matter – “giving them the linux driver under another license” is not necessarily a workable solution for everyone at all times. And that’s one good thing for the idea of being able to provide user-space bare-board drivers; they’re easily ported (relatively speaking) and the licensing can be sufficiently lubricated for any application you have in mind …
I meant initially we had to compile the kernel module, load it in the kernel and so on and so forth…but with the incoming of API in the front its a bit more relaxation time for embedded developers…and yeah I know this is for the embedded developers, and but being on the hardware end and not knowing much of the intricacies of kernel management, its a great thing for ppl like us to take part in the embedded development too, not wholly at least partly….
Whoa, Linux is becoming a modern kernel and emerging from the middle ages? Unbelievable. Now I just hope that Nvidia/ATI and other vendors (webcam vendors too!) will be able to start using this sort of stuff fast.
Actually, the same scheme I proposed with gid should work with uid if you create a special user for the applications that should get Internet access. Gid was just what immediately came to my mind for this purpose.
liquidat
This comment is most certainly misplaced and belongs to the application wise filtering post.
This whole thing is simply a substitute for doing the RIGHT thing and improving the compatibility across versions of the Linux kernel. There is absolutely no excuse for the cavalier attitude the Linux development community takes to upward compatibility. No other O/S “breaks” so many things each time it comes out with a new version. If the mainframe O/S’s, for example, took this same approach, the users would revolt.
TWQ: While I do agree that there could be done more in regards to the compatibility the reasons why there is no stable API in kernel space are totally clear and well outlined. When you judge them as wrong that is your opinion, but the reasons still exist.
Besides: most things which were broken in the last months were broken for pretty good reasons. Like the WLAN sub system. Or the soon to be expected break for video drivers because the subsystem is reworked as well.
Still, if you use hardware fully supported by Linux (as in “drivers are part of the kernel”) you will have no problem with these breaks because you will not see them.