Traditional ("monolithic") operating systems provide most services by the kernel, examples are Unix-like systems, including Linux, but also Windows. The kernel itself typically has some layered structure, as indicated at the left of the below diagram.
The idea of a microkernel is to reduce the kernel to a bare minimum, and provide system services outside the kernel, as unprivileged user-mode servers, as shown at the right of the diagram. In such a system, applications use the kernel-provided message-passing IPC mechanism to request services.
The (potential) advantages of building operating systems on top of a microkernel are:
- a microkernel-based system can have a smaller trusted computing base (TCB) than a monolithic system. The reason is that the kernel is inherently part of the TCB, irrespective of whether all of its services are actually used. In a microkernel-based system, services that are not (directly or indirectly) used by an application do not have to be part of its TCB. The ability to keep the TCB small is the main reason we are using a microkernel as the foundation of our embedded OS platform;
- a microkernel-based system is easily extensible — just add more servers. It is also easier to adapt to changing circumstances, as servers can easily be replaced by others with a compatible interface but different implementation (and potentially policies);
- a microkernel-based system can be made highly configurable and small. As services are implemented by separate servers, unneeded services do not need to be included;
- microkernels support software-engineering techniques. As all system components (including individual system services) are user-mode programs running in their own address space, they can only be invoked via a well-defined IPC interface. This approach encourages the use of small interfaces, and the kernel-maintained protection boundaries (address spaces) enforce component encapsulation by hardware means;
- a microkernel-based system provides fault isolation, as a faulting component is restricted to damaging its own address space. This provides a basic mechanism for implementing fault-tolerant systems;
- as all communication in a microkernel-based system is via communication channels established by message-passing IPC, it is possible to interpose security monitors which impose system-defined security policies.
As all system services are invoked by an IPC operation, the performance of this operation is highly critical to the performance of systems that can be built on top of the kernel. In fact, a microkernel-based system has inherently higher service-invocation overheads than a traditional system, as four mode switches and two complete context switches are required in order to obtain a service. A monolithic system can deliver services by two mode switches and no context switch. This additional, hardware-dictated overhead is of the order of 50–500 cycles, depending on the architecture. The challenge of designing and implementing a good microkernel is to keep the actual IPC overhead close to this hardware-imposed cost.
Another challenge, which is a present research topic, is the design of the microkernel-based system so that it can deliver on the promises listed above.
The basic idea of a microkernel goes back to Per Brinch Hansen's Nucleus (Brinch Hansen, Communications of the ACM, 13, 1970), who built such a kernel and identified its properties and advantages.
The Mach project at CMU in the 1980's coined the term microkernel and popularised the approach, and a large number of similar projects existed around 1990. However, these so-called first-generation microkernels generally failed to live up to their promises, and the systems built on top were performing badly. High IPC costs were generally blamed.
Jochen Liedtke then showed that IPC could be made an order of magnitude faster than what had been thought possible (Liedtke, 14th ACM Symposium on Operating Systems Principles, 1993). He went on to show that the poor performance of Mach-based systems was a result of poor design and implementation of Mach, resulting in excessive cache footprints (Liedtke, 15th ACM Symposium on Operating Systems Principles, 1995). Together with Hermann Härtig's group in Dresden he then showed that it was possible to do a well-performing port of Linux as a de-privileged server on top of the L4 microkernel (Härtig et al., 16th ACM Symposium on Operating Systems Principles, 1997). L4 and other systems of its kind are often referred to as second-generation microkernels.
- Operating system kernel
- An OS kernel (usually just referred to as the kernel) is the part of the system which executes in the privileged mode of the underlying hardware. As such, the kernel is not subject to any hardware-enforced protection mechanisms, and in fact is responsible for using those hardware mechanisms to provide protection (if the system has protection at all).
- A small kernel that provides mechanisms for building arbitrary systems, with all other components running unprivileged in user mode. The microkernel is free of policies and only provides minimal mechanisms. A very strict definition (due to Liedtke) of a microkernel states: A feature is only allowed to be inside the kernel if otherwise it is not possible to build arbitrary secure systems on top.
- Trusted computing base (TCB)
- The part of the system which can bypass its security policy. Hence, secure operation of a system depends on the correct functioning of its TCB. Note that the TCB is not absolute, different applications have in general different TCBs. However, the kernel is always part of the TCB.