DragonFly BSD explained

DragonFly
Developer:Matthew Dillon
Family:BSD
Working State:Current
Source Model:Open source
Latest Release Version:3.0.1
Language:English
Package Manager:pkgsrc
Supported Platforms:IA-32, x86-64
Kernel Type:Hybrid
Userland:BSD
License:BSD
Preceded By:FreeBSD

DragonFly BSD is a free Unix-like operating system created as a fork of FreeBSD 4.8. Matthew Dillon, an Amiga developer in the late 1980s and early 1990s and a FreeBSD developer between 1994 and 2003, began work on DragonFly BSD in June 2003 and announced it on the FreeBSD mailing lists on July 16, 2003.

Dillon started DragonFly in the belief that the methods and techniques being adopted for threading and symmetric multiprocessing in FreeBSD 5 would lead to poor system performance and cause maintenance difficulties. He sought to correct these suspected problems within the FreeBSD project. Due to ongoing conflicts with other FreeBSD developers over the implementation of his ideas his ability to directly change the FreeBSD code was eventually revoked. Despite this, the DragonFly BSD and FreeBSD projects still work together contributing bug fixes, driver updates and other system improvements to each other.

Intended to be the logical continuation of the FreeBSD 4.x series, DragonFly's development has diverged significantly from FreeBSD's, including a new Light Weight Kernel Threads implementation (LWKT) and a light weight ports/messaging system. Many concepts planned for DragonFly were inspired by the AmigaOS operating system.

System design

Kernel

Like most modern kernels, DragonFly is a hybrid, containing features of both monolithic and microkernels, such as the message passing capability of microkernels enabling larger portions of the OS to benefit from protected memory, as well as retaining the speed of monolithic kernels for certain critical tasks. The messaging subsystem being developed is similar to those found in microkernels such as Mach, though it is less complex by design. DragonFly's messaging subsystem has the ability to act in either a synchronous or asynchronous fashion, and attempts to use this capability to achieve the best performance possible in any given situation.

According to developer Matthew Dillon, progress being made to provide both device input/output (I/O) and virtual file system (VFS) messaging capabilities that will enable the remainder of the project goals to be met. The new infrastructure will allow many parts of the kernel to be migrated out into userspace; here they will be more easily debugged as they will be smaller, isolated programs, instead of being small parts entwined in a larger chunk of code. Additionally, the migration of select kernel code into userspace has the benefit of making the system more robust; if a userspace driver crashes, it will not crash the kernel.

System calls are being split into userland and kernel versions and being encapsulated into messages. This will help reduce the size and complexity of the kernel by moving variants of standard system calls into a userland compatibility layer, and help maintain forwards and backwards compatibility between DragonFly versions. Linux and other Unix-like OS compatibility code is being migrated out similarly. Multiple instances of the native userland compatibility layer created in jails could give DragonFly functionality similar to that found in UML, though DragonFly's virtualization does not require special drivers to communicate with the real hardware on the computer.

Threading

As support for multiple processor architectures complicates symmetric multiprocessing (SMP) support, DragonFly BSD limits its supported platforms to x86 and x86-64, with both single processor and SMP models. Since version 1.10, DragonFly supports 1:1 userland threading (one kernel thread for every userland thread), which is seen as a relatively simple and easy to maintain solution. Inherited from FreeBSD, DragonFly also supports SMP multi-threading imported.

In DragonFly, threads are locked to CPUs by design, and each processor has its own LWKT scheduler. Threads are never preemptively switched from one processor to another; they are only migrated by the passing of an inter-processor interrupt (IPI) message between the CPUs involved. Inter-processor thread scheduling is also accomplished by sending asynchronous IPI messages. One advantage to this clean compartmentalization of the threading subsystem is that the processors' on-board caches in Symmetric Multiprocessor Systems do not contain duplicated data, allowing for higher performance by giving each processor in the system the ability to use its own cache to store different things to work on.

The LWKT subsystem is being employed to partition work among multiple kernel threads (for example in the networking code there is one thread per protocol per processor), reducing competition by removing the need to share certain resources among various kernel tasks.

Shared resources protection

In order to run safely on multiprocessor machines, access to shared resources (like files, data structures) must be serialized so that threads or processes do not attempt to modify the same resource at the same time. In order to prevent multiple threads from accessing or modifying a shared resource simultaneously, DragonFly employs critical sections, and serializing tokens to prevent concurrent access. While both Linux and FreeBSD 5 employ fine-grained mutex models to achieve higher performance on multiprocessor systems, DragonFly does not. Until recently, DragonFly also employed spls, but these were replaced with critical sections.

Much of the system's core, including the LWKT subsystem, the IPI messaging subsystem and the new kernel memory allocator, are lockless, meaning that they work without using mutexes, and operate on a per-CPU basis. Critical sections are used to protect against local interrupts and operate on a per-CPU basis, guaranteeing that a thread currently being executed will not be preempted.

Serializing tokens are used to prevent concurrent accesses from other CPUs and may be held simultaneously by multiple threads, ensuring that only one of those threads is running at any given time. Blocked or sleeping threads therefore do not prevent other threads from accessing the shared resource unlike a thread that is holding a mutex. Among other things, the use of serializing tokens prevents many of the situations that could result in deadlocks and priority inversions when using mutexes, as well as greatly simplifying the design and implementation of a many-step procedure that would require a resource to be shared among multiple threads. The serializing token code is evolving into something quite similar to the "Read-copy-update" feature now available in Linux. Unlike Linux's current RCU implementation, DragonFly's is being implemented such that only processors competing for the same token are affected rather than all processors in the computer.

DragonFly uses a slab allocator, which requires neither mutexes nor blocking operations for memory assignment tasks and, unlike the code it replaced, is multiprocessor safe. It was eventually ported to be utilized outside the kernel in a replacement to the old userland malloc implementation.

Virtual kernel

Since release 1.8 DragonFly has a new virtualization mechanism similar to UML, allowing to run another kernel in the userland. The virtual kernel (vkernel) is run in completely isolated environment with emulated network and storage interfaces, thus simplifying testing kernel subsystems and clustering features.

The vkernel has two important differences from the real kernel: it lacks many routines for dealing with the low-level hardware management and it uses C standard library (libc) functions in place of in-kernel implementations wherever possible. As both real and virtual kernel are compiled from the same code base, this effectively means that platform-dependent routines and re-implementations of libc functions are clearly separated in a source tree.

The virtualized platform vkernel runs on is built on top of high-level abstractions provided by the real kernel. These abstractions include the kqueue-based timer, the console (mapped to the virtual terminal where vkernel is executed), the disk image and virtual kernel ethernet device (VKE), tunneling all packets to the host's tap interface.

Package management

DragonFly previously used FreeBSD's Ports system for third party software, but since the 1.4 release, NetBSD's pkgsrc is the official package management system. With pkgsrc, the DragonFly developers are largely freed of having to maintain a large number of third party programs while still having access to up to date applications. The pkgsrc developers also benefit from this arrangement as it helps to ensure the portability of the code.

CARP support

The initial implementation of Common Address Redundancy Protocol (commonly referred as CARP) to was finished in March 2007. As of 2011, CARP support is integrated into DragonFly BSD.

HAMMER file system

See main article: article and HAMMER. Alongside with Unix File System, which is typically the default file system on BSDs, DragonFly BSD supports HAMMER file system. It was developed specifically for DragonFly BSD to provide a feature-rich yet better designed analogue of then increasingly popular ZFS.

HAMMER supports configurable file system history, snapshots, checksumming, data deduplication and other features typical for file systems of its kind. Though its performance is currently beyond the similar file systems like ZFS or btrfs, it is recognised as an interesting and perspective option.

The next generation of HAMMER file system (HAMMER2) is being developed by Dillon, who stated his intent to focus on this project for the whole year 2012., the dedicated branch in DragonFly's source code repository was created. The earliest usable state of the file system is expected by July 2012; the final release is planned for 2013.

devfs

In 2007 DragonFly BSD received a new device file system (devfs), which dynamically adds and removed device nodes, allows accessing devices by connection paths, recognises drives by serial numbers and removes the need for pre-populated /dev file system hierarchy. It was implemented as a Google Summer of Code'2009 project.

Application snapshots

DragonFly BSD supports Amiga-style resident applications feature: it takes a snapshot of a large, dynamically linked program's virtual memory space after loading, allowing future instances of the program to start much more quickly than it otherwise would have. This replaces the prelinking capability that was being worked on earlier in the project's history, as the resident support is much more efficient. Large programs like those found in KDE Software Compilation with many shared libraries will benefit the most from this support.

Development and distribution

As with FreeBSD and OpenBSD, the developers of DragonFly BSD are slowly replacing K&R style C code with more modern, ANSI equivalents. Similar to other operating systems, DragonFly's version of the GNU Compiler Collection has an enhancement called the Stack-Smashing Protector (ProPolice) enabled by default, providing some additional protection against buffer overflow based attacks. It should be noted that, the kernel is no longer built with this protection by default.

Being a derivative of FreeBSD, DragonFly has inherited an easy-to-use integrated build system that can rebuild the entire base system from source with only a few commands. The DragonFly developers use the Git version control system to manage changes to the DragonFly source code. Unlike its parent FreeBSD, DragonFly has both stable and unstable releases in a single source tree, due to a smaller developer base.

Like the other BSD kernels (and those of most modern operating systems), DragonFly employs a built-in kernel debugger to help the developers find kernel bugs. Furthermore,, a debug kernel, which makes bug reports more useful for tracking down kernel-related problems, is installed by default, at the expense of a relatively small quantity of disk space. When a new kernel is installed, the backup copy of the previous kernel and its modules are stripped of their debugging symbols to further minimize disk space usage.

Distribution media

The operating system is distributed as a Live CD and Live USB (full X11 flavour available) that boots into a complete DragonFly system. It includes the base system and a complete set of manual pages, and may include source code and useful packages in future versions. The advantage of this is that with a single CD you can install the software onto a computer, use a full set of tools to repair a damaged installation, or demonstrate the capabilities of the system without installing it. Daily snapshots for both i386 and x86-64 architectures are available from the master site for those who want to install the most recent versions of DragonFly without building from source.

Like the other free open source BSDs, DragonFly is distributed under the terms of the modern version of the BSD license.

Release history

VersionDateChanges
3.0
  • multiprocessor-capable kernel became the default
  • HAMMER performance improvements
  • TrueCrypt-compatible encryption support
  • dm-crypt replaced with compatible BSD-licensed library
  • enhanced POSIX compatibility
  • device driver for ECC memory
  • major network protocol stack and SMP improvements
  • ACPI-related improvements
2.10
  • Giant lock removed from every area except the virtual memory subsystem
  • HAMMER deduplication
  • GCC 4.4
  • bridging system rewritten
  • significant performance improvements
2.8
2.6
  • swapcache
  • tmpfs ported from NetBSD
  • HAMMER and general I/O improvements
2.4
2.2
  • HAMMER officially production-ready
  • major stability improvements
  • new release media: LiveDVD and LiveUSB
2.0
  • major HAMMER improvements
1.12
  • OpenBSD's hardware sensors framework imported from FreeBSD
  • Bluetooth stack
  • GCC 4.1
  • DragonFly Mail Agent
  • support for the 386 CPU dropped
  • preliminary x86-64 support (not functional)
  • experimental HAMMER support
1.10
1.8
  • virtual kernel implementation
1.6
  • new random number generator
  • IEEE 802.11 framework refactored
  • major giant lock, clustering and userland VFS improvements
  • major stability improvements
1.4
  • GCC 3.4
  • pkgsrc used by default
  • Citrus imported from the NetBSD
1.2
1.0
  • technology showcase
  • new BSD Installer
  • LWKT subsystem and lightweight ports/messaging system
  • mostly MP safe networking stack
  • lockless memory allocator
  • variant symlinks
  • application checkpointing support.

See also