Database

Kernels, Contexts, Threads, and Extensible Database Architecture

If you are building a new database management system or simply want to write better database applications, you should be aware of the extensible architecture of operating systems and database managers.

Threads and Processes

The Oracle Database architecture for threads and processes differs depending on whether Oracle is running on a UNIX or Windows platform. On UNIX machines, Oracle Database uses a process to implement each background task. Each connection to the database results in the operating system spawning a process for that session. Client processes, such as sqlplus, connect to the TNS listener (tnslsnr), which does a fork() and then exec() in the database kernel program in $ORACLE_HOME/bin/oracle. The newly created process is known as a server process or shadow process.

Webcasts

On a Windows machine, there is a corresponding process (.exe) for each Oracle Database instance or SID. Each background task is a thread inside a single process. All background, server, and client processes are threads of a master Oracle Database process and all threads share resources.

For Sybase Adaptive Server Everywhere (ASE), connections to the database equate to tasks that are each assigned an engine. Prior to version 15.7, Sybase ASE executed each engine in a separate process. Starting with ASE 15.7, Sybase has offered a threaded kernel with each engine being a thread of a process and the engines communicating via shared memory. Engines run only user tasks, not I/O. Sybase has always used the thread-based model for ASE running on Windows platforms.

Continual Evolution of Database Kernels

SQL platforms have gained a large share of the database market over the past three decades. The instinct for wanting to build a better mousetrap is endemic in the computing and software community. It's no surprise that we've seen the development of new architectures and new kernels for SQL, NoSQL, NewSQL and in-memory database platforms.

The original code base for MySQL dates back more than 15 years. Several factors (modern CPU architectures, the growth of cloud computing, and a wide following for Web applications) contributed to a desire to refactor MySQL as a lightweight SQL DBMS.

Drizzle is a leaner version of MySQL built using a microkernel architecture and plugins. To build Drizzle, developers have moved code from the MySQL kernel into plugins. MySQL found success with pluggable storage engines, such as InnoDB. Drizzle extends the use of plugins for storage engines, authentication, replication, a function engine, string functions, catalog functions, schema dictionary, IPV6 functions, math functions, a multithread scheduler, and other features. When it starts up, Drizzle loads about four dozen default plugins.

Another interesting development is Monet, a main memory database kernel. Monet has roots in Troll, a relational kernel developed about three decades ago. MonetDB version 4 became an open source project in 2004. MonetDB is an implementation of a column-store database kernel, a departure for traditional SQL databases that are row oriented. Although column stores are traditionally associated with analytics and business intelligence, MonetDB is capable of supporting ACID transactions. MonetDB has an algebraic database kernel that makes it versatile enough to support multiple query languages, including SQL and XQuery. It's based on a layered architecture that provides multiple interfaces for plugging in extensions. Support for math functions, strings, and temporal data types is provided by linked-in libraries. MonetDB is the subject of ongoing research and a recent interesting development has been DataCell, a project that provides sensor stream-processing capabilities built over MonetDB.

Another innovation has come from McObject, maker of the eXtremeDB embedded database product. McObject offers eXtremeDB-KM, a version that deploys in kernel mode to operate in the same address space as the operating system. This provides direct access of data structures to kernel processes, eliminating the overhead of context switches for applications requiring extreme performance.

Database Plugins

The extensible database manager, with an architecture that supports plugins, gives developers the ability to adapt a database to an applications' requirements. We can write pluggable indexing and storage engines, for example. We can also write plugins that extend the SQL language, but that is a subject for a future article.

Aside from Drizzle, we've also seen several NewSQL platforms that represent a refactoring of the relational database. Plugins and refactoring provide a reminder that the database is not an example of static technology.

Ken North is a well-known expert in database technologies. He regularly contributes to Dr. Dobb's on the subjects of databases and software, including trends and techniques.

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task.
However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

Video

This month's Dr. Dobb's Journal

This month,
Dr. Dobb's Journal is devoted to mobile programming. We introduce you to Apple's new Swift programming language, discuss the perils of being the third-most-popular mobile platform, revisit SQLite on Android
, and much more!