File-based Databases?

File based databases use the server as a file server only; all data manipulations occur by accessing those common files over the network in each client process. With file-based databases, multiple user data access is synchronized using lock files or by physically locking certain portions of the data file. This has several disadvantages:

Data integrity is compromised - when a client process hangs or a network glitch occurs in the middle of data modification there is a good chance that corruption will occur. Often times all clients must be shutdown to rebuild indexes or recover from that data corruption.

Performance bottlenecks occur - as more users try to access the shared files across the network, bandwidth is multiplied. This is because all the filtering of the data occurs on the client side, so the server must send over all the data even if the client does not need certain portions of that data. The locking mechanisms that are used to synchronize data access are much slower than their Client/Server counterpart and restrict the ability to cache data of any kind.

Scalability is hindered - faster server hardware will not help a file-based database that much. There is just too much dependency on the overall network throughput and client side processing power. There is also no way to pool connections to share resources therefore scalability is nearly impossible to achieve.

The locking mechanisms employed by file-based databases have additional repercussions that should be discussed. Using a lock file requires a lot of overhead. The database must first acquire an exclusive lock on the file, modify the lock information and flush it back to disk. If the database locks portions of the data file via API calls, then problems can occur. These locks are enforced by the file server and their implementations are dependent on the network protocol. Also, Linux and Windows file servers have different locking semantics; therefore using SAMBA can cause problems.

The biggest repercussion is that there is no way to cache data. All operations must have locks at all times as there is no way to detect when one client has changed a value. Therefore, there is no way you can invalidate a cached data buffer. This makes reading more expensive than it is using a client/server model.

Why Client/Server?

Client/Server databases have a server process that is the only process that manipulates the data files directly. Clients communicate with the server in a well defined protocol and the server processes these requests on the client's behalf. All the data access synchronization is handled internally by the server. This architecture has several key benefits:

Data integrity is preserved - the server process has been engineered to be rock solid. If a client application crashes or a network problem occurs the server can detect this and recover from this situation. The server will rollback the changes and keep the database in a consistent state.

Performance is optimized - as more users access a client/server database the server is able to synchronize this access in an optimal way. Servers can use very fast Operating System Kernel locks or in the case of NexusDB employ Snapshot transactions which do not create locks at all. Servers can also optimize disk reads which prevent the disk from thrashing; the heads will scan in an elevator like fashion rather than jumping all over the place in a first-come first-serve basis. Network bandwidth is also minimized as the data sent to the client is only what the client needs.

Scalability is maximized - because the server can be on a totally different physical machine and the processing occurs on that machine, it is possible to increase the power of that hardware and realize corresponding performance gains. It is also possible to create n-tier applications that employ connection pooling which enhances scalability due to the sharing of connection resources.

NexusDB for the future

As more and more projects become distributed and require web integration a Client/Server database is the only logical choice. NexusDB's architecture has been designed to support these distributed applications. These applications include: desktop GUI applications that access the NexusDB server on your internal network; Intraweb/WebSnap ISAPI dll's that pool connections to a NexusDB server; SOAP-based "Web Services" applications that utilize NexusDB as its data store. NexusDB is perfectly suited to be your database in a distributed world.