Search This Blog

replica sets, Replication with mongoDB

Before we get into details of mongoDB replica sets, let's understand what is replication and why is it needed? Replication is a configuration that creates multiple copies of data by "replicating"(transferring) data from one site to one or more sites, provides redundancy and maximum availability for databases and its applications. Database is synchronized to remote sites in real time as soon as changes are loaded in to source database. These remote sites are called Disaster Recovery sites and they are strictly recommended to be spread across different geographical regions, different cities, countries, and even different continents. Nevertheless, some IT configurations do also have deployments called "Near DRs", a DR site that could be located in the same city.

Today's Disaster Recovery deployments

Is DR only used in a real disaster scenario? Well, Let me share my thoughts of Disaster Recovery in today's context. A DR site in today's super fast growing world is a necessity for all models of businesses and data, there's no escape of this expense for Enterprises. What about ROI, does one have to wait for a real disaster to strike to consume it, is that only built for sleeping but otherwise only to wait for a disaster and show its real deal? Well, long gone are those days. Most of the database platforms today support read replicas, while "The read feature" comes at an additional expense for some (like Oracle's active data guard), and others are free by default (mongoDB read replicas, Amazon RDS Read Replicas, and so on). Therefore, a DR site is daily driver of the business, power house for reporting, contributes to the growth of the business significantly. Alright, let's get into the topic.

Replication with mongoDB

The deployment of replication configuration in mongoDB is called, a replica set. Each replica set has below nodes.Primary node: This is the source of everything, source of input, is always in READ WRITE mode.Secondary node: This is the copy of the Primary node, a replica copy, is always in RECOVERY mode and also serves READ operations when requested. It is always recommended to have similar resource profile, hardware capacity, as Primary node. Multiple secondary copies could be maintained based on different requirements.Arbiter node: This is only to help the failover in the event of unforeseen primary node outage, this is optional. Arbiters come at cheap price, they do not require a dedicated hardware and they do not hold copy of primary but helps in election of new optimal Primary by responding to heartbeat and election requests by other replica set members in replication.
(Image by mongoDB Inc.)
Each replica set can have up to 50 members but only maximum of 7 members can participate in election.How it works?
"mongod", a core background daemon process, handles synchronization of changes between all replica set instances. "mongod" on Primary records all types of database changes in a fixed size collection called oplog (operations log), "oplog.rs". For people with background of Oracle, this could be compared to redo log file, and transaction log file in SQL Server. The "mongod" daemon of every secondary consumes entries from Primary's oplog and applies them to respective secondary. In addition to this, all replica set members also send heartbeat signal to every other replica set instance and get oplog entries. This replication process is asynchronous.

Setup

The replica set configuration is pretty straight forward. The replica setup initialization consists of below two taks, they are triggered automatically by mongod process, no manual intervention is required.
-Initial synchronization
-Ongoing replication

The demo example of the setup consists of 1 primary, 3 secondaries, and 1 arbiter node. For mongod to replicate oplog, each mongod instance should run on different port but IP could remain same. Using this advantage, for demonstration and testing purposes, we will configure replica set on same host. A Production deployment must be set up across different geographical regions for maximum availability and fault tolerance.

I assume you already have a working mongo installation setup of version 3.6. If it is not too late, Please go ahead and download here, and follow these instructions for installation. Be it installation of software, or, setup of replica sets, it does not make much difference in regards to steps that are followed. Here's my demo environment, FYR.OS and version : Oracle Linux Server release 7.4mongoDB version : mongoDB 3.6.3Software install location : /mongo/mongohome/binDatabase location : /mongo/data/db

Step(1) : Create directories.[mongo@cloud2 ~]$mkdir -p /mongo/data/db/rep1 /mongo/data/db/rep2 /mongo/data/db/rep3 /mongo/data/db/rep4 /mongo/data/db/arb
Verify if directories are created.[mongo@cloud2 ~]$ls /mongo/data/db/arb rep1 rep2 rep3 rep4
"rep1, this is the Primary database node. This could already exist in a Production deployment case. "rep2,rep3,rep4"these are secondary database nodes in replica set. "arb" is the arbitrary node.Note : For production deployments, /mongo/data/db must be a dedicated file system on each node with necessary storage configuration such as RAID, Disk type, and so on.

And the initialization syntax returns following shell prompt.MongoDB Enterprise rs:OTHER>
Hit enter and then prompt changes to.MongoDB Enterprise rs:PRIMARY>
It goes through so many transitions during this time, you can see that in the snippet below, from STARTUP to SECONDARY to PRIMARY.

Let's validate if this collection is imported into Primary and is also replicated to all secondaries.

On Primary.MongoDB Enterprise rs:PRIMARY>db.restaurants.count()
887565On Secondaries.
Secondaries can not be queried unless we set a read preference option in the mongo session as shown below with "rs.slaveOk()".On rep2[mongo@cloud2 ~]$mongo --host 192.168.1.80 --port 27018MongoDB shell version v3.6.3connecting to: mongodb://192.168.1.80:27018/MongoDB server version: 3.6.3Server has startup warnings:2018-03-18T11:54:14.697-0500 I CONTROL [initandlisten]2018-03-18T11:54:14.697-0500 I CONTROL [initandlisten] ** WARNING: Access control is not enabled for the database.2018-03-18T11:54:14.697-0500 I CONTROL [initandlisten] ** Read and write access to data and configuration is unrestricted.2018-03-18T11:54:14.697-0500 I CONTROL [initandlisten]MongoDB Enterprise rs:SECONDARY>rs.slaveOk()MongoDB Enterprise rs:SECONDARY>use reptestswitched to db reptestMongoDB Enterprise rs:SECONDARY>db.restaurants.count()887565On rep3[mongo@cloud2 ~]$mongo --host 192.168.1.80 --port 27019MongoDB shell version v3.6.3connecting to: mongodb://192.168.1.80:27019/MongoDB server version: 3.6.3Server has startup warnings:2018-03-18T11:54:15.282-0500 I CONTROL [initandlisten]2018-03-18T11:54:15.282-0500 I CONTROL [initandlisten] ** WARNING: Access control is not enabled for the database.2018-03-18T11:54:15.282-0500 I CONTROL [initandlisten] ** Read and write access to data and configuration is unrestricted.2018-03-18T11:54:15.282-0500 I CONTROL [initandlisten]MongoDB Enterprise rs:SECONDARY>rs.slaveOk()MongoDB Enterprise rs:SECONDARY>use reptestswitched to db reptestMongoDB Enterprise rs:SECONDARY>db.restaurants.count()887565On rep4[mongo@cloud2 ~]$ mongo --host 192.168.1.80 --port 27020MongoDB shell version v3.6.3connecting to: mongodb://192.168.1.80:27020/MongoDB server version: 3.6.3Server has startup warnings:2018-03-18T11:54:15.859-0500 I CONTROL [initandlisten]2018-03-18T11:54:15.859-0500 I CONTROL [initandlisten] ** WARNING: Access control is not enabled for the database.2018-03-18T11:54:15.859-0500 I CONTROL [initandlisten] ** Read and write access to data and configuration is unrestricted.2018-03-18T11:54:15.859-0500 I CONTROL [initandlisten]MongoDB Enterprise rs:SECONDARY>rs.slaveOk()MongoDB Enterprise rs:SECONDARY>use reptestswitched to db reptestMongoDB Enterprise rs:SECONDARY>db.restaurants.count()887565
As we could see, the count of number of documents in restaurants collection matches across the replica set configuration. This confirms the successful replication.

Important commandsdb.isMaster() confirms the role of the mongod instance. It also helps finding IP and Ports information of the replica set and so much more.To find all primary and secondary hosts.MongoDB Enterprise rs:SECONDARY>db.isMaster().hosts["192.168.1.80:27017","192.168.1.80:27018","192.168.1.80:27019","192.168.1.80:27020"

Popular posts from this blog

Sometimes, it is difficult to deal with Windows Platform as it drains the hell out of us!! Recently, I have come across a situation where one of my client's requirement was to input Arabic language into Oracle Database [or] read/retrieve the output into Various client Applications such as PL/SQL Developer/SQL Developer/TOAD etc.. Inputting the non-english language into Database has never been difficult as we are given plenty of language options within our beloved Oracle Database, but, the problem lies within the complex Windows OS when user wanted to view the data in his/her beloved language, in Applications such as PL/SQL Developer or SQL Developer etc.. This post is all about it.

Character Set, Character encoding, & Code point
Yes, it is a group of characters that is recognised by the Hardware through the OS Interface. Every character is allocated a number, called a code point, these code points will be represented in the computer by one or more bytes. So, every character h…

About Data GuardEfficient business operations, quality customer service, compliance with government regulations, and safeguarding corporate information assets all require high levels of data protection and data availability. Thus it is no surprise that data protection and data availability are among the top priorities for enterprises of all sizes and industries. A set of questions one needs to ask him/herself is what If a disaster impacts ongoing business transactions, How soon can those systems be back to business again, How much Data is affordable to loose, is it acceptable? The only answer for all these questions to position a proper "Business Continuity Plan (BCP)" in place so that it helps business to grow inline with its consumer expectations. Technically, this is where Oracle's Data Guard plays a vital role that ensures superior reliability, and rock solid performance. DataGuard is Oracle's complete Disaster Recovery Solution which can reliably deliver aggressi…

Some errors make us panic, nervous, and beyond belief when we positively attempt to resolve a panic situation. Specific to case of Clusterware where it attempts to resolve a conflict by removing dead resources from the cluster but it suffers because of lack of additional resources to complete the task, and makes the whole situation even worst so everything that has in contact with the key resource also suffers. The key resource what we are talking is a RAC Database and the Clusterware under a panic situation is Oracle Clusterware.