Replica Set Elections - MongoDB

Replica sets use elections to determine which set member will become primary. Elections occur after initiating a replica set, and also any time the primary becomes unavailable. The primary is the only member in the set that can accept write operations. If a primary becomes unavailable, elections allow the set to recover normal operations without manual intervention. Elections are part of the failover process.

In the following three-member replica set, the primary is unavailable. One of the remaining secondaries holds an election to elect itself as a new primary.

Elections are essential for independent operation of a replica set; however, elections take time to complete. While an election is in process, the replica set has no primary and cannot accept writes and all remaining members become read-only. MongoDB avoids elections unless necessary.

If a majority of the replica set is inaccessible or unavailable to the current primary, the primary will step down and become a secondary. The replica set cannot accept writes after this occurs, but remaining members can continue to serve read queries if such queries are configured to run on secondaries.

Factors and Conditions that Affect Elections

Replication Election Protocol

New in version 3.2: MongoDB introduces a version 1 of the replication protocol (protocolVersion: 1) to reduce replica set failover time and accelerates the detection of multiple simultaneous primaries. New replica sets will, by default, use protocolVersion: 1. Previous versions of MongoDB use version 0 of the protocol.

Heartbeats

Replica set members send heartbeats (pings) to each other every two seconds. If a heartbeat does not return within 10 seconds, the other members mark the delinquent member as inaccessible.

Member Priority

After a replica set has a stable primary, the election algorithm will make a “best-effort” attempt to have the secondary with the highest priority available call an election. Member priority affects both the timing and the outcome of elections; secondaries with higher priority call elections relatively sooner than secondaries with lower priority, and are also more likely to win. However, a lower priority instance can be elected as primary for brief periods, even if a higher priority secondary is available. Replica set members continue to call elections until the highest priority member available becomes primary.

Members with a priority value of 0 cannot become primary and do not seek election. 

Loss of a Data Center

With a distributed replica set, the loss of a data center may affect the ability of the remaining members in other data center or data centers to elect a primary.

If possible, distribute the replica set members across data centers to maximize the likelihood that even with a loss of a data center, one of the remaining replica set members can become the new primary.

Network Partition

A network partition may segregate a primary into a partition with a minority of nodes. When the primary detects that it can only see a minority of nodes in the replica set, the primary steps down as primary and becomes a secondary. Independently, a member in the partition that can communicate with a majority of the nodes (including itself) holds an election to become the new primary.

Vetoes in Elections

Changed in version 3.2: The protocolVersion: 1 obviates the need for vetos. The following veto discussion applies to replica sets that use the older protocolVersion: 0.

For replica sets using protocolVersion: 0, all members of a replica set can veto an election, including non-voting members. A member will veto an election:

• If the member seeking an election is not a member of the voter’s set.
• If the current primary has more recent operations (i.e. a higher optime) than the member seeking election, from the perspective of another voting member.
• If the current primary has the same or more recent operations (i.e. a higher or equal optime) than the member seeking election.
• If a priority 0 member [1] is the most current member at the time of the election. In this case, another eligible member of the set will catch up to the state of the priority 0 member member and then attempt to become primary.
• If the member seeking an election has a lower priority than another member in the set that is also eligible for election.

[1]	Hidden and delayed imply priority 0 configuration.

Non-Voting Members

Although non-voting members do not vote in elections, these members hold copies of the replica set’s data and can accept read operations from client applications.

Because a replica set can have up to 50 members, but only 7 voting members, non-voting members allow a replica set to have more than seven members.

For instance, the following nine-member replica set has seven voting members and two non-voting members.

A non-voting member has a members[n].votes setting equal to 0 in its member configuration:

{

"_id" : <num>

"host" : <hostname:port>,

"votes" : 0

}

Priority 0 Replica Set Members

A priority 0 member is a secondary that cannot become primary. Priority 0 members cannot trigger elections. Otherwise these members function as normal secondaries. A priority 0 member maintains a copy of the data set, accepts read operations, and votes in elections. Configure a priority 0 member to prevent secondaries from becoming primary, which is particularly useful in multi-data center deployments.

In a three-member replica set, in one data center hosts the primary and a secondary. A second data center hosts one priority 0 member that cannot become primary.

Priority 0 Members as Standbys

A priority 0 member can function as a standby. In some replica sets, it might not be possible to add a new member in a reasonable amount of time. A standby member keeps a current copy of the data to be able to replace an unavailable member.

In many cases, you need not set standby to priority 0. However, in sets with varied hardware or geographic distribution, a priority 0 standby ensures that only qualified members become primary.

A priority 0 standby may also be valuable for some members of a set with different hardware or workload profiles. In these cases, deploy a member with priority 0 so it can’t become primary. Also consider using a hidden member for this purpose.

If your set already has seven voting members, also configure the member as non-voting.

Priority 0 Members and Failover

When configuring a priority 0 member, consider potential failover patterns, including all possible network partitions. Always ensure that your main data center contains both a quorum of voting members and contains members that are eligible to be primary.

Priority 0 Member Configuration

When updating the replica configuration object, access the replica set members in the members array with the array index. The array index begins with 0. Do not confuse this index value with the value of the members[n]._id field in each document in the members array.

NOTE

MongoDB does not permit the current primary to have a priority of 0. To prevent the current primary from again becoming a primary, you must first step down the current primary using rs.stepDown().

Procedure

This tutorial uses a sample replica set with 5 members.

WARNING

• The rs.reconfig() shell method can force the current primary to step down, which causes an election. When the primary steps down, the mongod closes all client connections. While this typically takes 10-20 seconds, try to make these changes during scheduled maintenance periods.
• To successfully reconfigure a replica set, a majority of the members must be accessible. If your replica set has an even number of members, add an arbiter to ensure that members can quickly obtain a majority of votes in an election for primary.

1

Retrieve the current replica set configuration.

The rs.conf() method returns a replica set configuration document that contains the current configuration for a replica set.

In a mongo shell connected to a primary, run the rs.conf() method and assign the result to a variable:

cfg = rs.conf()

The returned document contains a members field which contains an array of member configuration documents, one document for each member of the replica set.

2

Assign priority value of 0.

To prevent a secondary member from becoming a primary, update the secondary member’smembers[n].priority to 0.

To assign a priority value to a member of the replica set, access the member configuration document using the array index. In this tutorial, the secondary member to change corresponds to the configuration document found at position 2 of the members array.

cfg.members[2].priority = 0

The configuration change does not take effect until you reconfigure the replica set.

3

Reconfigure the replica set.

Use rs.reconfig() method to reconfigure the replica set with the updated replica set configuration document.

Pass the cfg variable to the rs.reconfig() method:

rs.reconfig(cfg)