Sunday, June 14, 2015

A MongoDB Cluster Sharding Tutorial

This is going to be a straightforward guide to setting up a MongoDB sharded cluster[1] on your local machine.

Prerequisites: You'll want to be familiar with setting up VirtualBox with Vagrant and MongoDB Sharding. This guide was written for Mac OS X users.

Provisioning Our MongoDB Instances

To spin up the two MongoDB instances, we're going to be cloning a repo that contains a Vagrantfile and Ansible playbook that allows us to provision virtual machines with mongod and mongos installed. This is the only step of the tutorial we're going to automate. The rest of the steps will need to be performed manually, in order to give you an intimate understanding of how sharding works.

# clone the repo
git clone mongocluster
cd mongocluster

vi Vagrantfile

We're using this repo because it comes with an Ansible playbook we'll be using to install MongoDB. To set up our cluster, you'll want to update your Vagrantfile with the following content:

# now spin up the instance
vagrant up

Setting Up the Config Servers

After about 5-10 minutes, we should now have the following cluster:

name type
configsrv1 Config server
configsrv2 Config server
configsrv3 Config server
query1 Query router
query2 Query router
shard1 Shard server
shard2 Shard server
shard3 Shard server

Config Servers

Now that all eight of our MongoDB VMs are running, we can set up the config servers by running the following commands on each configserver. Make sure you log into configserver1, configserver2, and configserver3 separately.

# ssh in
vagrant ssh configsvr1

# create a metadata directory
sudo mkdir -p /data/configdb

# start up the configuration server
sudo mongod --configsvr --dbpath /data/configdb --port 27019

You should now have a mongod process running in the foreground, waiting for connections on port 27019. Make sure to perform the above steps on the other two config servers, before moving on. You'll have to open a new terminal window and cd into your mongocluster folder for each.

Setting Up the Query Router Instances

# ssh in
vagrant ssh query1

# stop mongodb
sudo service mongodb stop

# start the query router service
sudo mongos --configdb,,

You should now have a mongos process running in the foreground on port27019. Make sure to perform the above steps on query2, before moving on. You'll have to open a new terminal window and cd into your mongocluster folder before repeating these steps.

Adding Shards to the Cluster

# ssh in
vagrant ssh shard1

# connect to a query router
mongo --host --port 27017

# add each shard
# can run these commands on this machine
sh.addShard( "" )
sh.addShard( "" )
sh.addShard( "" )


1. ^ Wikipedia (15 May 2015). "Shard (database_architecture)"

No comments:

Post a Comment