Sunday, May 31, 2015

A Development Environment Using Docker Machine, Docker Compose and Gulp

Today, we're going to set up a development environment for a ReactJS application using Docker containers. Even if you decide this isn't the best setup for your team, I strongly urge you to have at least 3 different environments for your software development efforts. It's normally some variation of local development, staging, and production. Development is for all of your experimental / bleeding edge tech. Staging must be a mirror of Production, serving as your UAT[1] platform. Given that setup, if it works in staging, it'll work in production. Before we continue, here's a diagram of the workflow we're implementing:

Prerequisites: Read up on Docker basics and get familiar with this Docker workflow. This guide was written for Mac OS X users.

Preparing the Docker-Machine

This first command and all that follow can be found at the bottom of gulpfile.js.

# clone the repo
git clone git@github.com:roblayton/reactjs-shopping-cart
cd reactjs-shopping-cart

# prepare the machine
npm install
gulp prepare

[20:16:22] Using gulpfile ~/repos/reactjs-shopping-cart/gulpfile.js
[20:16:22] Starting 'create-docker-machine'...
time="2015-05-31T20:16:22-04:00" level="info" msg="Creating SSH key..."
time="2015-05-31T20:16:23-04:00" level="info" msg="Creating VirtualBox VM..."
time="2015-05-31T20:16:30-04:00" level="info" msg="Starting VirtualBox VM..."
time="2015-05-31T20:16:30-04:00" level="info" msg="Waiting for VM to start..."
time="2015-05-31T20:17:17-04:00" level="info" msg="\"shoppingcart\" has been created and is now the active machine."
time="2015-05-31T20:17:17-04:00" level="info" msg="To point your Docker client at it, run this in your shell: docker-machine env shoppingcart | source"
[20:17:17] Finished 'create-docker-machine' after 55 s
[20:17:17] Starting 'prepare'...
[20:17:17] Finished 'prepare' after 11 μs

# follow the instructions to connect
# it's usually this command
eval "$(docker-machine env shoppingcart)"

# take note of the IP address
docker-machine ip shoppingcart

Provisioning the Dependencies

Next, we're going to spin up all of the resources our application needs in order to run. We're also going to be pre-populating our database with dummy data. The full list of resources are indicated in the docker-compose.yml, and are as follows:

Note: This will take a while. Good news is you just have to sit back and let it do it's magic.

# spin up the instances
gulp provision

...
[20:26:29] Finished 'compose' after 2.33 min
[20:26:29] Starting 'init-db'...
[20:26:30] Finished 'init-db' after 577 ms
[20:26:30] Starting 'provision'...
[20:26:30] Finished 'provision' after 16 μs

Before we move onto the next section, I'd like to talk a little more about the containers we just spun up. Prior to working with containers, you may have been used to a development workflow where you would set up your database and backend service either on your local machine or some host that served as your development environment. As you may already be well aware, setting up these dependencies is extremely painful, especially if you're doing so manually:

  1. Manually spinning up and tearing down dependencies takes even longer and is error-prone
  2. Multiple dependencies on the same host introduces conflicts
  3. Application-specific data can be cumbersome to initialize / migrate
  4. New releases of these dependencies can be cumbersome to manage, manually

Docker Machine solves the first two issues by making it quick and easy to allocate VM slices, effectively isolating each service in its own container. We solve the third issue by using a gulp command to prepopulate our database after it spins up, and the fourth issue is resolved by leveraging and pushing releases of our dependencies to the Docker Registry. After that, it's easy enough to indicate which releases you need in your docker-compose.yml.

Another positive side effect of containerizing your dependencies rather than manually setting them up is that it encourages you to develop one component of your system at a time. Imagine we had set up our Python server manually, off of the source code rather than a tagged released. We might be inclined to update the code on the spot as we were developing the front end component. This is bad practice.

It's better to develop each component in isolation. If you're developing a backend service and API, you should be using curl to make your requests and writing tests to mimick how the client behaves. If you're developing a client-side application, you should be hitting a dockerized build of the services or mocking the data the backend service returns.

Ideally, you want to establish a stable set of software. When new releases are ready, they go out one by one, making it easy to rollback if there are any issues. Going a few steps further, it's useful to have canarying[2] and A/B testing[3] in place, but we'll go over those in a different tutorial.

Developing in "Watch" mode

Now, we're ready to continue developing our application with gulp watching our files for changes.

# watch our files for changes
gulp develop

[22:11:44] Using gulpfile ~/repos/reactjs-shopping-cart/gulpfile.js
[22:11:44] Starting 'replace-html-src'...
[22:11:44] Finished 'replace-html-src' after 7.41 ms
[22:11:44] Starting 'copy'...
[22:11:44] Finished 'copy' after 1.64 ms
[22:11:44] Starting 'watch'...
[22:11:49] Finished 'watch' after 5.9 s
[22:11:49] Starting 'develop'...
[22:11:49] Finished 'develop' after 6.12 μs
Updated
Updated

Now you'll need to point your application's HTTP requests to the Python service running in the container. Change the request uri in src/js/App.js and src/js/Cart.js to match the IP address of your Docker Machine.

# Line 7 of App.js
request({
  uri: 'http://[shoppingcart.docker.machine.ip]:5000/products',

# Line 22 of Cart.js
request({
  uri: 'http://[shoppingcart.docker.machine.ip]:5000/product?id=' + productId,

The main reason we don't build a Docker container for our this application and link the containers is it would take too long to build as we're developing.

Now point your browser to reactjs-shopping-cart/dist/index.html. Whenever you make changes to any of the files in the root directory or in src, gulp will rebuild the application.

Teardown

# destroy and remove the docker-machine...
# ...and associated containers
gulp clean

[23:16:27] Using gulpfile ~/repos/reactjs-shopping-cart/gulpfile.js
[23:16:27] Starting 'destroy-docker-machine'...
time="2015-05-31T23:16:33-04:00" level="info" msg="The machine was successfully removed."
[23:16:33] Finished 'destroy-docker-machine' after 5.82 s
[23:16:33] Starting 'clean'...
[23:16:33] Finished 'clean' after 12 μs

That about concludes this tutorial. Hopefully, you either benefit from this workflow, or are inspired to come up with a better one. Until next time.

References:


1. ^ Wikipedia (18 June 2015). "User Acceptance Testing"
2. ^ whatis.techtarget.com (August 2012). "Canarying"
3. ^ Wikipedia (17 June 2015). "A/B Testing"

3 comments:

  1. So I took a peek at your github repo for this code, it is doing exactly what I was looking for. I haven't actually tried it but I just wanted to say I like your approach!! I am assuming my implementation didn't work because node starts up it own shell(my assumption) which makes complete sense why your approach works!!. Good job again!!

    ReplyDelete
  2. Nice writeup. I tried it on my machine and the db-init step failed due to me not having mongo available as a command line command. It would be nice for the backend env to not have that kind of dependency.

    ReplyDelete