Challenges of micro-service deployments
by Usman Ismail on 24 Feb 2016I recently took part in a panel discussion discussing continuous delivery of micro-services hosted by ElectricCloud with Daniel Rolnick, Darko Fabijan and Anders Wallgren. The discussion is worth a listen if you have a spare hour however I would like to discuss some of the interesting points that came out of the discussion regarding the challenges inherent in micro-services deployments. Team Overhead Micro services architecture has a lot of benefits around the development process. It allows large teams to move rapidly semi-independent from each other. In addition it allows rapid prototyping and iteration on small feature sets. However, micro-services put... more
Impact of Docker and Rancher on Network Performance
by Usman Ismail on 23 Feb 2016One of the concerns that is expressed with using Docker in production is the impact on network performance. In today’s article we present the results of our study of network performance on Ec2 platform. We compare the performance between: stock AWS Linux AMI, docker with ports exposed, Rancher’s overlay network and the recently released Swarm overlay network. Before we get into the test results I would like to quickly cover our testing setup. For the network throughput tests we will be using iperf 3.0.1 running on AWS C4.XLarge instances. We are using those instances as they provide a “high” level... more
Setting up Seyren in a docker container
by Usman Ismail on 09 Jul 2014We belive strongly in the measure everything philosophy of server management and because of this we have a lot of metrics about our system. One of the ways we make use of this data is to create real time alerts that allow us quickly identify and respond to production issues. For this we make extensive use of a tool called Seyren to create and manage real-time alerts for our server deployments. This article walks you through setting up a Seyren instance using the docker platform. We use docker because it provides us a way of repeatably deploying a Seyren instance... more
Writing your first Hadoop Job
by Usman Ismail on 07 Jul 2014This article covers a very basic map reduce job which counts the occurrence of words in a document. The code for the job is adapted from the example which comes with Hadoop. This article is a follow up to an earlier article which walks through setting up a single node hadoop cluster. If you don’t already have a running cluster please follow the steps in Setting up your first hadoop cluster. The source code that accompanies this article is available on github at techtraits/hadoop-wordcount. Project Setup We will be using Apache Maven to help write our hadoop job and will... more
Setting up your first Hadoop cluster
by Usman Ismail on 06 Jul 2014Big Data is a common term thrown about in tech circles these days, almost every mid to large size company wants to use Big Data analysis to solve all their problems. Although proclamations of the omnipotent of Big Data analysis should be treated with some skepticism, however, there are use-cases where such a system is invaluable. This article covers the basics of setting up Apache Hadoop cluster, Hadoop is one of the most popular big data platforms out there and familiarity with it will give you a good overview of working with Big Data. In a subsequent article we will... more