Use Case : Development Environment On Demand

The general idea

Development teams need quite a lot of infrastructure in order to get their work done. They need source control systems, build systems, servers to assist them in tracking requirements and realised value, machinery for testing purposes and quality assurance, for performing proof of concepts or just as a training environment for the users of the software.

In the majority of the cases getting all of these resources in place before they are required in the development cycle can be quite a challenge,  purchasing the hardware consumes quite a lot of a project’s budget and procurement of it is often quite slow. Especially if the project is rather quite short, many project managers will not consider it an option to set up much of an environment because it weighs so heavily on their budget or the project’s delivery schedule suffers from it.

Why is cloud a good solution?

It turns out that even though development teams need quite a lot of different servers in their daily job, they only use the hardware for pretty small amounts of time.

First of all, the machines are only needed during the length of the project, which may be as small as a few months or even weeks. Furthermore, the machines are only utilized during working hours, I consider 16 hours a day to be the maximum if the team has a flexible schedule otherwise it’s probably even less. Besides that, team members aren’t doing the same job all of the time. Early in the project coding activities may be more abundant and there will be more need for source control systems and build servers than that there will be a need for testing resources for example. Further on in the project the need shifts away from these development oriented systems towards more test and quality assurance infrastructure.

Furthermore, different types of projects need different kinds of development components (like sql server, sharepoint, biztalk, etc..) depending on the architecture of the solution. Finally some resources are only used for a few hours, when you want to do a proof of concept to proof that the system scales out to a few hundred of machines for example?

So as you can clearly see, the development environment’s resources have very different usage patterns, and that is exactly where cloud computing fits in.  Cloud allows you to manage computing resources in a flexible way while you only pay for the resources effectively used.

What cloud offering do you need?

In order to set up a development environment such as described above, you need a public ‘Infrastructure As A Service’ offering where you can install and start virtual machines on demand. Examples of such offerings today are Amazon EC2, IBM’s Smart Business Development And Test Cloud, FlexiScale, GoGrid, and probably many more that I just don’t know about yet…  Microsoft has also announced, at PDC last year, that it will include this capability in it’s Azure offering, but today it isn’t available yet.

Cost Model

Now you probably wonder, what will it cost me to set up such an environment? Well that depends on your needs obviously, but let’s make a quick scenario to show you what it would cost for a small to mediums sized team with a common development environment based on Amazon’s Ec2

Imagine a development team of about 15 people delivering a product in a 6 months timeframe.

They would probably require 1 source control server to manage the created artifacts. In my view this server would have to be a ‘Large Instance’ type of machine which is used during the working hours, so that would cost about $0.48/h * 8h * 130 days = $499,2

In order to perform their builds a continuous integration approach works best in my view as the machines need to be available only when the team is operational: In order to handle the load I would probably foresee 2 continuous integration servers of the ‘Small Instance’ type. This would cost about $0.12/h * 2 * 8h * 130 days = $249,6

In order to perform integration tests of the different components of the system they will probably need something along the line of 2 development integration servers which can probably be small instances. However these machines are only required on average 1 or 2 days per week. So it would cost something in the lines of $0.12/h * 2 * 8h *  52 days = $99,84. Note :these 2 days could be spread out over the entire week when the integration tests are automated as a part of the build process, it would cost about the same if the build process turns of the instances after the tests have run.

When the developers have tested the integration of the components, the software is usually tested for functional completeness by the functional testing team. These team members aren’t performing tests all of the time either, most of their time goes to defining tests, documenting them, creating reports from the results etc. So I estimate their resource usage on about 3 days a week, for about half of the projects lifespan, which would result in $0.12/h * 2 * 8h * 39 days = $74,88

Besides the testing environments, the team usually needs a system that looks quite similar to the production environment in order to perform stress and load tests for example, or to delivery end user training etc. This environment is usually only used a couple of weeks, let’s say 4, per project. But the hardware has to resemble the production hardware, so this will probably become extra large instances, resulting in $0.96 * 2 * 8h * 20 days = $307,2

For the total sum of $1230,72, the price of about half a physical mid-range server, you could have a full development environment for a small to medium sized project. So next time you need to define a budget for a development project, think cloud !

Cloud is not hosting

To get on some common ground here, I decided to do this first introductory post on cloud computing. Many people think of cloud computing as yet another form of hosting, where it isn’t really different from the Application Service Provider model that has been quite popular for almost 10 years now. But there are significant differences that separate cloud from ASP:

  • Self service: Cloud service offerings allow you to get the needed capacity without the intervention of a human being. So there is no agent that you need to call in order to get a service, you can do it all yourself using a web browser and a credit card.
  • Pay for use: You only pay for the resources that you effectively use.
  • Elastic: At any point in time you are able to increase or reduce the number of services.
  • Open standards: Cloud Services must allow access through open standards, so that interchange of data is as simple as possible, preventing data lock-in.
  • Location independent: By default you don’t know, and preferably you shouldn’t know, the location where your service is hosted. For some kinds of data there are exceptions though because of legal issues. Some providers offer solutions for this problem through a feature called geo-location.

A second main difference with the ASP model, is that there are multiple deployment models to cloud. It’s not just about applications. There are 3 official models, and one commonly accepted one:

  • Infrastructure As A Service (IAAS): This boils down to on demand virtual machines, hosted in a full-fledged data center. But as a consumer you are still responsible for the maintenance of each instance.  Today this space is dominated by Amazon’s EC2, but if I’m not mistaken Windows Azure should support this scenario as well later this year.
  • Platform As A Service (PAAS): In this model, the infrastructure is abstracted away by a platform such as Azure. This platform allows you to run your own applications and services but all of the work related to machine maintenance has been alleviated from you. Big players in this space are Microsoft’s Windows Azure and Google’s AppEngine.
  • Software As A Service (SAAS): The natural incarnation of the former ASP model, applications hosted in the cloud and available on demand. Microsoft is pretty present in this space with it’s BPOS offerings as is
  • Business Process As A Service (BPAAS): This is the unofficial, but commonly accepted model, where entire business processes are available on demand. An example of this is LOKAD, which will do analysis and forecasting of time-series for you on demand.

I hope this overview helps you to understand the differences between cloud and earlier models, and I hope that it serves as a basis for future discussions.