Deploy an IBM BLU Service on SoftLayer Cloud in One Click

Nowadays, if you’re a huge technology company and you’re not venturing into the cloud, you’re living in the past.  So it’s not surprising that many companies are investing heavily in cloud across the board.  This is also true for IBM, that decided in 2013 to acquire SoftLayer for approximately $2B, like many others such as HP who invested just as largely in their Helion play, and this is only one example of many.
SoftLayer, which was at that time, the world’s largest privately held cloud computing infrastructure provider, served thousands of customers.  Today SoftLayer (under the auspices of IBM) maintains a global cloud infrastructure platform spanning 16 data centers (and counting…) all over the world, and only intends to expand this reach to serve their strategic plans and penetration into the cloud market.

Cloud Application Orchestration

However, venturing into the cloud is just the first step into actually being able to provide a full and sustainable ecosystem of cloud users.  In order to have an ability to build dynamic and scalable cloud environments with complex application stacks on them, you ought to have diverse cloud tools, and among these orchestration tools.
To that end, IBM chose Cloudify to be their application orchestration enabler for various POCs and in diverse IBM products that are deployed on SoftLayer and on AWS. In this post, I will dive into Cloudify’s role in the deployment of BLU on SoftLayer, a very interesting technical use case.

Cloudify – multi-cloud out of the box. Give it a whirl..  Go

Cloudify –  The Cloud Orchestrator

IBM DB2 with BLU Acceleration is, slated by IBM to be the next generation database technology that changes the game for in-memory computing. Its primary play is to provide breakthrough performance by delivering instant insight from real-time operational data and historical data.  IBM was looking to deploy BLU Acceleration to SoftLayer, and was looking for a cloud application orchestrator for this purpose.  That was when Cloudify took on this challenge.

Cloudify as a Cloud Enabler

Cloudify enables IBM users to deploy a full-blown BLU instance in two modes: Trial (Anonymous) and Commercial (Solo).

Trial mode:

This enables users to launch a BLU instance and play around with it for several hours, where the number of hours is customizable and can be configured.  The VMs in this case, belong to IBM and are killed after several hours unless the user kills them first.

Solo mode:

This mode enables users to insert their own SoftLayer/AWS credentials and launch a new instance of BLU on their own SoftLayer/AWS account respectively.  Users’ decision to use SoftLayer and not AWS or vice versa is, obviously, made according to the users’ needs,requirements, budget and preference.
In either case, using the Solo mode costs money, but the users are only charged for the actual usage (i.e. provisioning of the VMs) and not for the software products (BLU).
Note, that users can use both SoftLayer and AWS and can easily transition between the two because of the Cloudify technological backbone that supports multi-cloud, although, one may assume that most users if not all of them, will settle for one of the above.

The Architecture

In a very high level explanation (and we’ll get deeper in a second), we used a web page that hosts a Cloudify Widget/Player as the front end in order to enable IBM users to configure their desired configuration (CPU, memory size, hard disk etc.) and desired cloud provider (SoftLayer or AWS).

The Widget Server

We chose the Play framework 2.0 as a backend (for VM pool management and UI serving) and after a while, when the requirements grew and evolved, we added Angular to the frontend and separated it from the backend. We use MailChimp’s API to populate a MailChimp distribution list and send email to the users via Mandrill, for the purpose of receiving credentials, URLs for the application and other communications required to run the service.
To manage the VM pool (trial mode) we selected Java, and yes, we are aware of the fact that there are NodeJs cloud libraries.  However, since we, in all honesty, are more experienced in Java; using a threaded implementation seemed be the best approach for the job and has ultimately proven itself.
So we stayed with Java and we used Spring MVC with Jetty to expose a REST service for managing a VM pool. The UI is written with the latest technology and it uses Yeoman generator for AngularJS, which is pretty nifty. (You can check out this post of ours on just this UI workflow automation).

Cloudify and SoftLayer

We’ve developed a Cloudify-SoftLayer cloud driver that enables us to do pretty much everything on SoftLayer, from launching new VMs (CCIs,Flex images) and dedicated bare metal servers, to specifying the amount of memory, CPU etc. for each machine.
All of the above can be invoked via an interactive mode, a non-interactive mode and, obviously, via the Cloudify Widget.
The Cloudify cloud driver enables users to deploy, configure, orchestrate, monitor and self-heal any application on any SoftLayer account and on any of SoftLayer’s data centers, by specifying SoftLayer item IDs, which provides a pretty neat automation layer on top of SoftLayer.

The Chef Server

In general, Cloudify enables users to use any of their applications internally and externally, according to their needs.  That means, for example, that if your stack is comprised of several tiers/applications, then you can decide that one (or some) of them will not be a part of the runtime stack and they will be external services and applications that can be accessed at anytime by the other components.
An external application will not be configured nor deployed by Cloudify, but it can be referenced and used inside Cloudify and still be a part of your environment.  The BLU info and binaries, in our case, are retrieved from a Chef server, so for IBM we’ve developed a recipe that configures and deploys a Chef server. Eventually the Chef server was externalized to Cloudify by IBM – however, this can be internalized at any time without a hassle.  In the current setup the Chef server has been implemented as an external application, where the components are able to access it, and all the rest of the components’ orchestration is performed by Cloudify, where Cloudify injects the Chef server’s URL to the other components so they are able to access it. (i.e. The configuration, deployment, orchestration, monitoring and all runtime management of all the other components are all done by Cloudify.)
The Cloudify recipes (blueprints) are flexible in that aspect as well, the same recipe can be used with an internal Chef server and with an external one. It’s just a matter of configuration and preference.

Cloudify & BLU

Cloudify installs the “BLU Acceleration for Cloud” instances on SoftLayer by invoking Cloudify BLU recipes that have been developed for this purpose.  These recipes enable users to specify regions from which the BLU binaries are retrieved, thus they enable the BLU application to be efficiently deployed on any of SoftLayer’s regions. The Cloudify BLU recipes wrap and run a set of Chef Cookbooks and Roles.
The Chef Cookbooks and Chef Roles of the BLU instances can be retrieved either from an external Chef-server or from a Chef server which has been installed by a Cloudify recipe that has been developed for this purpose.

In Conclusion

This service, whose enabler is Cloudify, is currently available both in English and in Chinese and users can deploy it on any of SoftLayer’s data centers and on any of AWS’ data centers as well.
Adding support for more languages other than English and Chinese is relatively easy for us and we will add it in the near future and/or upon request. This project overall has been an interesting and challenging one, and we continue to develop and extend its functionality.