author: Eliran Malka
– Workflow Automation
– Cloud Automation
This post was originally posted at Dr. Dobbs.
Labor of love
When we started working on Cloudify’s new version, 3.0 – which is a multi-module project based on Python and YAML, that will have a whole new UI among many other features; as a by-product the dev team was exposed to many new and diverse technologies that required diverse skill sets. Before setting out on this new project, and with agility in mind, we realized that with all this coming-to-life of: more and more new modules in the product, the constant flow of new features and requirements, and the accumulating specs, we’re gonna have to come up with solutions to simplify the project management. We then decided to align our development workflows so that we could better work uniformly across teams and environments. This workflow automation, essentially would enable us to occupy ourselves with the work at hand, rather than the ramp up of getting acquainted with new methodologies each time a new project rolls around, eventually letting us do what we do best: programming.
Since we found ourselves writing new projects for nearly every component in the new product, we were required to cope with various configurations and constellations of projects. Before aligning our workflows, we found ourselves often times spending too much time just becoming familiar with new environments and formations, and with the handover of a project, than the actual tasks at hand. This cost us valuable development time (check back here for a link to an awesome post on other cool hacks for saving developer time), and so all those occupations really needed to be unified in order to help us remain in context and in focus as a team. In this post, I’m going to dive into the process, the workflows we instilled, and the tools we use to get the job done, ultimately to make our dev teams as portable as possible.
Ok, then what’d you do?
Cloudify 3.1 – workflow automation for cloud apps made easy. Give it a whirl.. Go
Your workflow, how goes it?
After you’ve built your project (or cloned it), you then need to do two things; fetch external dependencies and assemble it all together. To handle dependencies in our environment, we use the NodeJS Package Manager to include third-party libraries for the backend, and Bower that manages client-side components. As Bower, NPM and Grunt already have their own configuration files included in each project, this means that after we’ve decided on our components, and defined them in these configuration files, we then simply call an NPM install (which, surprisingly, involves running npm install in the command line), a Bower install (guess how?), and then call a Grunt build (well, you know the drill) and POOF, all of our components are included and injected into the project automagically (yeah, I hate that word, too). This lets us immediately start working.
Ready made components, you say?
Yes. We are developers. We hate to reinvent the wheel. So we use what other developers, like us, have written in the past to solve issues just like ours. For instance, if a developer needs some sort of a slider component to be used in a web page, she can just look up the bower repository with bower search, find an appropriate slider, let’s say angular-slider, and then all she’d have to do is bower install angular-slider – and it’s integrated into the project.
To be honest, some of the time, those developers that write components are ourselves. We have our very own Kitchen Sink of reusable components we built for generic use, that we can use in any project in the exact same manner – they’re all published into Bower!
Neat. How do you run it?
Glad you asked. Well, while developing in the client, we use preview servers – in our case provided by Grunt’s LiveReload and Watch plugins, so we really don’t have to refresh the browser page at any point – which is annoying for developers worldwide.
Once Grunt’s development server is up and running, it will watch for any change in the project files and reload that specific resource in the web-app using websockets. This allows us to work continuously, Grunt rewires the app into the browser as we go along, and no more clicking the refresh button. Ever. It’s all very neat.
Ok, what about integration?
There’s a lot to be said about continuous integration, that I won’t dive into here (which you can find more on in another excellent post), but I’ll just share that we use TravisCI and QuickBuild to manage our tailored/nightly builds.
Nice. Is that it?
Nope. There are also profiles! A profile means coloring an environment with certain configuration flavor, for a specific need. When developing, our environment has two flavors of configuration, one for production, and the other for development – which can also be used for testing. This separation provides us with convenient mocking abilities when testing or developing (e.g. why use a full-blown backend server when all you need is a fixed response? Mock it up with Jasmine!) and also protects us from incorporating any sensitive data into the production code and possibly have it checked-in into the repository. These include passwords, secret keys, API keys and so forth. Those all reside in the development profile, while the production profile will be built with the real credentials. Again, once developers form a habit of working with profiles, it becomes second nature, and they don’t even think about doing it. Just agree on a private place for your personal configuration files, switch the dev profile on, and roll with it.
When we, as developers, started using unified workflows and tools, we became highly portable were able to hop from one team to another easily, and work seamlessly across diverse projects, always keeping it fresh and interesting. From developing a UI component, via working on a feature at the backend, well into writing automated tasks and so forth. This means we can work intuitively and concentrate on the important things, increasing productivity and speeding up delivery of projects exponentially. And there was much rejoicing.
At the end of the day, aligning workflows means less friction. Less friction means less worries. Less worries means more coding, and more coding means happy developers. Well at least that’s the case with me. I love coding.