Rolling Your Own Cloud
As engineers we get a thrill out of building new and exciting things. However, the search for exciting and new uncharted territories, at least uncharted as far as we know, can lead us down a path of rediscovery as we find out that we are not the first ones to have embarked on that quest.
This phenomena usually displays itself whenever a new project is created that requires a certain functionality that we may not have had exposure to yet, our enthusiasm gets the best of us and we ignore that our first priority is to get the project done and thus we embark on a mission to reinvent the wheel. I have noticed this is just as prevalent today with Cloud Computing as it has been with other frameworks or libraries in the past. However, today I see Cloud Computing as a way to accelerate the implementation of a business venture without having to spend time and money on creating infrastructure, be it through adding code or hardware.
So what are some of the things that we continue to reinvent? As you can imagine that list is long but just to outline a few of the things that could be encompassed in it, let us take a look instead at some of the Azure offerings that we do not have to build from scratch and which maximize our investment, reduce shipping timelines and provide a sound quality foundation for our product/business:
- Visual Studio Team Services
- SQL Database
- Storage
- Document DB
- API Management
- Key Vault
- Azure Active Directory Domain Services
And that is just to name a few, there are many more but of course our solutions may only use a few or more offerings depending on what we are building and how it integrates into our architecture. Nevertheless, the heavy hit of building a lot of this infrastructure, fun to build as it may be, falls fully on Azure while leaving us free to concentrate on adding business value. Essentially we should envision architecting solutions by putting together the Cloud offering pieces we need in order to create our applications:
The world of Cloud Computing is quickly evolving, adding many new services at astonishing speed while providing a highly compelling reason for us not to roll out our own infrastructure. But the greatest thing about all of this is that by leveraging these Cloud offerings, we can take advantage of built in features that offer a competitive edge for the products and services we create. Take the API Management ‘Lego’ for instance, say that we are building an API that is going to be publicly available to many entities and we need to make sure our API is not too heavily used, so as to cause performance problems for all of our customers and we need a way to throttle the usage of the API. Now, we can spend the time to design and write the throttling functionality ourselves but that becomes yet another thing to maintain and it adds no immediate value to our offering because this is essentially something everyone offering a public API must do. Therefore, this is a case when using an existing Cloud feature such as the API Management, that already has this functionality built-in and which has been tested and designed with performance, security and scalability in mind makes sense from both, a technical and most importantly a business perspective.
The same can be said for many other Cloud service offerings, such as the ability to offer single sign on to our corporate and consumer clients, all done through the power of Azure AD. Now, all of us would like to think that we write secure code by default, but the truth is that writing secure code is difficult if the intent to do so was not part of the initial design. Thus, I see many times that a new solution is engineered to handle authentication/authorization along with new hand rolled security libraries that are most likely not as secure as they should be. This is usually true if the library authors are not security specialists, which if you ask any veteran security professional they would tell you that security libraries are best left to be written by experts. This is an important point to consider, because Microsoft Azure’s offerings are usually tied with supporting libraries such as ADAL, which are written by security specialists and are ready for us to consume.
In any case, I am not trying to deter you from creating new and exciting things, even if that means that you are ‘reinventing the wheel’ on purpose. I think you should write your own blog framework, or object relational mapper on your own time if you really want to. However, what I do not recommend you should do is start using your prototypes in production or much worse, if you are a lead in a project, to distribute your new Frankenstein version of an ORM or Key Rotation Library or whatever else you reinvented, to the rest of your team and mandate that this should be the new framework of choice for everyone to use. This unfortunately happens quite often, and the ones that suffer are not just the developers forced to use these home made, unsupported and unknown libraries, but the businesses that utilize these libraries in their products.
You will find that when you need a specialized library or a framework in your solution, a bit of googling will get you to the destination you seek and you won’t have to reinvent that particular wheel. I learned this to be true first hand a number of years ago when I first encountered a project that required parsing of CSV files. So instead of rolling my own I searched the web and found the excellent http://www.filehelpers.net/. This reduced my implementation time for the solution and at the end I had a happy customer. All without me having to write my own CSV parser, which I admit it would have been fun to do but it would not have added any value to my client’s solution when I could spend that time actually working on code that added specific business value.
And that is the bottom line why you should not reinvent your own Cloud(Wheel), because you owe it to your users, customers and stakeholders to maximize their investment while providing top quality services and products. The best way to do so is by leveraging tried and true solutions and plugging them into your own solution architecture. Unless there truly is nothing out there that does or supports the thing that you must do for your product to be successful, I suggest you first look at what is available and see how you can leverage it into your product. If after doing that you still find the existing frameworks/libraries wanting, then create or extend your own but do it as a group effort with the support from the business stakeholders. After all, they are the ones paying the bills, and they will certainly appreciate you adding more business value instead of debugging headaches and production nightmares.
That is all for now, until next time, happy Cloud Computing! 