A Manager’s Primer on Scaling Concepts
When a firm invents The Next Big Internet Thing and must decide which building blocks to use so that their software service can support the enormous traffic that inevitably follows, many highly technical questions arise. It’s quite common for these technical concepts to lie outside the manager’s area of expertise. One question that comes up immediately is whether to:
- Use one of the seemingly ubiquitous entry-level FOSS (Free and Open Source Software) options
- Invest more to obtain proprietary enterprise software
- Develop a bespoke solution that promises scalability from the very beginning.
This same question often arises when a business is upgrading their existing IT infrastructure.
FOSS options can save a lot of money but can also result in disaster if its limitations hamper scalability. Performance problems can doom a young brand. The proprietary and bespoke routes can cost a great deal before the business is built, potentially bankrupting a wobbly startup. They can also add myriad license fees or other expenses, thereby dooming profitability to what we call “death by a thousand cuts.”
With the stakes so high, it’s critical for a manager to choose a software platform wisely. In these circumstances, wise decisions are only possible with sufficient technical understanding. Extracting the technical know-how from the eggheads on one’s team can be a very time-consuming, frustrating, and expensive experience that may involve finger-pointing and recriminations if things get ugly. It helps to have some grounding in a few of the concepts involved in scalable software architecture so that you don’t have to rely on your Chief Nerd’s extemporaneous descriptions of abstruse technical concepts.
In this article, we hope to describe a few of the most salient “big ideas” that we encounter in a software project in the hope that we might help a troubled manager find some solace for their technology anxiety.
Concept 1: Customization
The bazillion-dollar question is always “How Much Will This Cost?” Sadly, there is no simple formula that will spit out the answer for every project. In Boomcycle’s experience, the costliest part of any software upgrade or buildout is typically the cost to develop customized functionality to serve a company’s unique and specific business processes. In some cases this involves installation and modification of plug-ins for some FOSS package. E.g., modifying a Joomla plugin or creating a custom feed for a Drupal site. In other cases, we might translate old VB6 code to .NET or translate some vintage, procedural PHP code to some modern framework like CodeIgniter. Perhaps the most common type of disillusionment we find in any organization is that which occurs when they realize the software package that was chosen does not do exactly what they need. If someone tells you that “Project X” does exactly what you need, do not believe them. At the barest minimum, some technical minion will either have to change its appearance for a public purpose (e.g., website) or re-code some section for adaptation to an intranet or internal use. Customization happens — all the time.
Concept 2: The Cloud
While The Cloud With A Capital C is not a particularly new concept now, it is still not well understood by most people. Your average consumer might conflate “the cloud” with some storage service that makes their video and music files available to them wherever they are. While this is a good example of cloud technology in use, it is considerably narrower than the big picture. The basic idea of the cloud is that companies like Amazon and Rackspace have begun to offer computing, storage, messaging, etc. as services. This is different than the historical way it was done: One acquired a server that was hosted somewhere, installed the necessary software to compute, store things, and send email.
So what advantage does the cloud provide? It’s a hard to summarize quickly, but the king-size advantage is that a software developer (like your friends here at Boomcycle) can summon armies of computers and vast amounts of on-line storage space, send bazillions of messages, or various other cloud services without having to call up some sales department at a hosting company which would in turn assemble an actual computer server and install it in a rack and spin up an operating system, etc. Cloud services are sold automatically at clearly publicized rates instantly. Once we don’t need the services any more, we can halt them instantly, so we are not on the hook for piles and piles of hardware when traffic is slow. The cloud provides computational services on demand — even programmatically in response to server performance metrics. The tradeoff for this flexibility in many cases is customer service. If there is no human involved in the process, it can be difficult to get help oftentimes. It’s important to have developers who know what they are doing.
Concept 3: Compute Instances
The computer-as-a-service concept touched upon in the previous section is accomplished by the idea of a compute instance. The basic idea is that cloud service providers have reduced the idea of a computer server to a software abstraction. It has an address you can use to connect to it, it behaves like the servers we know and love, but rather than being an actual motherboard, CPU, RAM, etc., it is just a software abstraction of an actual piece of computer hardware.
In reality, you typically have an actual server in a rack in a data center somewhere that runs a hypervisor which means that it can simulate the activities of numerous machines, each with their own separate and distinct operating system, memory, programs, etc. Such a virtual machine is called a “compute instance” or, more specifically, an “EC2 Instance” at Amazon or a “Cloud Server” at Rackspace. The essence of a compute instance is that it is not an actual machine itself but rather a sub-slice of an actual machine. The cloud abstraction of a computer makes a lot of interesting things possible:
- You can resize the compute instance to allocate more computing resources up to the limits of the actual physical hardware on which it runs.
- You can size it down during the “off-season” — and save some money.
- You can back it up by taking a snapshot.
- You can take the snapshot and launch an entirely new and essentially identical compute instance.
The virtualization of the computer away from specific hardware brings great flexibility.
Concept 4: Clustering
On a recent project, a Boomcycle client was attempting to deploy a WordPress site capable of serving vast amounts of traffic. The traffic targets they established were well beyond the capability of a single machine — even a very powerful one. In cases like this, it becomes necessary to distribute the large computing load to more than one machine. Using multiple machines to provide a service is often referred to as clustering. Some clustering, such as using a separate web server and database server, is trivially easy. At more extreme performance levels, it can be possible that you have multiple machines performing the same operations. It is helpful in those cases to use a load balancer.
Concept 5: Load Balancers
A load balancer can be an actual piece of hardware or it can be an abstracted software service and its purpose is to provide a single access point for a service (e.g., a website or email server, etc.) and to delegate fulfillment of that service to one or more other machines. Instead of requiring consumers to choose which web server to access on your domain, they simply enter YourDomain.com which points to the load balancer. The load balancer will subsequently delegate the request to a server that is not too busy according to various internal rules.
Concept 6: Distributed Computing
In certain applications, the traffic load is so great that a different class of problems comes into play and one must contemplate issues of distributed computing. In the case where one has many computers working together as one to host an internet service, there can be problems with data consistency across the machines. Under such circumstances, limitations of your application software can prevent additional scaling. This is what is often referred to as a “high class problem.”
In the event that you have developed an application that needs database clustering, you are obviously onto something big. If you find that you are rapidly outgrowing the capabilities of a single machine, you may want to have a professional examine your source code in search of potential roadblocks to your application’s scalability.
Re-architecting your software can be a time-consuming business and exponential traffic growth can really sneak up on you fast.
If you have any specific questions about scaling your application, give us a call! We’d be happy to speak with you.