Author: Shamsul Arefin
Software Architecture / Java / .NET / Machine Learning
The year was 2014, day unknown, month unknown. I had my worst days in production.
Very early in the morning, I received an urgent call from one of the PMs residing in a different timezone complaining about a very slow server response. The operations team and I were on the spot and started to investigate the issue. We couldn’t do much about it and the slowness persisted for almost an hour.
And it happened again the next day. And again. And again until we found the root cause after a week.
During that agonizing period of time, the following are what we went through:
- We were quick to figure out that the issue was in the database, but what’s causing the DB to respond slowly?
- We checked all the jobs scheduled in the database server.
- We stopped syncing to the warehouse DB.
- We even stopped replicating to the mirrored database during the affected time period.
- We worked day and night to optimize some batch operations and indexes that were affecting the monolithic databases badly.
- We checked for patches or any pending updates both for OS and the database.
- We even had to cancel out the Denial-of-Service attack option.
Finally, in the end, we figured out that a scheduled job running daily on a different server was copying a backup from the database.
That was my worst day in production and I was thinking that the best day would be when we do not have to manage any infrastructure at all.
That same year, in November 2014, AWS introduced Lambda, the heart of Serverless computing today.
No more drama. Let’s go straight to why I think that Serverless is my choice of computing platform.
No server management
There is no server to manage or provision. There is also no need to deal with OS provisioning and patching, load balancing, updating antivirus software like utility softwares. The Serverless technology itself solves a lot of complex engineering problems and provide elegant solutions for those. This level of excellence to the complex operational problem is nearly impossible to achieve for small teams like us. Serverless is also lowering the operational cost. However, don’t interpret “No server management” as “No-Ops”. Now that the Operations don’t have to manage the servers, they can concentrate more on tasks like monitoring, performance etc thus accomplish their responsibility more efficiently.
Near limitless scaling and high availability
Serverless provides built-in availability and fault tolerance. We don’t need to architect for these capabilities since the services running the application provide them by default. For example, AWS Lambda maintains compute capacity across multiple Availability Zones in each region to help protect our code against individual machine or data center facility failures.
Technology choice flexibility:
As a services company, we develop on a wide range of technology stack. Serverless function as a service (FaaS) enables us, for example, to deploy the Node js restful service and Python based Object Recognition service in the same manner. We can avoid the one-size-fits-all approach and pick the right tool for a specific job. Even in case of databases we have that flexibility – you can choose to use a relational AWS Aurora Serverless or NoSQL AWS DynamoDB in the different components of the same application with worrying much about maintainability and scalability.
Principles of microservice
Serverless is a natural fit for Microservice architecture. For small teams like ours with little experience with Microservices architecture, it’s always faster to start developing the system as monolithic. However, as the system grows and complexity increases, it becomes more feasible and natural to break down this monolithic architecture into smaller services. Thanks to the recent announcement of Lambda Layers from AWS which makes it easier for us to manage non-domain specific generic libraries that needs to be shared across services. Since all parts do not need the same amount of resources, by breaking down the monolithic architecture into smaller services we can more precisely allocate the computing resources to the specific services.
Event-driven asynchronous architecture
Many usages of Serverless applications are inherently event-driven. We are taking advantage of AWS Lambda’s event sources or triggers for asynchronous workloads like generating thumbnails for images, annotating images with ML predictor when a new image is uploaded, indexing documents into CloudSearch/ElasticSearch from DynamoDB Streams etc.
Here at Infolytx we develop a lot of POC or pilot projects. In the last couple of years, we have developed over 15 such projects. We have to keep them all alive even though most of them are infrequently used. However, we cannot predict the scalability needs for each application. By deploying them as a FaaS, we are not only saving a lot of unnecessary server costs but at the same time, we are achieving near “limitless” scalability for each. Additionally, we are optimizing the cost for a single request by defining the optimal memory size. Do note that providing only the minimum required memory won’t give you the minimum cost since serverless platforms like AWS Lambda is billed in 100-ms increments. We can define the optimized memory size for each function from a graph like the following from where we can find the threshold value of 512 MB for this function.
The often misunderstood and ignored parameter regarding cost estimation of serverless application development is the Total Cost of Ownership (TCO) – the Personnel cost savings. We often calculate the cost incurred by the infrastructure only, compare the results and come up with a conclusion that serverless is expensive. I’ll ask you – did you calculate the cost to develop the engineering framework and the cost associated with maintaining those? Did you calculate the cost of human expertise to design and implement the portion of such a complex system, for example, that AWS Lambda provides out of the box? Did you consider the cost of a database administrator to maintain such large scale databases?
“According to McKinsey and Company, typical servers in business and enterprise data centers deliver between five and 15 percent of their maximum computing output on average over the course of the year.” — Forbes
A lot of computing resource is wasted by over-provisioning. Even if we use auto up and down scaling of computing resources, our triggering algorithm is most of the time either inaccurate (since its really hard to measure how busy a server is in terms of CPU, Memory (easy ones), network, caches, memory buses) or very conservative. For example, adding another server if CPU utilization is 70%.
Faas platforms like AWS Lambda is built for optimizing the utilization. Since one Fass unit is tailored and custom defined by the developer for optimal computing resource in terms of cost and performance, a single unit of this function invocation is optimized for compute utilization. Load balancing is much easier now since a single unit of Sandbox (for function invocation) is either utilized or not utilized. So the Load Balancer can now easily remove unutilized Sandboxes.
Infrastructure as code
We simply maintain a Cloudformation or Serverless Framework template which is the description of the infrastructure. That includes all the dependencies of the Serverless applications, for example, API Gateway, Lambda functions, SQS, S3 Buckets, DynamoDB tables, Kinesis Streams etc. This infrastructure as a code pattern allows us to easily replicate the infrastructure across different environments.
Pay as you go
With Serverless, you only have to pay for the resources you are actually consuming. When your application is not using the resources you don’t have to pay for it thus eliminating the need for pre-provisioning or over-provisioning the storage and computing. Rather than paying for an individual server unit, you pay for consistent throughput or execution duration.
I hope that the platform like AWS Serverless Application Repository will fulfill the dream of a wide range of reusable applications or application components ready for assembling and deployment in powerful ways. You just search for your desired application, configure the environment variables, parameters etc and deploy the application in your own AWS account.
Serverless applications reduce the number of moving parts making it easier to maintain. You write less code with lesser components and complexity and thus easier to maintain.
At Infolytx, we are true fans of DevOps philosophies and we diligently try to follow the DevOps practices. In a serverless world, DevOps culture is even more important since the SysAdmin related tasks like Networking, Security and Monitoring is carried out in a different way and some of these tasks are accomplished by the developers or very closely connected to their work.
Yes, we understand that there are limitations in terms of available tools for debugging, testing and monitoring for serverless applications. But we have seen significant new progress in this area as well with the AWS Serverless Application Model (SAM), Serverless Framework, Apex Up/Ping leading the way.
There are obviously issues like the VPC cold start that we hear about frequently. But by making appropriate technology choices and caching, this effect can be minimized to an acceptable state. Moreover, AWS recently announced that in 2019 they are going to minimize the VPC cold start issue by using the ENI (Elastic Network Interface) in a multi-tenant fashion.
Some developers who have worked with serverless will complain about its limitations. Yes, there are certain application use cases for which Serverless is probably not the best solution right now. However, there are many application use cases for which, when designed and architected correctly, Serverless puts a lot of power to the developers’ hand. However, since the technologies and techniques are relatively new there is a strong need to educate both the Development and Operations team to achieve the best results.