Retrieving key EC2 metrics for AWS monitoring #DevOps #Automation.
As companies increasingly shift workloads and major operations to the public cloud, cloud computing has moved from a “sound knowledge” to a core competency in the enterprise. This trending transaction requires a new set of skills to design, deploy, and manage applications in cloud computing, which unfortunately are not a part of Engineering(B.Tech/B.E) curriculum .
As the market leader and most mature provider in the cloud computing space, such as AWS(Amazon Web Services),has spurred organizations to embrace Infrastructure-as-a-Service (IaaS) to build, deploy, automate, and scale their systems. Over the years, AWS has expanded beyond basic compute services. In 2021, AWS continues to lead in public cloud service providers, and it currently offers 10+ certifications that cover both foundational and specialty cloud computing topics.
Although this ever-lasting expansion of AWS ecosystem allows developers and operations teams to rapidly deploy and scale their infrastructure in the cloud, it has also made it more challenging to track the real-time health and performance of those services.
What is Amazon EC2?
Amazon Elastic Compute Cloud (EC2) allows you to efficiently provision and scale your infrastructure on demand. EC2 instances, or virtual servers, are available in a range of instance types that offer various levels of CPU, memory, storage, and network capacity. EC2 instances integrate seamlessly with other AWS services, such as Auto Scaling and Elastic Load Balancing. As containerization continues to gain steam, EC2 has evolved to serve the needs of orchestrated applications.
This article will explore some key EC2 metrics that will help you monitor widely, and later store these metrics for further analysis.
Problem statement : Create a functional code to collect operational metrics across all AWS EC2 server pool and store them in a central location for additional analysis.
Prerequisite:
- AWS account (free tier recommended)
- Create one or two EC2 instances in your account.
So let’s get started
- Create one or two EC2 instances in your account, be sure to only stand up t2.micro instance(s).
2. Create a Lambda function to:
a. List all running EC2 instances.
i. Hint: Using AWS SDK for Python (boto3) to create an ec2 client and list instances
b. For each of the EC2 instance from Step 2a, we should retrieve instance metrics such as CPU utilization, disk read operations, disk write operations, volume of incoming and outgoing network traffic.
i. Hint: fetching the metrics by investigating CloudWatch.
c. The retrieved metrics should be written in a CSV file on an S3 bucket.
3. The Lambda function should get triggered every hour using scheduled CloudWatch event. So, every hour a new file will be written to the S3 bucket.
As you deploy and scale EC2 instances to support your applications, you’ll need to continuously monitor them to ensure that your infrastructure is functioning properly. The following metrics provide a foundation for understanding the performance and availability of your instances.
Click here to check out other CloudWatch metrics for EC2 instances
- Key points for implementation :
I. IAM role for Lambda execution
II. Lambda function
III. CloudWatch Event Rule
IV. S3 bucket
Now for the most important and final part of this project,
Lambda function code:
CLICK HERE
Invoke your lambda function, and yeah! that's it.
If your lambda function is executed without any errors, you can check your s3 bucket where your EC2 metrics data file is stored.
File contents of a typical CSV file generated by your lambda function for two or more EC2 instances that is written to S3 bucket should look like this:
In this article, we’ve explored the automation of AWS infrastructure, and the key EC2 metrics to monitor if you are running several instance(s) to keep your applications running at peak performance. Adopting an automated, scalable AWS monitoring strategy will enable you to keep tabs on your infrastructure, even as hosts and services dynamically scale and analyze in real time.
Thank you!
LinkedIn : Mohith G C
