Pre-requisites
The following are required to run this automation:
python3
boto3
numby
arparse
AWS Access keys with Read-Only Cloudwatch permission and EC2 permissions
Installation
To install the dependent packages and automation, follow these steps:
“pip install boto3 numpy argparse”
AWS CLI
Installation instructions can be found here.
Deploy Code
Copy the attached files “ec2-data-collection.sh” and “ec2-data-collection.py” to a location where you plan to execute the automation and make the files executable.
chmod a+x ec2-data-collection.sh
chmod a+x ec2-data-collection.py
Steps to be followed for executing Automation
Setup AWS Credentials
Configure aws credentials in the host environment where you run this code.
You can get the aws programmatic access from AWS Console and for more details, please refer to the link.
shell> aws configure
AWS Access Key ID : [****************]
AWS Secret Access Key : [****************]
Default region name : us-east-1
Default output format : None
Prepare Source Database List
Create a text file named “instances_list.txt” with the list of EC2 Instance identifier name(s) separated by a new line.
You can get the EC2 instance id names from the AWS console or CLI.
The instance IDs should be formatted as follows: “i-123456789”
You can specify n number of EC2 instances in the text file but ensure that they run in the same region. If the instances are in different regions, you need to gather the information separately.
Example Usage
Run the automation with the number of days to be analyzed along with Cloudwatch metric sampling interval. This will take a few minutes depending upon the number of db instances to be analyzed.
NOTE: It is recommended to run this script for 7 Days and a Period of 600
NOTE: This script will need to be run for each region and each account where there are EC2 instances that need metrics gathered.
python3 ec2-data-collection.py --file instances_list.txt --output ec2_metrics.json --region us-east-1 --days 7 --period 600
ec2-data-collection.sh -f instances_list.txt -o ec2_metrics.json -r ap-south-1 -d 7 -p 600