{"id":2517,"date":"2021-04-04T18:20:01","date_gmt":"2021-04-04T11:20:01","guid":{"rendered":"http:\/\/vticloud.io\/?p=2517"},"modified":"2021-09-20T10:01:15","modified_gmt":"2021-09-20T03:01:15","slug":"amazon-cloudwatch-custom-metrics","status":"publish","type":"post","link":"https:\/\/vticloud.io\/en\/amazon-cloudwatch-custom-metrics\/","title":{"rendered":"Amazon CloudWatch Custom Metrics"},"content":{"rendered":"

<\/p>\n

On AWS, with basically any EC2 instance using Amazon CloudWatch, you can access default metrics and monitor the CPU, Network, Disk usage (I\/O, not the available hard disk space), and status of the individual virtual machine. What you can’t monitor with default metrics, however, is RAM Utilization (or Storage of virtual disks). This is instead executed using custom metrics<\/strong>, and in this article, we\u2019re going to find out how to install a custom metric on our EC2 instance.<\/span><\/p>\n

Take a look back at Amazon CloudWatch<\/span><\/h2>\n

As mentioned in the article: Amazon CloudWatch Introduction (Part 2) | VTI CLOUD<\/a><\/span><\/p>\n

Standard monitoring is every 5 minutes<\/strong> and Detailed monitoring is every 1 minute.<\/strong><\/span><\/p>\n

The key idea here is to create a custom metric that transfers data to Amazon CloudWatch every minute.<\/span><\/p>\n

How to create Custom Metrics of Amazon CloudWatch<\/span><\/h2>\n

Firstly, you would need to log into the AWS console, select the region, provide an EC2 instance and send some custom metrics up to Amazon CloudWatch.<\/span><\/p>\n

In order to do that, we need permission to talk to Amazon CloudWatch so we need to:<\/span><\/p>\n

A. Create a Role for EC2 Instance<\/strong><\/span><\/h3>\n
    \n
  1. \n

    Go over to Identity Access Management (IAM<\/strong>).<\/span><\/p>\n<\/li>\n

  2. \n

    Go to Roles<\/strong> and create a new one.<\/span><\/p>\n

    \n
    \n
    \n
    \n
    \n
    \n
    \n
    \n
    \n
    \n
    \n
    \"\"<\/span><\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/li>\n
  3. \n

    The service that\u2019s going to use this role is EC2<\/strong>, go ahead and hit Next<\/strong>.<\/span><\/p>\n

    \n
    \"\"<\/span><\/p>\n<\/div>\n<\/div>\n<\/li>\n
  4. \n

    Choose \u2018Create Policy<\/strong>\u2019<\/span><\/p>\n

    \n
    \"\"<\/span><\/p>\n<\/div>\n<\/div>\n<\/li>\n
  5. \n

    On Create Policy page, choose JSON tab<\/strong> and complete the following fields and move to the next step.<\/span><\/p>\n

    \n
    {<\/span><\/span><\/code><\/span><\/pre>\n
       \"Version\"<\/span>:\u201c2012<\/span>-<\/span>10<\/span>-<\/span>17<\/span>\u201d<\/span>,<\/span><\/code><\/span><\/pre>\n
       \"Statement\"<\/span>:[<\/span><\/span><\/code><\/span><\/pre>\n
          {<\/span><\/span><\/code><\/span><\/pre>\n
             \"Action\"<\/span>:[<\/span><\/span><\/code><\/span><\/pre>\n
                \"\u201ccloudwatch\"<\/span>:<\/span>\"PutMetricData\u201d\"<\/span>,<\/span><\/code><\/span><\/pre>\n
                \"\u201cec2\"<\/span>:<\/span>\"DescribeTags\u201d\"<\/span><\/span><\/code><\/span><\/pre>\n
             ],<\/span><\/code><\/span><\/pre>\n
             \"Effect\"<\/span>:\"Allow\"<\/span>,<\/span><\/code><\/span><\/pre>\n
             \"Resource\"<\/span>:[<\/span><\/span><\/code><\/span><\/pre>\n
                \"*\"<\/span><\/span><\/code><\/span><\/pre>\n
             ]<\/span><\/code><\/span><\/pre>\n
          }<\/span><\/code><\/span><\/pre>\n
       ]<\/span><\/code><\/span><\/pre>\n
    }<\/span><\/span><\/span><\/code><\/span><\/pre>\n<\/blockquote>\n
    \n
    \"\"<\/span><\/p>\n<\/div>\n<\/div>\n<\/li>\n
  6. \n
    Add tags if you\u2019d like to and hit next to the review the whole Policy.<\/span><\/p>\n
    \n
    \"\"<\/span><\/p>\n<\/div>\n<\/div>\n<\/li>\n
  7. \n
    The final step to create a Policy is choose a Policy name<\/strong> and Create Policy<\/strong>. You can also take a look at the policy\u2019s permissions on the Summary table.<\/span><\/p>\n
    \n
    \"\"<\/span><\/p>\n<\/div>\n<\/div>\n<\/li>\n
  8. \n
    After the notification of successfully created a Policy, go back<\/strong> to the Create Role page and hit refresh<\/strong> button on the top right, search<\/strong> for the name of the Policy you have created, tick on the box<\/strong> and hit next<\/strong> to attach that Policy to the Role.<\/span><\/p>\n
    \n
    \"\"<\/span><\/p>\n<\/div>\n<\/div>\n<\/li>\n
  9. \n
    Add tags if you\u2019d like to and hit next to the review the Role.<\/span><\/p>\n
    \n
    \"\"<\/span><\/p>\n<\/div>\n<\/div>\n<\/li>\n
  10. \n
    Choose a Role name, describe the Role function and hit Create role.<\/span><\/p>\n
    \n
    \"\"<\/span><\/p>\n<\/div>\n<\/div>\n<\/li>\n<\/ol>\n
    We\u2019ve created a Role and you can find it here at IAM,Roles, it\u2019s gonna be pretty basic JSON.<\/span><\/p>\n
    \n
    \"\"<\/span><\/div>\n<\/div>\n
     <\/p>\n
    B. Attach the Role you have created to an EC2 Instance<\/strong><\/span><\/h3>\n
     <\/p>\n
    Next, we will take additional steps to attach that Role to your EC2 Instance, and push the metrics to Amazon CloudWatch.<\/span><\/p>\n
      \n
    1. \n
      Go over to the EC2 console and provision<\/strong> a EC2 Instance<\/span><\/p>\n<\/li>\n
    2. \n
      Ch\u1ecdn Amazon Linux 2 AMI<\/span><\/p>\n
      \n
      \"\"<\/span><\/p>\n<\/div>\n<\/div>\n<\/li>\n
    3. \n
      Choose Instance Type, hit next<\/span><\/p>\n
      \n
      \"\"<\/span><\/p>\n<\/div>\n<\/div>\n<\/li>\n
    4. \n
      In Configure Instance Details, IAM Role, choose the Role you\u2019ve just created<\/strong><\/span><\/p>\n
      \n
      \"\"<\/span><\/div>\n<\/div>\n
       <\/p>\n
      In Advanced Details, User data, we\u2019re going to run a bootstrap<\/strong> script provided bellow.<\/span><\/p>\n
      \n
      \"\"<\/span><\/div>\n<\/div>\n
      \n
      \n#!\/bin\/bash<\/code><\/span><\/p>\n
      yum update -y<\/code><\/span><\/p>\n
      sudo yum install -y perl-Switch perl-DateTime perl-Sys-Syslog perl-LWP-Protocol-https perl-Digest-SHA.x86_64<\/code><\/span><\/p>\n
      cd \/home\/ec2-user\/<\/code><\/span><\/p>\n
      curl https:\/\/aws-cloudwatch.s3.amazonaws.com\/downloads\/CloudWatchMonitoringScripts-1.2.2.zip<\/a> -O<\/code><\/span><\/p>\n
      unzip CloudWatchMonitoringScripts-1.2.2.zip<\/code><\/span><\/p>\n
      rm -rf CloudWatchMonitoringScripts-1.2.2.zip<\/code><\/span><\/p>\n<\/blockquote>\n
      Let\u2019s look at what does this script do:<\/span><\/p>\n
      Firstly, it is updating the EC2 Instance with all the security patches<\/span><\/p>\n
      Secondly, it is installing a bunch of different Perl services and then changing the directory over to \/home\/ec2-user\/<\/span><\/p>\n
      Finally, it is downloading the Amazon CloudWatch monitoring scripts, and unzipping it and deleting the zip file.<\/span><\/p>\n<\/li>\n
    5. \n
      Config the Storage, Tags and Security Group or just leave it as default, Hit \u2018Review and Launch\u2019, download or use your existing Key pair and launch the Instance<\/span><\/p>\n
      \n
      \"\"<\/span><\/p>\n<\/div>\n<\/div>\n<\/li>\n
    6. \n
      Ssh to the Instance.<\/span><\/p>\n<\/li>\n<\/ol>\n
      In the terminal, type in \u2018ls\u2019 and we can see the directory is there. That is what the bootstrap script has unzipped and installed.<\/span><\/p>\n
      Go inside that directory, we can notice two files in here: mon-get-instance-stats.pl and mon-put-instance-data.pl<\/span><\/p>\n
      \n
      \"\"<\/span><\/div>\n<\/div>\n
      To send some data to Amazon CloudWatch, we\u2019re going to use the command:<\/span><\/p>\n
      \n
      sudo \/home\/ec2-user\/aws-scripts-mon\/mon-put-instance-data.pl --mem-util --mem-used --mem-avail --disk-space-util --disk-space-avail --disk-path=\/<\/code><\/span><\/p>\n<\/blockquote>\n
      \n
      \"\"<\/span><\/div>\n<\/div>\n
      For this command, we\u2019re gonna send our memory utilization, used, and available data; also our disk space utilization and available.<\/span><\/p>\n
      And you are able to see that it says successfully reported metrics to Amazon CloudWatch, and the Reference Id.<\/span><\/p>\n
      It can take up to 1 minute for this to appear on the Amazon CloudWatch console.<\/span><\/p>\n
      For Custom Metrics, we call data having a one-minute granularity is Standard Resolution.<\/span><\/p>\n
      Now we can see a new metric under Custom Namespaces called Linux System, and five new metrics inside for your insance: MemoryUtilization, MemoryAvailable, MemoryUsed, DiskSpaceUtilization, and DiskSpaceAvailable.<\/span><\/p>\n
      \n
      \"\"<\/span><\/div>\n<\/div>\n
       <\/p>\n
      You will see only a single data point at the beginning because we\u2019ve only pushed this information into Amazon CloudWatch once.<\/span><\/p>\n
      What we are going to do is push it in every 1 minute. To do this, you need to edit your crontab file:<\/span><\/p>\n
      \n
      sudo nano \/etc\/crontab<\/code><\/span><\/p>\n<\/blockquote>\n
      and put this command into your crontab<\/span><\/p>\n
      \n
      *\/1 * * * * root \/home\/ec2-user\/aws-scripts-mon\/mon-put-instance-data.pl --mem-util --mem-used --mem-avail --disk-space-util --disk-space-avail --disk-path=\/<\/code><\/span><\/p>\n<\/blockquote>\n
      \n
      \"\"<\/span><\/div>\n<\/div>\n
      By pasting this in here, we\u2019re going to run this command every minute as root and put the information up into Amazon CloudWatch.<\/span><\/p>\n
      Basically you can set it to run a command every minute, every day, week, or month.<\/span><\/p>\n
       <\/p>\n
      After a while, go back to the Amazon CloudWatch console, choose a metric and you will see we\u2019ve been getting datapoints ever since starting. Be aware that if you don\u2019t have Detailed Monitoring turned on, you will only getting five-minute intervals.<\/span><\/p>\n
      \n
      \"\"\u00a0<\/span><\/div>\n
      \"\"<\/span><\/div>\n<\/div>\n
      \n
      \u00a0<\/div>\n<\/div>\n
      The cool thing about Amazon CloudWatch is you can also install these scripts on your on-premise instances and have them sending data to Amazon CloudWatch, and use Amazon CloudWatch as a holistic monitoring tool for both your AWS and on-premise environment.<\/span><\/p>\n
      Besides, we can use High Resolution for application that need low granularity (< 60 seconds). Follow VTI Cloud for more topic about Amazon CloudWatch.\u00a0<\/span><\/p>\n
       <\/p>\n
      About VTI Cloud<\/span><\/h2>\n
      VTI Cloud<\/strong>\u00a0is an\u00a0Advanced Consulting Partner<\/a>\u00a0<\/span>of AWS Vietnam with a team of over 50+ AWS certified solution engineers. With the desire to support customers in the journey of digital transformation and migration to the AWS cloud, VTI Cloud is proud to be a pioneer in consulting solutions, developing software, and deploying AWS infrastructure to customers\u00a0in Vietnam<\/strong>\u00a0and Japan.<\/strong><\/span><\/p>\n
      Building safe, high-performance, flexible, and cost-effective architectures for customers is\u00a0VTI Cloud\u2019s<\/strong>\u00a0leading mission in enterprise technology mission.<\/span><\/p>\n
      \n
      \u00a0<\/div>\n<\/div>\n
      <\/p>","protected":false},"excerpt":{"rendered":"
      On AWS, with basically any EC2 instance using Amazon CloudWatch, you can access default metrics and monitor the CPU, Network, Disk usage (I\/O, not the available hard disk space), and status of the individual virtual machine. What you can’t monitor with default metrics, however, is RAM Utilization (or Storage of virtual disks). This is instead […]<\/p>\n","protected":false},"author":7,"featured_media":2554,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[13],"tags":[111,46,126,45,109],"_links":{"self":[{"href":"https:\/\/vticloud.io\/en\/wp-json\/wp\/v2\/posts\/2517"}],"collection":[{"href":"https:\/\/vticloud.io\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/vticloud.io\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/vticloud.io\/en\/wp-json\/wp\/v2\/users\/7"}],"replies":[{"embeddable":true,"href":"https:\/\/vticloud.io\/en\/wp-json\/wp\/v2\/comments?post=2517"}],"version-history":[{"count":1,"href":"https:\/\/vticloud.io\/en\/wp-json\/wp\/v2\/posts\/2517\/revisions"}],"predecessor-version":[{"id":3164,"href":"https:\/\/vticloud.io\/en\/wp-json\/wp\/v2\/posts\/2517\/revisions\/3164"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/vticloud.io\/en\/wp-json\/wp\/v2\/media\/2554"}],"wp:attachment":[{"href":"https:\/\/vticloud.io\/en\/wp-json\/wp\/v2\/media?parent=2517"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/vticloud.io\/en\/wp-json\/wp\/v2\/categories?post=2517"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/vticloud.io\/en\/wp-json\/wp\/v2\/tags?post=2517"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}