Optimizing AWS EC2 Instances

Optimizing AWS EC2 Instances


This use case includes API-controlled data collection, analysis, and generation of right-sizing recommendations for EC2 instances within an AWS account. The recommendations, which can be downloaded on demand or posted to a webhook-specified receiver, provide extensive details that can be used to feed downstream automation engines in either JSON or terraform-map formats.

After the reporting tables have been updated with the analysis results, you can view the recommendation reports from the Densify Console. In addition, a comprehensive PDF report can be downloaded for each instance to supply supporting evidence for a recommendation; this can be reviewed by a system owner before approvals are given for the suggested changes.

Note: The AWS EC2 optimization opportunities dashboard in the Densify Console contains recommendations for all AWS accounts for which you have collected EC2 instance data (see Optimizing Your Public Cloud: EC2). Each /analysis/cloud/aws/<analysisId>/results API request returns recommendations for one AWS account, since each analysis entity is defined by the AWS account connection. Therefore if you have more than one AWS account, an API request against a single account represents a subset of the full set of EC2 recommendations displayed in the Densify Console.

To learn more watch the video:

The diagram below provides an overview of the three major processes in Densify:

  1. Data Collection—Collect AWS data into Densify.

    Use the Densify API or the Densify Console to initiate this process.

  2. Analysis—Analyze collected data based on pre-defined policies to make recommendations.

    The analysis process is initiated automatically after data collection completes. Results can be immediately pulled from the Densify API or pushed to a webhook URI once the analysis is complete.

  3. Reporting Database Update—Update reporting database tables with analysis results for reporting.

    The reporting database update process is scheduled to run nightly after the analysis process. The Densify Console optimization opportunity reports and the Impact Analysis and Recommendation reports are available after the reporting database update is complete.

Note: Before you can collect data via AWS CloudWatch API, you need to create and configure an IAM role for Densify to have a trust relationship with your AWS account. See AWS Data Collection Prerequisites for an IAM Role for details.

The steps below outline a typical API workflow to optimize AWS EC2 instances:

  1. Authenticate using JWT—Retrieve token to authenticate each subsequent API call.
  2. Optional: Review Policy—Review a list of available policies to be used for analysis.

  3. Collect Data and Analyze—Start collecting data and initiate analysis (specify optional policy, optional webhook URI).

  4. Check for Analysis Status—Check for data collection and analysis status or wait for a webhook trigger to indicate that the analysis finished and downstream processing has initiated.

  5. Download EC2 Recommendations for Action—Retrieve recommendations for actioning or forward to orchestration engine.

  6. Optional: Download Impact Analysis and Recommendation Instance Report for system owners.

Postman Collection

Examples of Densify API request for this use case can be found in the following Postman collection:

Download the Densify Public Cloud Postman Collection (v 12.1.7)

Follow the steps below to use the downloaded Postman collection:

  1. Unzip the downloaded file and import it into your Postman application. The Densify API Collection (12.1.7) collection and Densify environment are loaded into your Postman workspace.
  2. Modify the variables in the Densify environment to match your Densify and cloud-specific settings and credentials.
  3. Note: If you already have a previous Densify environment in your Postman application, you can either delete the previous version or rename it. Otherwise, you will have duplicate Densify environments after the new collection is imported.

  4. Review the Documentation section of the collection for an overview of the workflow and API requests.

Use this sample collection to familiarize yourself with Densify API requests.

Authenticate using JWT

Use the /authorize resource to retrieve an authenticated token to make subsequent API calls. See Authorize for details. By default, the token expires in 5 minutes, therefore you will need to ensure that you have an active token for each Densify API request.

Review Policy

Use the /analysis/cloud/aws/policy resource to view a list of policies available in Densify. You can use one of these policies for optimization analysis. This is an optional step as the analysis runs with the default policy if no policy is specified. See GET /analysis/cloud/aws/policy for details.

Collect Data and Analyze

Initiate data collection and analysis for a given AWS account using the /analysis/cloud/aws/analyze resource. See Analysis: AWS Analyze for details.

You can specify an override policy to use for your analysis and/or specify a webhook to publish results when analysis is complete.

Check for Analysis Status

To find your AWS analysis entity, request for a list of existing AWS analyses available. Refer to List all analyses for a particular platform and vendor for details.

Use the "analysisStatus" resource element from the analysis entity to check for progress status. See Analysis: Status for details on this resource.

The analysis is complete and recommendations are available when "analysisStage": "Completed".

Download EC2 Recommendations for Action

Use the /analysis/cloud/aws/<analysisId>/results resource to retrieve EC2 recommendation results and forward them to a downstream orchestration engine for actioning. See Analysis: AWS Recommendations for details.

Download Impact Analysis and Recommendation Instance Report

Use the rptHref resource element provided in the JSON recommendation output to download a PDF Impact Analysis and Recommendation Report for each instance. This report is available after a reporting database update, which is scheduled to run nightly. See rptHref and Example: Downloading an Impact Analysis and Recommendation Report for details.