Optimizing GCP Compute Engine Instances

Optimizing GCP Compute Engine Instances

#340540

This use case includes GCP cloud infrastructure data collection, analysis, and generation of right-sizing recommendations for Compute Engine instances using a combination of Densify Console and Densify API operations. The recommendations, which can be downloaded on demand via Densify API, provide extensive details that can be used to feed downstream automation engines in either JSON or terraform-map formats.

A comprehensive PDF report can be downloaded for each instance to supply supporting evidence for a recommendation; this can be reviewed by a system owner before approvals are given for the suggested changes.

The diagram below provides an overview of the three major processes in Densify:

  1. Data Collection—Collect GCP data into Densify.

    Use the Densify API or the Densify Console to initiate this process.

  2. Analysis—Analyze collected data based on pre-defined policies to make recommendations.

    The analysis process is initiated automatically after data collection completes. Results can be immediately pulled from the Densify API or pushed to a webhook URI once the analysis is complete.

  3. Reporting Database Update—Update reporting database tables with analysis results for reporting.

    The reporting database update process is scheduled to run nightly after the analysis refresh is completed. The Densify Console GCP optimization opportunity report and the Impact Analysis and Recommendation reports are available after the reporting database update is complete.

Note:  Before you can collect GCP cloud infrastructure data in Densify, you need to create a GCP service account with services enabled and permissions configured. See Google Cloud Platform Data Collection Prerequisites for details.

The steps below outline a typical workflow to optimize GCP Compute Engine instances:

  1. From the Densify API: Authenticate using JWT—Retrieve a token to authenticate each API call.

  2. Collect GCP data through the Densify API or the Densify Console. See Configuring a Google Cloud Platform Connection for details.

    After data collection completes, an analysis is initiated for each project associated with the GCP account configured for data collection.

  3. API: List All GCP Analyses—Query for the analysis entity of each project for which data was collected (from step 1). You can use the returned analysis entity to check for status and recommendations.

  4. API: Poll for Analysis Status—Check for data collection and analysis status. The recommendations are available when analysis is complete.

  5. API: Download Results to Apply Recommendations—Retrieve recommendations for actioning or forward to orchestration engine.

    Alternatively, you can Add a Webhook to an Existing GCP Analysis to push recommendations to an external URI once the next analysis process completes.

  6. API/Console (optional): Download Impact Analysis and Recommendation Instance Report for system owners. This can achieved from the Densify API or from the Densify Console. SeeViewing the Impact Analysis and Recommendation Report for details on the content of the report.
  7. Console (optional alternative): Review GCP Virtual Machine optimization opportunity report from the Densify Console. See GCP for details. The optimization opportunity report is available after the nightly reporting database update.
  8. Console (optional): Review GCP optimization policy settings used during analysis. Contact [email protected] for details.

Postman Collection

Densify provides a Postman collection of sample API requests for working with your public cloud environments. See the Postman Collection.

Authenticate using JWT

Use the /authorize resource to retrieve an authenticated token to make subsequent API calls. See Authorize for details. By default, the token expires in 5 minutes, therefore you will need to ensure that you have an active token for each Densify API request.

Request:

Copy 
POST /authorize
{
    "userName": "apiUser",
    "pwd": "apiPassword"
}

Response:

Copy 
{
  "apiToken": "eyJhbGciOiJIUzUxMiJ9.eyJqdGkiOiIzNzI2Yzk0NC0wMmE4LTRlYzQtOGE2Ny04ODBmMDM2OTRhZDciLCJpYXQiOjE1NDI2NTI0MDUsInN1YiI6InZhbiIsImlzcyI6IkRlbnNpZnkuY29tIiwiZXhwIjoxNTQyNjUyNzA1fQ.cJd8qFJfRoPnMEU7GzcdYGBT8WwlgmviQ1OQp8P_w9VUcjQA3FJaB9QkqJJ6d7zbrY5yjc4w0rOWjY-PPdbmqw",
  "expires" : 1542652705869,
  "status" : 200
}

Use the apiToken value in your Authorization request header for subsequent Densify API calls.

Initiating GCP Data Collection and Analysis

The following example shows you how to initiate GCP data collection and analysis, and send the results to a WebHook. See Analysis GCP Analyze.htm for details.

Request:

POST /analysis/gcp/analyze

{

  "serviceAcctJSON": "gcpAcctEast-608378",

  "policyInstanceId": "4a63f651-a583-4157-97ff-35651370ffbe",

  "webHook": {

       "uri": "http://mywebhookserver/webhook/results",

       "authType": "basic",

       "authValue": "tester:testerpassword"

  }

}

Response:

{

  "href": "Not available"

  "message": "Analysis in progress",

  "status": 200

}

List All GCP Analyses

To see a list of all GCP analyses currently in Densify, use the /analysis/cloud/gcp resource. See List all analyses for a particular platform and vendor for details of the resource operation.

Request:

Copy 
GET /analysis/cloud/gcp

Response:

Copy 
{
   "analysisId": "cf25d8c1-4fdf-42b1-b3a3-1a8d8a425d13",
   "analysisName": "gcpAcctEast-608378",
   "analysisCompletedOn": 0,
   "href": "/analysis/cloud/gcp/cf25d8c1-4fdf-42b1-b3a3-1a8d8a425d13",
   "analysisResults": "/analysis/cloud/gcp/cf25d8c1-4fdf-42b1-b3a3-1a8d8a425d13/results",
   "analysisStatus": "/analysis/cloud/gcp/cf25d8c1-4fdf-42b1-b3a3-1a8d8a425d13/status"
}

Poll for Analysis Status

Use the "analysisStatus" resource element from the List All GCP Analyses response to poll for status of the data collection and analysis. See Check for analysis status for details of the resource operation. The analysis is complete and recommendations are available when "analysisStage": "Completed".

Request:

Copy 
GET /analysis/cloud/gcp/cf25d8c1-4fdf-42b1-b3a3-1a8d8a425d13/status

Response:

Copy 
{
  "analysisStage": "Completed",
  "webHookStatus": "",
  "statusMessage": "Analysis gcpAcctEast-608378 was last completed on Wed Feb 27 12:06:10 EST 2019."
}

Add a Webhook to an Existing GCP Analysis

You can add a webhook definition to an existing GCP analysis. The analysis will push the optimized results to the specified webhook URI. See Add webhook to an analysis for details on adding a webhook.

Request:

Copy 
POST /webhook/analysis/cloud/gcp/cf25d8c1-4fdf-42b1-b3a3-1a8d8a425d13
{
  "uri": "https://myInstanceConfigServer:443/api/densify/results",
  "authType": "bearer",
  "authValue": "eyJhbGciOiMsIUzUxMiJ9.eyJqdGkiOiJhYWU2MjIxOS1iOWQyLTQ3OGMtYWI3Mi00NGU2OTUzY2RjMDEiLCJpYXQiOjE1NDM2MDgxMTEsInN1YiI6ImFkbWluIiwiaXNzIjoiRGVuc2lmeS5jb20iLCJleHAiOjE1NDM2MDg0MTF9.h3bJrAP-Z2LeqzjN3FYpFDyoaADvYT1MdLw5SuguqkGE7sjB4c7YgQgv3saj15r2IsgTWH8PW7eNnoZwFP9eiQ"
}

Response:

Copy 
{
  "message": "ok",
  "status": 200
}

Download Results to Apply Recommendations

Use the /analysis/cloud/gcp/<analysisId>/results resource to retrieve recommendation results and forward to orchestration engine. See Analysis: GCP Recommendations for details.

Request:

Copy 
GET /analysis/cloud/gcp/cf25d8c1-4fdf-42b1-b3a3-1a8d8a425d13/results

Headers:

Copy 
Accept: application/terraform-map
Authorization: Bearer <apiToken>

Download Impact Analysis and Recommendation Instance Report

Use the rptHref resource element provided in the instance recommendation output to download a PDF Impact Analysis and Recommendation Report for each instance. This report is available after a reporting database update, which is scheduled to run nightly. See rptHref and Example: Downloading an Impact Analysis and Recommendation Report for details.

Request:

Copy 
GET /systems/b374d9d5-a529-4e19-9a14-077123f322b4/analysis-report

Headers:

Copy 
Accept: application/octet-stream
Authorization: Bearer <apiToken>