Receive Metrics Jobs
#340410
Description
The Data Ingestion API framework allows you to customize and extend data transfer capabilities (such as transferring metrics, configuration, attributes, or commands) for Densify SaaS deployments in a secure job execution framework. The framework encapsulates a custom data management tool into a Job entity in Densify, where you can create, list, schedule, execute, and delete. When you schedule the Densify job for execution, the custom data management tool is invoked. The framework also allows you to upload and download job artifacts, such as data or log files.
By default, Densify is installed with the Metrics data management tool which allows SaaS users to invoke a standard Receive Metrics Jobs resource from the Densify REST API. This resource allows you to upload a metrics file into a Densify job and schedule the job for processing, which transfers the metrics data to their respective services for analysis. You can also use the Receive Metrics Jobs resource to download any result files or logs of the job execution.
Resource
/receive/metrics/jobs/
Supported Operations
Table: Receive Metrics Jobs Supported Operations
Operation |
HTTP Method |
Request Input Parameters |
Response Elements |
Description |
POST /receive/metrics/jobs |
Request Body Parameters:
|
Creates a job in Densify with the support structure for uploading utilization metrics. If the [name] parameter is not supplied, then the job's name will be set to the job GUID. |
||
GET /receive/metrics/jobs |
None |
Collection of jobs: |
Returns a list of all jobs in Densify. |
|
GET /receive/metrics/jobs/<jobId>?[lines_to_tail_in_logs=n] |
Path Parameter: Query String Parameter: |
Returns the details of the job specified by job ID provided in the request. |
||
GET /receive/metrics/jobs/<jobId>/input |
Path Parameter: |
Returns a list of input files for the job. |
||
GET /receive/metrics/jobs/<jobId>/logs?[lines_to_tail_in_logs=n] |
Path Parameter: Query String Parameter: |
Returns a list of log files for the job. |
||
GET /receive/metrics/jobs/<jobId>/audit_info |
Path Parameter: |
Returns audit information for the job specified. |
||
POST /receive/metrics/jobs/<jobId>?[execute=true]&[time=HH:MM] |
Path Parameter: Query String Parameters: Request Body Parameter: |
Uploads a file into an existing job. |
||
GET /receive/metrics/jobs/<jobId>/download/files?[file] |
Path Parameter: Query String Parameters:
|
See Download Files. |
This request returns all files designated for download of the specified job. You also have the option to download a specific file, if the file name is known. |
|
GET /receive/metrics/jobs/<jobId>/logs/files?[file] |
Path Parameter: Query String Parameters:
|
See Download Files. |
This request returns all log files of the specified job. You also have the option to download a specific log file if the filename is known. |
|
DELETE /receive/metrics/jobs/<jobId>/contents/input |
Path Parameter: |
Delete all input files associated with the specified job. |
||
DELETE /receive/metrics/jobs/<jobId>/contents/logs |
Path Parameter: |
Delete all log files associated with the specified job. Use this request to clean up the job log files on the Densify server. |
||
DELETE /receive/metrics/jobs/<jobId> |
Path Parameter: |
Deletes all content associated with the job (i.e. input, download, and log files) and removes the job from the scheduled job list. See Example: Delete Job. |
||
PUT /receive/metrics/jobs/<jobId>/parameters |
Path Parameter: Request Body Parameters: |
Updates the parameter attributes of an existing job. |
The following is a complete list of possible parameters for the /receive/metrics/jobs endpoint. Path, query string, and/or request body parameters are required depending on the method requested.
Table: Receive Metrics Jobs - Path Parameters
Parameter Name |
Type |
Description |
string |
Specify the job GUID to identify the job. |
Table: Receive Metrics Jobs - Request Body Parameters
Parameter Name |
Type |
Description |
(optional) |
string |
The name of the job. During job creation, if the name parameter is not set, then name will be automatically set to the job globally unique identifier (GUID). Example of setting the job name: {"name": "my-sample-job"} If you do not want to specify any parameters for creating a job, you must still provide an empty JSON body element when using the POST operation: POST /receive/metrics/jobs { } |
(optional, depending on the metrics custom endpoint) |
Array of name-value pairs |
The job parameters element is an array of "name", "value" pairs that you can provide in the request body. The "parameters" element needs to be provided in the following JSON format: "parameters": [ {"name": <string>, "value": <string>}, ... ] The parameters required during job creation is dependent on the metrics custom endpoint used. Below is an example of setting optional parameters during job creation: { "name" : "My new job", "parameters": [ { "name": "JobPriority", "value": "urgent" }, { "name": "Licence", "value": "true" }, { "name": "Area", "value": "NewArea" } ] } In an update parameters request, you need to provide the complete list of parameters, including the updated ones and the non-updated ones. The new list of parameters used for the update command overwrites the entire old list of parameters. Below is an example of the request body for updating the "Area" parameter. Notice that the entire parameters list is provided. [ { "name": "JobPriority", "value": "urgent" }, { "name": "Licence", "value": "true" }, { "name": "Area", "value": "west-2b" } ] |
multipart/form-data |
To upload a file into an existing job, attach the file into the file form-data key in the body of the POST request. |
Table: Receive Metrics Jobs - Query String Parameters
Parameter Name |
Type |
Description |
(optional) |
integer |
Specify the number of lines from the bottom of the log files to display. For example, to display the last 100 lines of logs, you would specify the following: lines_to_tail_in_logs=100 The default value of -1 denotes that the entire log files will be displayed. |
(optional) |
string |
The job execute option specifies whether the job should be executed with the uploaded file. Possible values for the execute option:
|
(optional) |
string |
The job time option is used in conjunction with the execute option to specify when the job is to be executed next. The time value must be in HH:mm 24-hr format. |
(optional) |
string |
The file option allows you to download a specific file. You must specify the exact download filename in order for this operation to succeed. For example, to download a file named "output.txt", the following call is made: GET /receive/metrics/jobs/455fa7bb-10fb-41a7-96a9-f4b13bd7a05c/download/files?file=output.txt Here is an example to download a log file named "output.log": GET /receive/metrics/jobs/455fa7bb-10fb-41a7-96a9-f4b13bd7a05c/logs/files?file=output.log |
The following is a complete list of possible response elements that are returned for the /receive/metrics/jobs resource. If the response element does not apply to the API request, then the element is not displayed in the results.
Table: Receive Metrics Jobs Response Schema
Element |
Type |
Description |
string |
The globally unique identifier (GUID) assigned to the job. |
|
string |
The name of the job. During job creation, if the [name] parameter is not set, then it will be automatically set to the job globally unique identifier (GUID). |
|
Array of
|
The job parameters element is an array of "name", "value" pairs that is dependent on the data management tool used. The Metrics tool is the default data management tool for the Receive Metrics Jobs resource. |
|
|
If the job has uploaded metric files which have been audited, the last audit details are displayed. The audit_info element displays the following information: "audit_info": { "audit_name": <string>, "audit_date": <string>, "audit_path": <string>, "target_audit": <int>, "targets_failed": <int>, "audit_end_date": <string>, "load_date": <string>, "load_end_date": <string>, "load_status": <string>, } |
|
Array of:
|
If the job has uploaded files, details of those files are displayed: "input_files": [ { "name": <string>, // name of the metrics file "size": <int>, // size of the file, in bytes }, ... { "name": <string>, "size": <int>, } ] |
|
integer |
Specifies the number of lines from the bottom of the log file to display. The default value of -1 denotes that the entire log file will be displayed. |
|
|
Contains the details of each file in the job's log directory. For each log file available, the following information is displayed:
|
|
|
An element that provides the status of the executed Metrics job operation, which contains the following items:
The job_status element is displayed in the following form: {"code": <int>, "message": <string>, "files": [ { "name": <string>, // filename "size": <int>, /size in bytes }, ... ] } Below is a response example if there are additional non-status files in the job status folder: "job_status": { "code": 0, "message": "File Loaded Successfully.", "files": [ { "name": "production_status.zip", "size": 908756 }, { "name": "test_status.zip", "size": 453903 } ] } The default Metrics data management tool produces a statusmessage.txt file, a corresponding statuscode.txt file, and other status files in the job status directory for each job execution. The Data Ingestion API framework provides flexibility for a custom data management tool to produce their own status message, code, and files by generating files with the exact name (i.e. statusmessage.txt, statuscode.txt) in the custom tool's job status directory. Note: File names are case sensitive. Both message and code files must exist in the job status directory for the framework to override the job_status element. |
|
Download Files Response |
||
octet-stream zipped file containing files |
The response for a file or log download request is an octet-stream zipped file containing all the files produced when executing the job. For a single file download, the zipped file will contain only one file. The suggested filename (in the response Content-Disposition) is in the following format:
{jobId}_logs.zip {jobId}_download.zip Where {jobId} is the job GUID. |
|
Messages and Error Handling |
||
string |
A response message for the request from the job_status element. The job_status: message string can be one of the following:
|
|
integer |
The status code of the job request. Possible "status" values are:
|
|
HTTP error |
Request is malformed. For example, when the time format is invalid, the following error message is displayed: { "message": "Invalid schedule time format <HH:MM>. Schedule task cannot be updated or created.", "status": 400 } |
|
HTTP error |
"Limit number of active jobs has been reached." The number of active jobs on the server exceeds the Densify configuration setting of maximum job limit. |
|
HTTP error |
The job does not exist in the system. |
|
HTTP error |
|
Examples
The following example shows you how to create a job:

Request:
POST /receive/metrics/jobs/
JSON Body:
{ "name": "sample-job" }
Response:
{
"jobId": "483e2a46-1f40-4e8a-9ee2-532f66dc549b",
"name": "sample-job",
"job_status": {
"code": 0,
"message": "Job created successfully"
}
}
The following example shows you how retrieve all jobs:

Request:
GET /receive/metrics/jobs
Response:
[
{
"jobId": "e79518b7-fe0f-4bd4-ae5a-2ca51974d77b",
"name": "sample-job"
},
{
"jobId": "58419075-6009-4c73-8783-54a49e347384",
"name": "my-job"
}
]
Example: Get Specific Job Details
The following example shows you how retrieve details of a specific job:

Request:
GET /receive/metrics/jobs/58419075-6009-4c73-8783-54a49e347384?lines_to_tail_in_logs=2
Response:
{
"jobId": "58419075-6009-4c73-8783-54a49e347384",
"name": "CPU utilization - upload job",
"parameters": [{
"name": "hint",
"value": "Sample"
},
{
"name": "license",
"value": "true"
}
],
"audit_info": {
"audit_name": "58419075-6009-4c73-8783-54a49e347384",
"audit_date": "2018-02-13 09:23:37.000",
"audit_path": "output\00001518_M0500_58419075-6009-4c73-8783-54a49e347384",
"targets_audited": 323,
"targets_failed": 0,
"audit_end_date": "2018-02-13 09:45:11.333",
"load_date": "2018-02-13 09:52:23.930",
"load_end_date": "2018-02-13 09:52:26.577",
"load_status": "Loaded"
},
"input_files": [{
"name": "allmetrics_production.csv",
"size": 203945
}, {
"name": "h2metrics_test.csv",
"size": 19932
}
],
"lines_to_tail_in_logs": 2,
"logs": [
{
"name": "main.log",
"contents": "[INFO] [2017-10-10 17:37:33,682] 1)****************** end testing RMI connectivity to thedocsmachine/192.168.2.123\n[INFO] [2017-10-10 17:37:33,682] \n"
}
],
"job_status": {
"code": 0,
"message": "Success"
}
}
Example: Get Job Input File Details
The following example shows you how retrieve input file details of a specific job:

Request:
GET /receive/metrics/jobs/483e2a46-1f40-4e8a-9ee2-532f66dc549b/input
Response:
{
"jobId": "483e2a46-1f40-4e8a-9ee2-532f66dc549b",
"name": "My Upload Job"
"parameters": [{
"name": "Priority",
"value": "High"
},
{
"name": "License",
"value": "True"
}
],
"input_files": [{
"name": "allmetrics_production.csv",
"size": 203945
}, {
"name": "Eastmetrics_test.csv",
"size": 28635
}, {
"name": "Normetrics.csv",
"size": 27812
}
],
}
Example: Get Job Log File Details
The following example shows you how retrieve log file details for a specific job:

Request:
GET /receive/metrics/jobs/58419075-6009-4c73-8783-54a49e347384/logs?lines_to_tail_in_logs=2
Response:
{
"jobId": "58419075-6009-4c73-8783-54a49e347384",
"name": "CPU utilization - upload job",
"parameters": [{
"name": "hint",
"value": "Sample"
},
{
"name": "license",
"value": "true"
}
],
"logs": [
{
"name": "main.log",
"contents": "[INFO] [2018-05-10 17:37:33,682] 1)****************** end testing RMI connectivity to thedocsmachine/192.168.2.123\n[INFO] [2018-05-10 17:37:33,682] \n"
},
{
"name": "logs_load.txt",
"contents": "[INFO] [2018-05-10 09:45:10,403] Process host:notApplicable, converter type:DataImporter\n[INFO] [2018-05-10 09:45:11,355] Audit Converter complete.\n"
}
]
}
Example: Get Job Audit Information
The following example shows you how retrieve details of a specific job:

Request:
GET /receive/metrics/jobs/483e2a46-1f40-4e8a-9ee2-532f66dc549b/audit_info
Response:
{
"jobId": "483e2a46-1f40-4e8a-9ee2-532f66dc549b",
"name": "CPU utilization - manual upload",
"parameters": [{
"name": "region",
"value": "north"
},
{
"name": "license",
"value": "true"
}
],
"audit_info": {
"audit_name": "483e2a46-1f40-4e8a-9ee2-532f66dc549b",
"audit_date": "2018-05-13 09:23:37.000",
"audit_path": "output\002215185311_M0511_483e2a46-1f40-4e8a-9ee2-532f66dc549b",
"targets_audited": 323,
"targets_failed": 0,
"audit_end_date": "2018-05-13 09:45:11.333",
"load_date": "2018-05-13 09:52:23.930",
"load_end_date": "2018-05-13 09:52:26.577",
"load_status": "Loaded"
},
}
The following example shows you how to upload a file to an existing job:

Request:
POST /receive/metrics/jobs/8e5fafe4-1709-46b5-9bdb-f0a12ed4910d?execute=false&time=00:00
The request body contains a multipart/form-data file with the file key:
{
"file": <file content>
}
Response:
{
"jobId": "8e5fafe4-1709-46b5-9bdb-f0a12ed4910d",
"name": "sample-job",
"job_status": {
"code": 0,
"message": "File uploaded successfully."
}
}
The following example shows you how to download files from an existing job:

Request:
GET /receive/metrics/jobs/8e5fafe4-1709-46b5-9bdb-f0a12ed4910d/download/files
Response:
The response is an octet-stream with the Content-Disposition suggested filename in the following format:
<jobId>_download.zip
The downloaded zipped file contains all the files, designated as "download", for the specified job.
The following example shows you how to download log files from an existing job:

Request:
GET /receive/metrics/jobs/8e5fafe4-1709-46b5-9bdb-f0a12ed4910d/logs/files
Response:
The response is an octet-stream with the Content-Disposition suggested filename in the following format:
<jobId>_logs.zip
The following example shows you how to delete the input files from an existing job:

Request:
DELETE /receive/metrics/jobs/483e2a46-1f40-4e8a-9ee2-532f66dc549b/contents/input
Response:
{
"message": "Contents of directory deleted successfully.",
"status": 200
}
The following example shows you how to delete log files from an existing job:

Request:
DELETE /receive/metrics/jobs/483e2a46-1f40-4e8a-9ee2-532f66dc549b/contents/logs
Response:
{
"message": "Contents of directory deleted successfully.",
"status": 200
}
The following example shows you how to delete a job:

Request:
DELETE /receive/metrics/jobs/883e2a46-1f40-4e8a-9ee2-532f66dc549b
Response:
{
"message": "Job deleted successfully.",
"status": 200
}
Example: Update Job Parameters
The following example shows you how to update parameters for an existing job:

Request:
PUT /receive/metrics/jobs/acb37a49-cd7c-4bf1-a94b-884fdce11ed7/parameters
JSON Request Body:
[
{"name": "License",
"value": "true"
},
{"name": "Priority",
"value": "low"
}
]
Response:
{
"jobId": "acb37a49-cd7c-4bf1-a94b-884fdce11ed7",
"name": "SuperDuperJob",
"parameters": [{
"name": "License",
"value": "true"
},
{
"name": "Priority",
"value": "low"
}
],
"input_files": [],
"lines_to_tail_in_logs": -1,
"logs": [],
"job_status": {
"code": 0,
"message": ""
}
}