CPG Scripted REST APIs (available since New York release)

[Badarinarayan B]

Note: This is an article about Scripted REST APIs. In the spirit of explanation CURL samples and API Response samples have been provided. These should not be used as-is or treated as absolute references. Such samples have been provided purely for explanatory assistance and have been redacted where necessary. Please refer to appropriate payloads while executing on your ServiceNow instance.

Scripted REST APIs are offered by ServiceNow as an easy to use interface fronting operations on the NOW platform applications. Cloud Management is ServiceNow for Hybrid Cloud Management solutions and has its own share of Scripted REST APIs under the name: "CMP Catalog API".

In New York release, we have made several enhancements to the existing Scripted REST APIs and introduced three new APIs. Let's go over these briefly. It is worth noting that all APIs have been changed to use ACL checks in New York. Basically, the logged in user is allowed to perform an operation if she has "admin" or "sn_cmp.cmp_root_admin" role. In other cases, permissions have to be given explicitly.

List Cloud Services

The first API is "List Cloud Services". As the name suggests, this api lists the "Cloud Services" which are basically catalog items.

Whereas in pre-NY releases, this API listed only Blueprints (catalog_type = Provision), in NY it lists both Blueprints and Day 2 Operation related catalog items (catalog_type = lifecycle). Further, we have added three filter parameters to this API to make the API more useful and configurable. Let's take a look at them.

Filter parameters

The first filter parameter is named "catalog_type". Permissible values are ("provision", "lifecycle"). The user of the API has option to list a particular type of catalog items: Blueprint or Day 2 operation related, with this filter parameter.

Second is the filter parameter named "display_variables". Permissible values are (true, false or _blank_). If set to true, the API will also list catalog item variables associated with each catalog item. If not set or set to false, the variables won't be shown.

Third filter parameter is "blueprint_name". The user can enter part (or whole) of the blueprint name in this which will be used by the API as a pattern and only those blueprints (Provision and Lifecycle) will be listed whose name "STARTSWITH" the value in the parameter blueprint_name. For ex, if you want to list all blueprints which start with, say, "Servicenow*", then you can set the variable as: blueprint_name = Servicenow. 

Response structure

The response structure of this API has also been changed to provide more information. In New York, this API will show below information in addition to what exists now:

1. Blueprint Name (Blueprint which backs the Catalog Item)
2. Blueprint Category (Compute, Storage etc.)
3. Catalog Type (Provision or Lifecycle)
4. Created On
5. Updated On 

CURL request sample

curl "http://myinstance.service-now.com/api/now/cmp_catalog_api/services?catalog_type=provision&display_variables=true&blueprint_name=aws" \
--request GET \
--header "Accept:application/json" \
--user 'admin':'admin'

Response sample

[
  {
    "catalog_name": "AWS VM With Volume",
    "blueprint": "db8047e284f833008ba2097d5bf31f88",
    "blueprint_name": "AWS VM With Volume",
    "blueprint_category": "57c0dbd9934b22004a9032bfa67ffb79",
    "active": true,
    "category": "8641fe260b212200eff7ef9bb4673a3d",
    "blueprint_category_name": "Compute",
    "shortDescription": "AWS VM With Volume description...",
    "id": "436103a684f833008ba2097d5bf31f38",
    "last_updated_on": "2019-04-17 06:04:18",
    "last_updated_by": "admin",
    "catalog_type": "1",
    "lifecycleOperations": [
      {
        "id": "1d7187a684f833008ba2097d5bf31f88",
        "name": "ModifySchedule"
      },
      {
        "id": "3971c7a684f833008ba2097d5bf31ff2",
        "name": "Stop"
      },
      {
        "id": "417187a684f833008ba2097d5bf31f3c",
        "name": "Deprovision"
      },
      {
        "id": "8e710ba684f833008ba2097d5bf31f94",
        "name": "Start"
      },
      {
        "id": "bc7147a684f833008ba2097d5bf31fe1",
        "name": "Cleanup"
      },
      {
        "id": "fd71c7a684f833008ba2097d5bf31f55",
        "name": "ModifyLease"
      }
    ],
    "created_on": "2019-04-17 06:04:17",
    "created_by": "admin",
    "operation": "74900fe284f833008ba2097d5bf31f68",
    "blueprint_type": "Custom Blueprint",
    "operation_name": "Provision"
  },
  
],
"created_on": "2019-04-17 06:26:00",
"created_by": "admin",
"operation": "d8a687ea84f833008ba2097d5bf31f30",
"blueprint_type": "Cloud Template",
"template_type": "612f41e49f00320048111f80a57fcf82",
"template_name": "ARM Template",
"template_cloud_product": "9d921e66930b22004a9032bfa67ffb66",
"template_cloud_product_name": "Azure ARM",
"operation_name": "Provision"
}
]

List resource catalog items by RB and List resource catalog items by CI class.

Let's look at the next two APIs which are related to listing of "resource catalog items". CMP distinguishes between two classes of operations that the cloud admin can do on the cloud provider. They are called as "Day 1" and "Day 2" operations. Day 1 operations are basically creation, deployment and provisioning scenarios whereas Day 2 operations span over a wide range of activities that include but are not limited to backups, scaling, maintenance, extensibility, governance etc. Resource catalog items refer to Day 2 operations' related Catalog items. By default, CMP ships some Day 2 operations like Start, Stop for Virtual Machines etc. However, users can create many more.

These APIs concern listing of resource catalog items based either on Resource Block or CI Class. The APIs will read the CI Class or Resource Block and return the list of resource catalog items that "CONTAIN" the particular CI class or Resource Block. The response of these two APIs will be same as that of "List cloud services". As in the case of previous API, these two APIs also have their own filter parameters. First, let's look at the common ones:

Common filter parameters

"display_variables" - As explained in the case of first API above.

"resource_interface" - Each Day 2 operation or resource catalog item is hosted on a resource interface. For ex, Virtual Machine related operations are hosted on "Virtual Server Interface" resource interface. Thus, you can use this filter to list all Day 2 operations on a particular interface.

"operation_name" - The user can specify the part (or whole) of the name of the operation she is interested in listing. The API will use this as a pattern and resource catalog items will be listed if their name "STARTSWITH" the value in operation_name.

Filter parameters for "List catalog items by RB" only

For the API "List resource catalog items by RB", there is an additional mandatory parameter named "resource_block". The user can provide name of the resource block in CMP based on which listing of resource catalog items needs to be done. For ex, "Virtual Server" etc.

Please note that this is a mandatory parameter and the API won't work without this. If this parameters is left blank, the API will throw an error (HTTP 400: Bad Request).

CURL request sample

curl "http://myinstance.service-now.com/api/now/cmp_catalog_api/catalogsbyrb?display_variables=true&operation_name=start&resource_block=virtual%20server" \
--request GET \
--header "Accept:application/json" \
--user 'admin':'admin'

Response sample

[
  {
    "display_name": "Virtual Server - Start",
    "id": "4fa40d7d931813004a9032bfa67ffb1b",
    "variables": [
      {
        "id": "244d86f90b522300e937c97193673a7e",
        "name": "resourceBlock",
        "question_text": "Resource Block",
        "default_value": "Not specified"
      },
      {
        "id": "537d0af90b522300e937c97193673abf",
        "name": "resourceInterface",
        "question_text": "Resource Interface",
        "default_value": "Not specified"
      },
      {
        "id": "596dc6f90b522300e937c97193673ad8",
        "name": "resourceId",
        "question_text": "Resource Id",
        "default_value": "Not specified"
      },
      {
        "id": "699d4af90b522300e937c97193673aa6",
        "name": "operationName",
        "question_text": "Operation Name",
        "default_value": "Not specified"
      },
      {
        "id": "b2ad8af90b522300e937c97193673a86",
        "name": "credential",
        "question_text": "Credential",
        "default_value": "Not specified"
      }
    ],
    "created_on": "2018-02-16 00:10:06",
    "updated_on": "2018-02-16 00:10:06",
    "name": "Virtual Server - Start",
    "short_description": "Are you sure you want to perform this operation?",
    "active": true
  }
]

Filter parameters for "List catalog items by CI Class" only

For the API "List resource catalog items by CI Class", there is an additional mandatory parameter named "ci_class". The user can provide name of the CI Class in CMDB based on which listing of resource catalog items needs to be done. For ex, "cmdb_ci_vm_instance". Please note that the API will only return valid values if the CI class has a corresponding Resource Block in CMP.

Please note that this is a mandatory parameter and the API won't work without this. If this parameters is left blank, the API will throw an error (HTTP 400: Bad Request).

CURL request sample

curl "http://myinstance.service-now.com/api/now/cmp_catalog_api/catalogsbyciclass?display_variables=true&ci_class=cmdb_ci_vm_instance&operation_name=start" \
--request GET \
--header "Accept:application/json" \
--user 'admin':'admin'

Response sample

[
  {
    "display_name": "Virtual Server - Start",
    "id": "4fa40d7d931813004a9032bfa67ffb1b",
    "variables": [
      {
        "id": "244d86f90b522300e937c97193673a7e",
        "name": "resourceBlock",
        "question_text": "Resource Block",
        "default_value": "Not specified"
      },
      {
        "id": "537d0af90b522300e937c97193673abf",
        "name": "resourceInterface",
        "question_text": "Resource Interface",
        "default_value": "Not specified"
      },
      {
        "id": "596dc6f90b522300e937c97193673ad8",
        "name": "resourceId",
        "question_text": "Resource Id",
        "default_value": "Not specified"
      },
      {
        "id": "699d4af90b522300e937c97193673aa6",
        "name": "operationName",
        "question_text": "Operation Name",
        "default_value": "Not specified"
      },
      {
        "id": "b2ad8af90b522300e937c97193673a86",
        "name": "credential",
        "question_text": "Credential",
        "default_value": "Not specified"
      }
    ],
    "created_on": "2018-02-16 00:10:06",
    "updated_on": "2018-02-16 00:10:06",
    "name": "Virtual Server - Start",
    "short_description": "Are you sure you want to perform this operation?",
    "active": true
  }
]

Get request item status

The "Get request item status" API has been enhanced to return more information that earlier. Pre-NY, this API used to only return the RITM (request) number for the submitted request. In NY, it returns below additional information:

1. Catalog item used to submit the request.
2. Stage the request is in.
3. Assigned To
4. Assignment Group
5. Opened At
6. Opened By
7. State the request is in. Different from Stage.
8. Submission data in JSON notation.

This information will help users identify and trace the request exactly to its origin and monitor its flow.

 CURL request sample

curl "http://myinstance.service-now.com/api/now/cmp_catalog_api/status?req_item=RITM0010006" \
--request GET \
--header "Accept:application/json" \
--user 'admin':'admin'

Response sample

{
  "catItem": "5707072e84f833008ba2097d5bf31f07",
  "stage": "Deployment Cancelled",
  "sys_id": "d5ae8be2843c33008ba2097d5bf31fe5",
  "assignedTo": null,
  "assignmentGroup": null,
  "number": "RITM0010006",
  "openedAt": "2019-04-17 07:02:03",
  "openedBy": "6816f79cc0a8016401c5a33be04be441",
  "state": "Closed Incomplete",
  "submissionData": "{\"CloudAccount\":\"AWSPlusAzure\",\"AWS_CFT_1_SSHLocation\":\"0.0.0.0/0\",\"LeaseEndDate\":\"\",\"ScheduleProfile\":\"-- No Schedule --\",\"UserGroup\":\"cfcbad03d711110050f5edcb9e61038f\",\"ScheduleTimeZone\":\"America/Los_Angeles\",\"BusinessService\":\"\",\"CostCenter\":\"\",\"Application\":\"\",\"AWS_CFT_1_InstanceType\":\"t2.small\",\"AWS_CFT_1_KeyName\":\"admin-default\",\"StackName\":\"bburli-cft1\",\"Location\":\"AWS Datacenter - us-west-1\"}"
}

List catalog items for Provision Type

The case of listing stacks is one of the important and very frequently used scenario for the users of Cloud Management. Keeping this in mind, a special API has been added to list only stacks. This API is very similar to "List cloud services" with the only distinction of listing only stacks (Provision type of catalog items).

The user can use "display_variables" and "blueprint_name" filter parameters in this API and their behavior is already explained in "List cloud services" API.

CURL request sample

curl "http://myinstance.service-now.com/api/now/cmp_catalog_api/stacks?blueprint_name=aws" \
--request GET \
--header "Accept:application/json" \
--user 'admin':'admin'

Response sample

[
  {
    "catalog_type": "1",
    "catalog_type_name": "Provision Operation (Day 1)",
    "catalog_name": "Azure Arm 1",
    "blueprint": "3696c3ea84f833008ba2097d5bf31ff1",
    "blueprint_name": "AWS Arm 1",
    "blueprint_category": "57c0dbd9934b22004a9032bfa67ffb79",
    "blueprint_category_name": "Compute",
    "active": true,
    "category": "8641fe260b212200eff7ef9bb4673a3d",
    "shortDescription": null,
    "id": "e456cfaa84f833008ba2097d5bf31f29",
    "last_updated_on": "2019-04-17 06:28:46",
    "last_updated_by": "admin",
    "blueprint_type": "Cloud Template",
    "template_type": "612f41e49f00320048111f80a57fcf82",
    "template_name": "ARM Template",
    "template_cloud_product": "9d921e66930b22004a9032bfa67ffb66",
    "template_cloud_product_name": "Azure ARM",
    "lifecycleOperations": [
      {
        "id": "00b6cbea84f833008ba2097d5bf31fb3",
        "name": "ModifySchedule"
      },
      {
        "id": "0fa68bea84f833008ba2097d5bf31f6d",
        "name": "Cleanup"
      },
      {
        "id": "3aa68bea84f833008ba2097d5bf31f06",
        "name": "Deprovision"
      },
      {
        "id": "6fa6cbea84f833008ba2097d5bf31f1e",
        "name": "ModifyLease"
      },
      {
        "id": "bfa6cbea84f833008ba2097d5bf31f65",
        "name": "Start"
      },
      {
        "id": "d7a68bea84f833008ba2097d5bf31fc5",
        "name": "Stop"
      }
    ],
    "created_on": "2019-04-17 06:26:00",
    "created_by": "admin",
    "operation": "d8a687ea84f833008ba2097d5bf31f30",
    "operation_name": "Provision"
  }
],
"created_on": "2019-04-17 06:04:17",
"created_by": "admin",
"operation": "74900fe284f833008ba2097d5bf31f68",
"operation_name": "Provision"
}
]

Submit resource operation request

This API is introduced in NY. This API is designed for users to submit or execute Day 2 operations via Scripted REST APIs. This API requires a mandatory parameter named "cat_id". The parameter cat_id denotes the sys_id of the catalog item that you want to invoke. This must be a resource operation catalog item for the API to work correctly.  

This is a HTTP POST request and this it expects a body of the form below:

{"resourceId":"12345678KJHUSDFHLKKNOD", "StackName": "AWS-Stack", "Location": "us-west-1"}

Where the field resourceId refers to the sys_id of the resource on which the operation will be invoked.

CURL request sample

curl "http://myinstance.service-now.com/api/now/cmp_catalog_api/submitoprequest?cat_id=436103a684f833008ba2097d5bf31f38" \
--request POST \
--header "Accept:application/json" \
--header "Content-Type:application/json" \
--data "{\"CloudAccount\":\"AWSPlusAzure\",\"LeaseEndDate\":\"\",\"ScheduleProfile\":\"-- No Schedule --\",\"UserGroup\":\"cfcbad03d711110050f5edcb9e61038f\",\"ScheduleTimeZone\":\"America/Los_Angeles\",\"Storage_Volume_SizeTotalGB\":\"1\",\"Virtual_Server_NodeName\":\"bburli-aws-node4\",\"Storage_Volume_VolumeName\":\"disk7\",\"Application\":\"\",\"StackName\":\"bburli-aws2\",\"Location\":\"AWS Datacenter - us-west-1\"}" \
--user 'admin':'admin'

Response sample

{
  "number": "RITM0010007",
  "catItem": "436103a684f833008ba2097d5bf31f38",
  "stage": "request_approved",
  "sys_id": "7bad1b2e843c33008ba2097d5bf31f0b",
  "assignedTo": null,
  "assignmentGroup": null,
  "openedAt": "2019-04-17 08:07:46",
  "openedBy": "6816f79cc0a8016401c5a33be04be441",
  "state": "Open",
  "submissionData": <Same as that of request>
}

Submit cloud services request

The "Submit cloud services request" API is designed to invoke a cloud services request to provision stacks. In NY, it has been enhanced to accept provisioned requests for both stacks and resource operation catalog item or Day 2 operations. The payload for Day 2 operation is as described in the above API. The payload for provisioning of stack is different for different cloud providers and cloud resources being provisioned. Thus, the best way to build this payload is to request the stack once manually and then copy the submission data from the sn_cmp_order table's order_data field.

CURL request sample

curl "http://myinstance.service-now.com/api/now/cmp_catalog_api/submitrequest?cat_id=436103a684f833008ba2097d5bf31f38" \
--request POST \
--header "Accept:application/json" \
--header "Content-Type:application/json" \
--data "{\"resourceId\":\"58ebcf22843c33008ba2097d5bf31fa3\",\"LeaseEndDate\":\"\", \"operationName\":\"ModifySchedule\",\"UserGroup\":\"cfcbad03d711110050f5edcb9e61038f\",\"ScheduleTimeZone\":\"America/Los_Angeles\",\"CostCenter\":\"\",\"ScheduleTimeZone:America/Los_Angeles\",\"ScheduleProfile\":\"Always On\",\"Storage_Volume_SizeTotalGB\":\"1\",\"BusinessService\":\"\",\"Virtual_Server_NodeName\":\"bburli-aws-node2\",\"Storage_Volume_VolumeName\":\"disk3\",\"Application\":\"\",\"StackName\":\"bburli-aws2\",\"Location\":\"AWS Datacenter - us-west-1\"}" \
--user 'admin':'admin'

Response sample

{
  "number": "RITM0010011",
  "catItem": "436103a684f833008ba2097d5bf31f38",
  "stage": "request_approved",
  "sys_id": "23636fee843c33008ba2097d5bf31f1a",
  "assignedTo": null,
  "assignmentGroup": null,
  "openedAt": "2019-04-17 08:32:52",
  "openedBy": "6816f79cc0a8016401c5a33be04be441",
  "state": "Open",
  "submissionData": <Same as that of request>
}

Conclusion

For the users of Cloud Management, the above scripted APIs are very handy tools to build custom flows on top of existing CMP capabilities. It is important to note that, in case of "Submit cloud services request" and "Submit resource operation request", the usual policies, business rules etc will be triggered accordingly since we're not bypassing any systemic checks of the NOW platform. 

These scripted APIs can also help in performing bulk operations. At this point in time, CMP does not perform bulk operations on cloud resources. For ex, "Starting 100 Virtual Machines on AWS" or "Deleting 20 Volumes on Azure". Such operations can be performed using Scripted REST APIs by looping through the required resources and calling the APIs via script on each one.

We're always open for feedback and suggestions. Please drop in your comments and share your opinions on this article.

Badarinarayan Burli

Staff Software Engineer - Cloud Management.

[Ashok Madhavan1]

Hi Badari,

Thanks for the article on the updated APIs for end user cloud management. Can you add some curl samples for a few operations (one for stack, one for day-2) and with some input parameters.

regards

Ashok Madhavan

[Badarinarayan B]

Hi Ashok,

Thank you for your comments. I have included CURL and response samples with a note at the top of the article regarding their nature of use as well.

Regards,

Badari.

[jain2]

Hi Badarinarayan Burli,

thank you. could you please provide an example of Brownfield resource too? since Brownfield resource doesn't has Stack name. CURL for Brownfield resource (day 2 operations). 

[Badarinarayan B]

Hi jain,

You raise a good point. While "StackName" and "Location" are checked for completeness, "resourceId" is really the only needed value here. Thus, you can pass any valid string value to "StackName" and "Location" when performing on brownfield resources and the API should work as it would for a greenfield resource.

Regards,

Badari.

[jain2]

Hi Burli,

i've tried but i am getting below error:

FYI Cloud user Portal with same user i can able to perform resource operation

here is the sample request

{
"sysparm_quantity": "1",
"variables": {
"CloudAccount": "Cloud",
"Location": "AWS Datacenter - eu-west-1",
"StackName": "Stack26VM",
"userId":"903bc2f40fefbb0027c27f5ce1050e9f",
"userName":"Admin",
"resourceId":"2c049a340f23fb0027c27f5ce1050e97"

}
}

Postman response:

{
"number": "RITM0010345",
"catItem": "3a114939931813004a9032bfa67ffb9d",
"stage": "check_permissions",
"sys_id": "bc54e6f00f6bbb005c1f1b1e51050e1c",
"assignedTo": null,
"assignmentGroup": null,
"openedAt": "2019-08-26 10:20:27",
"openedBy": "903bc2f40fefbb0027c27f5ce1050e9f",
"state": "Open",
"submissionData": "{\"sysparm_quantity\":\"1\",\"variables\":{\"CloudAccount\":\"Cloud\",\"Location\":\"AWS Datacenter - eu-west-1\",\"StackName\":\"Stack26VM\",\"userId\":\"903bc2f40fefbb0027c27f5ce1050e9f\",\"userName\":\"Admin\",\"resourceId\":\"2c049a340f23fb0027c27f5ce1050e97\"}}"
}

it is stuck at check_permissions stage. if you see the response "resourceid" is same. i think you miss to mention the permission needed for resource operation for the rest user?

please help me so i can validate the steps. 

i am awaiting your response. thank you in advance. 

[Badarinarayan B]

Are you passing below in request:

{
"sysparm_quantity": "1",
"variables": {
"CloudAccount": "Cloud",
"Location": "AWS Datacenter - eu-west-1",
"StackName": "Stack26VM",
"userId":"903bc2f40fefbb0027c27f5ce1050e9f",
"userName":"Admin",
"resourceId":"2c049a340f23fb0027c27f5ce1050e97"
}

I think you should only pass:

{

"CloudAccount": "Cloud",
"Location": "AWS Datacenter - eu-west-1",
"StackName": "Stack26VM",
"userId":"903bc2f40fefbb0027c27f5ce1050e9f",
"userName":"Admin",
"resourceId":"2c049a340f23fb0027c27f5ce1050e97"
}

If you have executed one manually, then check the sn_cmp_order table and see what payload exists in the order table. You can reuse that. Also, the error I see is "Invalid resourceId", which is why I am suspecting payload.

[jain2]

Hi Burli,

do i have to check Order form or API steps of the parameter? 

i am sure there must be something is missing, when i check cloud resource request workflow. it is always skipping and activity.result pass to else action

code snip from workflow

var user = gs.getUser();
activity.result = (user.hasRole("sn_cmp.cmp_root_admin")
|| user.getID().equals(resourceGr.getValue("assigned_to"))
|| user.isMemberOf(resourceGr.getValue("assignment_group")))? 'success' : 'failure';
if (activity.result == 'failure') {
var msg = 'User ' + gs.getUserName() + ' does not have permission for operation ' + current.variables.operationName + ' on ' + resourceGr.name;
createCloudTrail('Check User Permissions', 4, true, msg, msg);
}
} else {
createCloudTrail('Invoke Pre-Process Workflow', '4', false, 'Exception: Resource Id is invalid/null', '');

}

i am getting the above error. it is failing on permission. 

Thanks

[Badarinarayan B]

We saw this error mostly when the request payload was incorrect. Can you share the entire request payload you're using as-is for us to help further?

Also, you have to take the request payload from a successful order that you would have requested manually. Can you please follow below steps:

1. Request the desired day 2 operation manually. Ensure it succeeds.

2. Go to instance and open sn_cmp_order table and check the latest order. 

3. For the successful order, get the order_form_data value. This is proper request payload.

4. Use the request payload from Scripted REST API.

Let me know if you face issues even then. Also, can you share your contact point to debug one-on-one, if needed.

[jain2]

Hi Burli,

i've interesting stuff to show you.

my email id: abhishek.jain@atos.net.

let me know if we can connect and show you what i have experienced. 

Thanks,

Abhishek Jain