Embedding ServiceNow IT Ops into Slack

vNickNOW · ‎09-04-2018

Slack has become the market leader in what Gartner calls the workstream collaboration space, and companies of all sizes are using Slack as their primary method of collaborating. ServiceNow's IT Operations Management (ITOM) portfolio is the most comprehensive closed-loop remediation solution on the market. Combine the two and you get an efficient, easy-to-use environment in which to solve complex problems. As the ServiceNow engineering teams continue to create great new features for triaging and handling alerts (like the Alert insights in London), customers are also looking to bring some of that work to the other collaboration tools they use on a daily basis.

This article will show a repeatable structure for creating Slack integrations (though certainly not the only one). These integrations may take many forms, slash commands, action buttons, etc. We will focus on slash commands in this article as they are very simple to setup on the Slack side, and only require a bit of Scripted REST API coding on the ServiceNow side.

In the Kingston release, the flow designer / integration hub provided a baseline (a.k.a. out of the box) spoke for Slack integration when initiated from the ServiceNow side. This will come into play for our ITOM use case.

NOTE: A step 0 could be inserted below, which would be the planning phase whereby you want to determine what your integration will accomplish. For me, I wanted to create a programming style environment (operators love CLI) where alerts could be diagnosed and remediated all within Slack. Many of my colleagues have been working on some more UI friendly options with interactive message buttons, so if that's the desired user interaction (admittedly more intuitive), don't worry, I'm sure content around that will be made available soon. To accomplish my goals, I followed the steps below.

Step 1: Create Slack Application (one time)

Creating an application in Slack is an extremely easy process. As such, I won't spend too much time on this topic except to say that you need to enable incoming webhooks, so that once you have added the application to your workspace / channel, you are able to copy the necessary webhook that will be used in Step 2 below. Later in Step 6, we will be coming back to the application to define our slash commands, but let's wait on that for now.

Step 2: Create Flow in Flow Designer (one time)

To get our critical alerts to appear in the necessary Slack channel, we need a simple Integration Hub flow. We only need to define 2 things to make this happen... sounds too easy doesn't it. The only pre-requisite is that the Integration Hub plug-in must be enabled or you will not see the Slack actions (this does require a transaction-based orchestration license).

First, we have to define the trigger of the flow. This is determined by what CRUD operation against a certain table and filter you want to take action against. Here is a shot of the trigger I used that just looks for the creation of a critical severity em_alert (Alert) record.

Flow Trigger

Second, we will only have 1 step in the flow once it's triggered, and that step will be to post some details about the alert into the Slack channel (presumably monitored by NOC operators)

Slack Message Post

The step above requires us to copy and paste in the webhook from the Slack application you created in Step 1. Likewise, you should specify the Slack channel you want the messages to appear within. When choosing the Slack action to execute from the flow designer, you will notice there are options for posting details about incidents, changes, and problems, but we are going to use the catch-all generic action to just post any message we want. You can specify in the "Message" value and details and record specifics you want.

Step 3: Create Scripted REST API (multiple times)

Now comes the fun stuff. We created our Slack app and are now getting critical alerts posted into a channel called "alerts-critical", so let's see what we can do with those alerts from within Slack. Because we are going to use slash commands from Slack (step 6 will have more details), we need to create a Scripted REST API to handle those commands. I created 2 Scripted REST API's and you may need many more depending on what all you want to do from within Slack. We will focus on the structure of the API that will facilitate retrieving data about alerts (the other will initiate remediation via an orchestration workflow, but the structure of everything is the same). Here is some code for use to discuss:

(function process(/*RESTAPIRequest*/ request, /*RESTAPIResponse*/ response) {
	
	// Always send back a successful 200 response and handle error notifications in message
	response.setStatus(200);
	
	// Validate the Slack app token
	var authtoken = gs.getProperty("itom.slack.app.token01");
	if(request.queryParams.token != authtoken) {
		return "Unauthorized Request";
	}
	
	try {
		var respText = "Your " + request.queryParams.command + " " + 
				request.queryParams.text + " request has been received... processing";
		// Format an initial response
		var payload = {
			"response_type": "ephemeral",
			"text": respText
		};
		
		gs.log('---> About to request data', 'Slack ITOM Data');
		
		// Send ephemeral response back to Slack to notify user command is processing
		
		var slackResponse = new sn_ws.RESTMessageV2();
		slackResponse.setEndpoint(request.queryParams.response_url);
		slackResponse.setHttpMethod('POST');
		slackResponse.setRequestBody(JSON.stringify(payload));
		slackResponse.setRequestHeader('Content-type','application/json');
		var slackOutput = slackResponse.execute();
		
		// Get the requests query parameters and create an event to process the request async
		
		var parms = JSON.stringify(request.queryParams);
		gs.eventQueue("itom.slack.getdata", current, parms);
		
		return;
	}
	catch (e){
		gs.log('---> Slack Integration error: ' + e, 'Slack ITOM Data');
		response.setBody('Error in processing: ' + e);
		return;
	}
	
})(request, response);

REST API Code Sample

The first thing to note API's for Slash commands is that they will come as a POST, so your API resource needs to be a POST resource that does not require authentication (as there's nowhere to put in ServiceNow credentials on the Slack side). Likewise, Slack does NOT send data in the request body. Instead, they use the queryParams to hold the data, so be mindful when trying to access request values. One very important value is the "token" sent from Slack. When you created your app in step 1, there was a verification token created. This will be sent with each request from a Slack command, so you can validate it's a legitimate request for you API that does not require authentication. I chose to put mine into a custom system property, "itom.slack.app.token01", that I verify first.

Once the request is verified as coming from a trusted source, you need to send back an immediate response or Slack will timeout if more than 3000ms elapse without receiving an acknowledgement. To that end, I decided to always send a 200 response and leverage the platform's event queue to process all requests. This allows an immediate response to be sent back to Slack, and using another value sent by the Slack request, "response_url", I can asynchronously send back responses and the processes finish (long running query, workflow executions, etc). You'll notice that I specify a "response_type" of "ephemeral" in the payload back to Slack. This will result in the message only being seen by the person using the slash command and will not be retained permanently in the channel (which is good for just an acknowledgement that isn't relevant for all channel members or for long term retention).

As you see in the code, the gs.eventQueue invocation places an event onto a new queue I created and passed in the entire queryParams object as a parameter. In the following steps, we'll discuss the setup of those components that will now execute our request. However, the basic construct of the API call is what can be reused for any Slack request and you simply have to change the event queue you use as each should have their own logic.

Step 4: Create New Event Registry (multiple times)

Creating a new event queue is a quick task as well, and you can see from the screenshot below that there are minimal inputs required. From the Scripted REST API code above you can see where I referenced "itom.slack.getdata" when calling gs.eventQueue.

Create a New Event Queue

Each Script REST API you create should ideally have their own event queue as you will see in the next step for creating a scripted action, which references the event and does the bulk of the heavy lifting in terms of logic execution.

Step 5: Create Script Action (multiple times)

Now that we have our REST API putting an event onto our new queue, we can build out the logic that will processing the event and send the necessary response to the "response_url" we passed in (events can take a parm1 and parm2 parameter). The logic will be built in a script action that is associated with the event queue we just created.

Create a New Script Action

I'm not going to paste all the code for this action into the blog, but have attached the code to this blog. A few things I do want to call out are around validating parameters and message longevity of the responses. In regards to validating parameters, this will be dependent upon how you want your slash command to operate. For me, I want a fairly rigid format like "/datanow alert:Alert0001234". This format allows for potential future additions where a customer may add logic to request data for other record types from the ITOM spectrum (e.g. "/datanow ci:serverxyz" would return all recent changes, incidents, etc about a CI).

With regards to longevity of responses, use "ephemeral" or "in_channel" as necessary. I like "ephemeral" for messages like a "help" response where it would only apply to the requestor and does not need to remain permanently. The use of "in_channel" will make the message available for all to view and remain. Similarly, I chose to put most of the message into an "attachments" attribute, and only have the data about who and what was requested in the main "text" of the message. Putting data into the attachments attribute builds-in automatic real-estate control. What I mean by that is the "attachment" will not scroll through the channel out of control (image you want a list of all open alerts and have 1,000 of them). You will get "Show more" and "Show less" links automatically with attachments.

As the Slack docs also note, always create a "help" option that will show proper usage of the request, similar to a *nix man page.

Step 6: Define Slash Commands in your Slack App (multiple times)

Our final step to making this interaction between Slack and ServiceNow real is to create our slash commands for the application created in step 1. Once again, Slack has made this super easy for us. All we have to do is decide what we want the command called and then provide the endpoint for the Scripted REST API created in step 3.

Slash Command Definition

That's it! Once you have your slash command, you can go to the channel where you have your application registered and try it out. At this point you can correlate that each slash command will either have its own Scripted REST API, or its own resource for a given API.

Output

Let's see what some of this interaction looks like... notice that the response "Your /datanow listopen request has been received... processing" is "Only visible to you" because it's ephemeral and will eventually go away.

"/datanow listopen"

Below we can see details about a specific alert.

"/datanow alert:Alert0010382"

Notice here that we have the "Show more" button because we place that part of the message as an attachment. And finally...

"/datanow help"

To this point I have created a "/datanow" and "/actnow" command. The first is what we've shown in this blog and I have attached the code for the script action. The second, /actnow, is very similar except that it is much more dependent upon what orchestration activities you have available in your instance. If you're interested, you can ping me directly but I don't want to cause confusion (including the code with this blog) about why it doesn't work if you just drop in the code given you won't have the same workflows as me. While this was a bit of a lengthy post, I hope you were able to stick with me and can see how easily you can extend ServiceNow IT Operations Management into collaboration services like Slack.

ServiceNow Community servicenow community