Migrating Virtual Agent to Natural Language Understanding (NLU)

[Nabil Asbi]

Perhaps the most pivotal Virtual Agent question is the following: How do I ‘lift and shift’ my Virtual Agent topics now that NLU is available in New York release?

In this article, we will provide recommended practices designed to help you migrate your existing Virtual Agent topics to work with NLU. Currently, your topic discovery works with predefined keywords. With the New York release, ServiceNow introduced to the platform Natural Language Understanding capabilities that will enable the user to use natural language in the conversational experience. For additional details on what’s new for Virtual Agent in the New York release, please refer to our product documentation here.

Benefits Of Virtual Agent With NLU

Incorporating Natural Language Understanding into your chatbot strategy has several benefits. Your employees are likely familiar with the Virtual Agent already, and expect the same help and guidance experience at work, in their preferred workspace, using their modal of choice.

Benefit 1: First and foremost, NLU will improve the end-user experience because it allows for a human-centric conversation designed to support the employee where and when they need it most. The NLU component enables Virtual Agent to think outside traditional keyword-constructs to understand, process, and respond using language just as peers and support teams do.

Benefit 2: Virtual Agent with NLU humanizes the chatbot by allowing users to type in a word or phrase they would use when asking a question or requesting assistance. Virtual Agent also provides 24x7 assistance when traditional channels such as phone support, email, and live chat are unavailable or not preferable.

For adaptation to drive adoption, Virtual Agent must display human-like qualities as employees grant their trust in the Virtual Agent’s ability to provide them with intelligent solutions. Last time you suspected your paycheck was incorrect – or you needed to check the status of your ticket – there’s a good chance you were eager to get help – fast. NLU provides a more expected, empathetic response to the issue and can turn a bad moment into a better one leaving the employee feeling supported.

Benefit 3: The NLU model can also be tuned to sharpen its vernacular to blend organization and industry terms into the conversational experience. Think of the typical employee who gravitates to the Virtual Agent for help – is this person going to ask using the Merriam Webster-version of the word, or the term they use every day? This means that over time, the Virtual Agent experience is more personalized and becomes a trusted advisor to employees as they navigate through their daily activities. Besides, who wouldn’t want to engage a genuinely personalized Virtual Agent for quick answers when we are all looking to maximize our productivity in the workplace?

What Needs To Be Modified For NLU

As you start evaluating how this new technology will best serve your goals, it’s crucial to begin with the end goal in mind. This means identifying your Virtual Agent objective(s) and what good should look like before and after you implement it. Operationally, we recommend you leverage your decision strategy toolkit – data, analytics, trends, and inflections – to determine a few initial topics aligned to what you set out to accomplish. For most organizations, this could be anything from frequent requests, search, case deflection, or routine incident automation.

Many organizations choose to begin with the out of the box (OOTB) topics and then venture on to create their own. If you are used the OOTB topics from a previous release (London or Madrid), we recommend reviewing the new OOTB topics in NY as they may have changed and now include NLU support. ServiceNow also provides OOTB NLU models that help you jumpstart your unique experience. These OOTB models were built using recommended practices, customer inputs, and designed to be a template or guide to help you get started.

Also included with NLU activation are Setup Topics or topics that can be reused as part of multiple Virtual Agent topic flows. Examples of Setup Topics include the Greeting, Error, Fallback Topic, and End Conversation. Including Setup Topics within your Virtual Agent Topics up-levels the employee experience from deterministic to probabilistic to display the desired human-like responses. Small Talk is another category that can be included as part of the Virtual Agent experience that allows for natural conversational variances as requests may not follow a linear pattern.

How To Migrate From Virtual Agent Using Keywords To NLU

We recommended you start with Phase 1 by replacing the Keywords with Intents. In Phase 2, you will proceed with enabling the entities by mapping the entities to the Intents. Once you are finished with establishing the Intents and mapping the Entities, you can then test and tune the NLU model for a truly personalized conversational experience.

Phase 0: Baseline configuration of NLU

One of the first decisions is to determine whether you will import the OOTB NLU models into your instance and use these as recommended-practice templates. Remember that the OOTB NLU models are read-only and cannot be modified. This is by design. The intent is for you to use the OOTB models as upgrade-proof templates to build your own models unique to your organization. To properly get started, with the option to leverage the OOTB models as they were intended, complete the following steps to configure the NLU model.

1. Create the NLU Model. In Studio (Filter Navigator > Studio), you must first create a new Studio application with its own scope (you may name this accordingly). The application you want to select is Natural Language Understanding. The default confidence threshold for your model is 60 percent. It is recommended you adjust this to 70-85 percent for more accuracy. This should reflect a ‘fitted’ model but not an ‘overfitted’ model.
2. Option to import OOTB NLU Models. If you choose to import the OOTB NLU models, select Import Intents from your new NLU model. To do this, check the box next to the Intents you want to import. You may import as many as ten at once.  Recommended practice: the OOTB Intents are read-only and can be found by selecting the appropriate application scope – ITSM, HR, or CSM. Importing the model will import the Intents so you can then modify this content to train, test, and tune the NLU model.
3. NLU Model readiness. You should now see the imported Intents activated in your NLU model, complete with a starter set of Utterances and Entities. From here, you may proceed with Phases 1, 2, and 3.

Phase 1: Replace your Keywords with Intents for better Topic discovery

1. Create Intents. Create Intent or use the OOTB Intents that will correspond to or match your VA topics. To create a new Intent, click New Intent. Intent name = [nameofVAtopic]. It will display as ‘#[nameofVAtopic]’ in the NLU model.
2. Map Utterances to your Intents. To build out your Intents, gather user Utterances (in addition to those delivered with the OOTB Intents) that employees would commonly use to interact with Virtual Agent. An utterance is how a person would ask a question. Recommended practice:
3. • Use all variations of a word, use capital letters and lower case letters, avoid ‘I want’ and duplicate utterances as this may result in a model training error.
   • For best results, use a minimum of 35 Utterances. You may use as few as 10 Utterances, but more are highly advised. Utterances can be gathered via survey, focus group, or existing data from cases or tickets.
4. Train and Test the NLU Model Builder. Train your model and once you have your Intents ready, complete with Utterances. To train the NLU model, click Train for each Intent. Once the model has completed training, test the model with a sample Utterance and observe the confidence rate. Continue Training and Testing the NLU model for each Intent until you get the desired results. You will tune the NLU model in a later Phase. For now, this will be sufficient to proceed. Once you have finalized your testing in the NLU Model Builder, make sure you publish the NLU model. This will activate the NLU model.
5. Enable NLU in Virtual Agent Settings. Enable the NLU mode in the Virtual Agent by navigating to Collaboration > Virtual Agent > General Settings. On the NLU Settings Tab, make sure the ‘Enable NLU in Virtual Agent’ is selected. The NLU Service Provider is ‘ServiceNow NLU.’ The Setup Topics Tab should default to the OOTB Setup Topics.
6. Map Intents to Virtual Agent Topics. Navigate to Collaboration > Virtual Agent > Designer. For each Topic (OOTB or custom), you will start mapping the Intents to the published Topics. Select your first Topic and open Topic Properties. In the NLU Model dropdown menu, select your new NLU model (that was published above), then add in the correct Associated Intent. You can select ‘Resume Topic Flow’ if you want a particular topic to support Small Talk Topics. a. Recommended practice: it is important to note that you cannot have both Keyword Virtual Agent Topics and NLU-enabled Virtual Agent Topics in the same instance. If you activate NLU, all Virtual Agent Topics will need to be modified for NLU.
7. Preview and Re-Publish. You will need to re-publish each Virtual Agent Topic once you have completed in the previous step. Previewing each Topic is optional.
8. Test the Topic discovery with NLU. Once your Virtual Agent Topics are published, test out each topic as if you were requesting assistance. Observe which Topics provided a great experience, and observe which Topics were not discovered. For those Topics with a sub-optimal experience, make a note of which phrases produced poor results.a. Recommended practice: the best way to debug the topic discovery, in case you are getting unexpected results, is to navigate to the [open_nlu_predict_log] table and begin exploring the root cause.
9. Tune the NLU Model. To achieve optimal results, navigate back to your NLU Model Builder in Studio. Tune the Intents by providing additional or more varied Utterances to improve the discoverability of the Topic. Train and Test the NLU model for the Intents you wish to improve upon. You may also adjust the confidence interval up or down as needed.

Phase 2: Add Entities

1. Identify the Entities. Map in Entities to the NLU Model Intents by first identifying which Entities are good mapping candidates. Entities can be inputs that you are using in the existing topic, in the form of Utterances. Utterances contain multiple words since they represent a common phrase a requestor would ask. Some of these words can be identifiable Entities that can be used to sharpen the NLU model. An entity can be a type of laptop, a type of address, or anything else that represents something that can be more accurately named in the NLU model builder. a. Recommended practice: Entities sit at the model level and not at the Intent level. You can view all mapped Entities by navigating back to the model in Studio > select Entities.
2. Add in Entities. To add an Entity, hover your pointer or mouse over one of the Utterances. You’ll notice that the words are highlighted as you do so. Click once into a word that is appropriate for an Entity mapping. You should see there are three types of Entities – simple, list, and pattern. Select ‘Create New Entity’ to add in an Entity, and you will then select the Entity type.  Recommended practice: 
3. • For a Simple Entity, this could be another way of phrasing the highlighted word (or anything that is not a List or a Pattern). Date, Time, Duration, and Location are all examples of Simple Entities.
   • For a List Entity, this could be anything that comes from a table and comprises a list of items you want the user to select from (i.e., Laptop Entities could be MacBook, IBM).
   • For a Pattern Entity, this is usually a number sequence, like a phone number or case number. Pattern Entities must be entered using regular expressions. Examples of regular expressions can be found in the product documentation.
4. Revisit the Intents and annotate Entities. Once you add in your Entities, you can now annotate these Entities with synonyms, in addition to the other definitions of Simple, List, and Pattern Entities. From the NLU model builder, you will notice a Vocabulary section to the right of the Entities. Click the plus sign to add in synonyms. Synonyms should reflect the Utterances and offer multiple variations of a single word. You will be prompted to enter a Base Word (i.e., Address), followed by a type (regular or pattern), and then a list of synonyms. Once you are finished annotating the Entities, train, test, and re-publish the NLU model. Recommended practice:
5. • Synonyms also live at the model level, along with the Intents and Entities. It is in the NLU model builder where you will add in common synonyms to words used in the Intents’ Utterances. Simply stated, this is just a way to add in additional synonyms for commonly used words to fine-tune the model for better topic discovery.
   • Recommended practice: it is advised that you look to the OOTB NLU Model(s) for recommendations and guidance on annotating your Entities.
6. Map Entities to VA Topics at User Input Nodes. Navigate to Collaboration > Virtual Agent > Designer. This step now ties the NLU Model Entities into the Virtual Agent designer. For each VA Topic, at each User Input node as part of that Topic (note: the 3 VA topic flow design tools include user input, bot response, and utilities; this pertains only to user inputs as this is how topic discovery and responses are humanized from the NLU model). To do this, select each User Input node (blue boxes in the VA designer flow) and select the following to be enabled:
7. • Enable NLU at Input Node
   • Confirm Entity Recognition
8. Once you have completed this exercise, you will preview and then save each Virtual Agent Topic Designer Flow with both the NLU at Input Node and Confirm Entity Recognition options activated.
9. Re-publish Virtual Agent Topics. Once you have previewed and saved your changes in the previous step, it is recommended the practice to re-publish each Virtual Agent Topic to avoid duplicates. This also ensures you are reviewing each Topic in detail and adjusting the Topic flows to reflect the desired user experience with the NLU component. Recommended practice: after re-publishing each Virtual Agent Topic, test out the user experience to ensure accuracy. If there are any errors or corrections, please refer to the previous steps to correct any inconsistencies.

Phase 3: Personalize and humanize the conversational experience

To read more about the power of setup topic, check this post.

1. Enable the Setup Topics. Navigate to Collaboration > Virtual Agent > General Settings > Setup Topics Tab. Here you will find the OOTB Setup Topics available to use. The Setup Topics with the (*) are recommended categories that inherit one of the Setup Topics. If the field is blank, you may select any one of the available Setup Topics to render for conversational components such as Live Agent topic, Error topic, or Anything Else topic. For a more detailed synopsis of the Setup Topics, please review our product documentation.a. Recommended practice: to create a custom setup topic, duplicate any of the existing setup topics. Be sure to consider if you want to map an Intent to the NLU model for this setup topic. The Anything Else topic and the Error Handling topic do not utilize Intents. Publishing your custom setup topic will make it available from the list of Setup Topics to choose from. b. Recommended practice: to clear a topic selection from the Survey Topic or the Anything Else topic, click on ‘please select’ in the topic list dropdown menu.

2. Personalize the Greeting Topic. If you are curating a unique, personalized Virtual Agent greeting, you will want to proceed with personalizing the Greeting Topic. This ensures the user receives a warm welcome reflective of your organization’s culture. To personalize the Greeting Topic, click the pencil icon to the right of the Greeting Topic field. Select Edit Topic Flow to modify the Greeting. Note that the associated Intent is Greetings and the category is Setup Topics.a. Recommended practice: the Setup Topics are all in the Global scope. Make sure you adjust your scope accordingly before you begin to edit any of the Setup Topics. The scope-picker is not extended into the popup window for editing, so you’ll need to select the Global scope before you select the pencil icon to make any changes. Once you are in the Topic Designer view, review the prompt on the right side of the screen. This is the default greeting message to continue consistency from keyword VA. It is designed to be modified. To edit this, click on the prompt [</>] to open the default script message. From here, you can modify the Greeting Topic to display a personalized message to the user. The next prompt by default will be the Script Action Utility titled Send Topic Picker. Preview and then save any changes. From there, re-publish the Greeting Setup Topic.b. Recommended practice: the Send Topic Picker Script Action Utility cannot be modified from its default format and will automatically display ‘Show Me Everything’ as it allows the user to select their Topic of choice as an alternative to typing in text to prompt topic discovery.

3. Connect to a Live Agent. If you plan to use live agent transfer from the Virtual Agent client interface, you will want to make sure the Live Agent Setup Topic is enabled. If you do not plan to use live agent transfer, navigate to Collaboration > Chat Setup > Uncheck the box next to Live Chat Enabled. From here, you will need to enter a No Agents Available Message and select Update.

4. Enable the Live Agent Setup Topic loop. When an error in a Virtual Agent conversation occurs, the user is automatically transferred to a live agent. You will want to make sure that Live Chat is enabled by navigating to Collaboration > Chat Setup. To enable the Live Agent Setup Topic, you can either select the Error Handling topic or create a custom topic with a script to transfer the user to a live agent.a. Recommended practice: if you are using Agent Workspace, the Virtual Agent conversation is automatically routed to an available agent based on the Chat service channel configuration and the queues that the Agent supports, as configured in Advanced Work Assignment (AWA).b. Recommended practice: if you are using Connect Support, the Virtual Agent conversation is routed to the appropriate chat support queue. For the user, the conversation with the live agent continues in the Virtual Agent client interface.

5. Personalize all other Setup Topics. You can personalize the scripts for the other Setup Topics as well. To do this, follow the defined process outlined in Step 2.

6. Map Setup Topics Intents and Entities. You must define an Intent and corresponding Entities for each of the Setup Topics (with the exception of the Anything Else topic and the Error Handling topic) in your NLU model to utilize the benefits of NLU for a personalized conversation. For the Greeting Setup Topic, an Intent could be #GreetingSetupTopic. To map the NLU Model and corresponding Intent to the correct Setup Topic, select the pencil icon next to each Setup Topic and make sure to select your NLU Model and the associated Intent on the Topic Properties page. Repeat this process for all Setup Topics with the exception of the Anything Else topic and the Error Handling topic.a. Recommended practice: while most of the Setup Topics do have corresponding NLU Models, the Anything Else topic and the Error Handling topic do not. This is intentional. The Anything Else topic shows up automatically at the end of a conversation as part of the framework behavior. It is not meant to be invoked stand-alone, and as a result, there is no Intent. The same holds true for the Error Handling topic.

Would you like to know more? Check out the Virtual Agent and Natural Language Understanding index for quick start resources! Click here.