Help
Join the Community
-
Documentation
Find detailed info about ServiceNow products, apps, features, and releases.
-
Impact
Drive a faster ROI and amplify your expertise with ServiceNow Impact.
-
Partner
Grow your business with promotions, news, and marketing tools for partners.
-
Store
Download certified apps and integrations that complement ServiceNow.
NLU FAQ, best practices, and general troubleshooting
Vancouver release
1. Frequently Asked Questions
What’s the difference between using keywords and NLU to discover topics in Virtual Agent?
Keywords searches for like terms between the user input and the topic name to discover the right topic. NLU better understands natural human expression and context to infer the right topic. NLU can also find entities within the user entry to improve topic discovery and simplify the Virtual Agent conversation. While we recommend that customers first start with keyword search in your Virtual Agent piloting phase, eventually you can migrate to NLU which better understands what users want and what context to answer questions or resolve issues.
How do I install/use NLU in ServiceNow?
NLU is automatically available and enabled in instances. In the navigator, type “NLU Workbench” to access the NLU Workbench. Admins need the nlu_admin (or admin) role to use the NLU workbench. It is also recommended that you install and update the "NLU Workbench - Advanced Features" plugin.
Do I need engineers and data scientists to configure NLU?
No! ServiceNow’s NLU provides a low-code interface to create, tune, and test NLU models. You can also use our pre-built NLU models optimized for IT, HR, and CSM use cases.
Models, intents, entities? What do all these NLU terms mean?
Please see the NLU documentation with a glossary of NLU terminology with examples.
What languages are supported in ServiceNow’s NLU?
See the following chart for languages currently supported. Although not all languages have entity support, admins can still create entities to improve model accuracy. Additional support for languages will come in future releases.
Can I use other NLU providers with the ServiceNow Virtual Agent?
Yes. If you’ve built NLU models with other services, you can also use NLU built in IBM Watson, Microsoft Luis and Google Dialog Flow.
What are NLU service updates? How does it impact me?
NLU Service updates are different from ServiceNow releases. The NLU service is updated every two months independent of your instance upgrade. Minor updates occur automatically, and you will use the new version when you (re-)train an NLU model. So long as you do not re-train your model, it will still use a previous service update. Major updates are aligned with ServiceNow releases like Rome, San Diego etc. and when you upgrade your instance to the next release and create and train a model it will use the latest version. The release notes for NLU service updates can be found here.
Is NLU supported on-prem/self hosted?
Yes, NLU is supported on-prem but requires additional configuration, see: KB0782052.
Creating an NLU model seems like a lot of work. Any help?
Admins don’t need to start from scratch. ServiceNow provides prebuilt NLU models optimized for IT, HR, and CSM use cases. They can be downloaded from the ServiceNow store or your instance Plugins page. You can start with a prebuilt model and modify it further with more intents or utterances to fit your needs. You can also import utterances and intents via CSV. You can do so by navigating to the model and clicking “Import data from CSV”. There are also links to resources and guides in the right-hand panel of the Workbench.
What’s a good number of utterances I should have in an intent? What are some good tips on creating utterances?
When populating an intent with utterances, you should have over 15 utterances. A minimum of 5 utterances is required to publish an intent. In the San Diego release, there is a limit of 20,000 utterances per model.
- Utterances should contain more than a couple words. Single word utterances are less accurate.
- Likewise, utterances should not be overly long and should be just one sentence. It should contain fewer than 200 characters though that’s not a hard limit.
- Punctuation, special characters, and double-spaces are not necessary. You can completely ignore punctuation by changing the system property: glide.mlpredictor.option.nlu.insensitiveToPunctuation to 'true'.
- Do not duplicate utterances within an intent or across intents. Utterances should have at least a couple words different between them.
- Utterances are case-sensitive! Use capitalized words only for proper nouns or acronyms in your utterances. Do not create utterances in all-caps. If using acronyms in caps, label them as entities.
What’s a good number of intents in a model? What are some good tips on creating intents?
A minimum of one intent is required for a model to be published with a max of 750 intents per model. The more intents in a model, the longer it may take to train.
- Each intent should be a desired action or goal for the user. There should be a 1:1 link between intent and Virtual Agent topic. Leaving NLU intents without a linked Virtual Agent topic may affect topic discovery.
- Do not enter special characters or spaces in the intent name. The “#” sign is added by the system automatically.
- Do not name your intents the same as your Virtual Agent topic names as you’ll want to easily confirm which is which while binding the intent to the topic.
I have hundreds of intents and my NLU model is taking a while to train. How come?
At a certain number of intents (~300), the NLU engine will switch from synchronous training to asynchronous training. This means that the training may take several hours as opposed to the seconds and minutes it takes for synchronous training.
How do I make sure that my different NLU intents don’t overlap?
The NLU workbench provides a built-in Conflict Review tool to ensure that multiple intents don’t overlap, I.e. an utterance picking up more than one similar intent.
How many models should I create?
We recommend creating as few models as necessary. Most organizations operate using a single NLU model, even across different business functions such as IT, HR, Finance, Legal, Procurement, etc. ServiceNow provides separate out-of-the-box NLU models for IT and HR intents, and it is possible to have a couple models per subject area/portal. It is not recommended to create extra models for a single subject area simply to split up otherwise related intents. Further details can be found in KB1002559.
NLU isn’t picking up on my synonyms, how come?
You can only have one synonym for each vocabulary. If you need to add multiple synonyms for an utterance word, use a vocabulary source or a regex Pattern type Vocabulary Item. More details on using Vocabulary Sources can be found here: KB0995575
I want to create entities for all my catalog items mentioned in the utterances, but that’s a lot of work. Is there a shortcut?
Yes, you can use vocabulary sources. Rather than calling out your catalog items individually, you can create a vocabulary source that points to the catalog table. You can then use an entity that references the vocabulary source each time you want to mention a catalog item. See this Virtual Agent Academy video for a demo video of how to do it and the results.
How do I measure and test how accurate my NLU model is?
There are a variety of tools at your disposal to test the accuracy of your NLU models. First, there is the NLU model tester on the Intent page. You can also test utterances in the Virtual Agent designer and see the intent discovered, confidence threshold, and any entities. Finally, you can use the batch testing tool to import a dataset (CSV) of utterances and expected intents, test it against your model, and view the anticipated accuracy percentage.
How do I change the confidence threshold when NLU isn’t accurate enough?
If you find that your model is too strict (not discovering the desired intent) or too lax (discovering multiple or different intents), you can adjust the confidence threshold in the model’s Settings tab. The default threshold is 65%.
What happens when the Virtual Agent does not pick up on an NLU intent?
If no NLU intent is found above the confidence threshold, then the system will fall back to keyword search. If a Virtual Agent topic is not bound to an NLU intent, keyword search will execute. If a Virtual Agent topic is bound to an NLU intent, keyword search will execute only when the com.glide.cs.nlu.keywords.include_topics_bound_to_lang property = ‘true’ (default setting is false).
Where can I view analytics related to the performance and usage of NLU?
You can view NLU metrics in the Conversational Analytics Dashboard, in the “NLU prediction” tab. You can also view the overall performance of your NLU in the “Performance” page. Navigate to “NLU Workbench > Performance” to view it.
2. General troubleshooting:
I’m running into an error message when training my NLU model.
First, ensure that you have updated to the latest patch available for your instance version.
- “Some features may not be available because the connection to NLU Service is not available.”, “Unable to predict.”, or “Malformed URL.” - Ensure that system property: glide.shared_service_scheduler.url is populated with “https://sncmlscheduler.service-now.com/”. If populated and issues persist, log a Case with the support team for assistance.
- “Training failed due to: Model information (...) has no reference for ...)”, “Error in training static lookups.” - There may be an error syncing vocabulary sources, e.g. an unsupported language.
- "Unable to train the model at this time. Please try again after some time.", “Unable to understand annotated entity sample." - Check that entity value and settings are filled out, and that vocabulary sources are synced.
- In general, you can open the ml_solution table which contains the records for training models and delete the current training solution (progress = 0 or progress = NULL).
- “Cannot convert null to an object” - This is due to missing attachments to the NLU model (in the ml_model_artifact table) when migrating models between instances. When cloning instances or migrating models, include large attachments in the clone option.
- If all else fails, log a Case with the support team.
The NLU model is returning strange results in the Virtual Agent (even if in the NLU workbench results are fine.)
Check to make sure that the sys_nlu_model records align with what's in ml_solution and ml_capability_definition base table. If ml_capability_definition table has extra records, delete it. Then go to the Virtual Agent topic, and then unbind and rebind the NLU model/intent, and click 'Save'.
Vocabulary sources take a long time to sync (several hours).
Our ml scheduler and predictor servers may be having issues. Please wait a day if possible. Otherwise, log a Case with the support team.
I migrated my NLU model from another (sub-prod/dev) instance and it isn’t working. E.g. the NLU workbench is grayed out.
After migrating a model, you need to re-train and re-publish the NLU model, even if the model's status says “Published”. You may also need to re-sync Vocabulary Sources if present. Also, ensure that all the related entities were also migrated.
How do I view the back-end NLU prediction data for what users typed?
You can use the open_nlu_predict_log table to see what users typed and the response model, intent, and confidence threshold.
Where do I view the back-end NLU intent to VA topic binding?
You can use the sys_cs_topic_language table to ensure that the NLU model and intent are binded to the right Virtual Agent topic per language. There should not be duplicate records in the table.
Additional Resources:
Now Learning – NLU Fundamentals
Additional NLU troubleshooting KBs:
