The Mystery of Search Results — Solved in 7 Questions

[shruti_tyagi]

One of the most powerful tools within the ServiceNow application is the Global Text Search, which searches multiple tables within ServiceNow with a single search string. Text search enables us to find things easily without knowing which table or column the data is stored in. Search brings a logical collection of results grouped by table. Admin and users with the ts_admin role have the ability to make configuration changes to the text search to provide a better search experience for their users.

I have seen lot of threads in the Community about Global Text Search results, so I put together some answers to these seven commonly asked questions.

1. How are global search results sorted?

The answer to this question is a little complicated, but I will try to explain it using example. As we all know, global search results are sorted by relevancy. Let's see how the system decides the sorting sequence for relevant records.

In order to return the most relevant search results, each record is given a score that represents its relevancy to the search.

Scores are calculated based on these three things:

• Frequency: How often the search terms appear in the document
• Sequence: How often the search terms appear in the same order as the search query
• Weight: How heavily weighted the source field is in which the search terms appear

Here is one example: I searched for "windows vista" and here is my result:

(Image 1: Search Result for Windows Vista)

This result is sorted truly on the basis of frequency and sequence. We can see the article named "Windows Vista" is listed toward the end. The reason for that is this article does not have the words "windows" or "vista" in the body. As I mentioned above, the sort sequence is based on Frequency, which means the document will score more if it has more occurrences of the words "windows vista," "windows," and "vista."

The article titled "About Windows Vista" is listed first since it has 6 occurrences of "windows vista," 5 occurrence of "windows," and 17 occurrence of "vista." Because the article has 6 occurrence of "windows vista," which is the same Sequence as the search query, it is scored high and considered the most relevant search for the user.

How to set different weights for global search

Now let's see how an admin can make configuration changes to add more value to a user's global search. The admin can control the weight score by adding relative weights for each table:

Set the Relative Weight of the field

1. Add ts_weight attribute to the dictionary entry of the field you want to give more points. I added attribute ts_weight=50 on kb_knowledge.short_description:

(Image 2: Dictionary Record of Short Description)

This adds 50 points each time the search term appears in the short description. Here is my search result after I added ts_weight to short_description:

   

(Image 3: Search result after adding weight)

The article titled "Windows Vista" now appears as the second search result. Note that the "About Windows Vista" article is still more relevant as it has more points from Frequency and Sequence. Some fields in knowledge article and task table have elevate scoring weight by default.

Using meta tags to improve search results

Admins can also use meta tags in knowledge articles and catalog items to improve search results. Add searchable meta tags in the Meta field to return a positive hit even if the word never appears in the record. The meta field has a ts_weight of 10 in kb_knowledge by default. Can this weight value be changed? Let's see how this works by using an example. I added "windows vista" in the meta field of the articles "Windows Vista" and "What is the Windows key?" and to the catalog item "Access." Here are my results:

(Image 4: Search Result after adding string to Meta)

"Windows Vista" is now the most relevant article and "What is the Windows key?" is now third in the list as they get more points from the meta field (with a ts_weight=10). The catalog Item "Access" is also searchable even though it doesn't contain the term "windows vista" in any of the text.

For more information about meta tags, check out this blog by heidi.schnakenberg: Using meta tags in Knowledge Base articles to improve SEO and reduce incidents

2. How do you index two-digit numbers and single-character words?

By default, Zing (the search engine used in ServiceNow) does not index for two-digit numbers and single-character words (unless they are Chinese or Japanese characters) because they are considered "junk" phrases. You can disable this junk filter for a table if these are relevant to your search. For example, to search for "error 52" in the knowledge base, turn off junk filtering for kb_knowledge.

To turn off junk filtering, add the attribute text_index_filter_junk=false to a collection record of the table you want to configure. Add it to an extensible (parent) table if you are expecting results in child tables. For example, if you want to search two-digit numbers in incident table, add the text_index_filter_junk=false attribute on a collection record of the task table.

Remove stop words

Make sure the two-digit number you are trying to search for is not added as a stop word. All stop words are stored in table ts_stop under System Definition > Text Index Stop Words.

In this example, I added the attribute text_index_filter_junk=false to a task collection record. Then I did a global search for string "52" and here are my search results:

(Image 5: No Search Result for 52)

I checked to see if "52" is a stop word in the ts_stop table and confirmed that it is:

(Image 6: '52' As Stop Word)

So I deleted this stop word and did a search again for "52." Here are my search results this time around:

(Image 7: Search Results for 52)

Note: For optimal performance, disable junk filtering only if it is really required, otherwise it results in a larger table index and can cause slow performance on your global searches.      

3. How do you configure Form and List views for global text search results?

Admins have the capability to control the form and list view for global search results.

The glide.ui.text_search.match_view property controls the form view for exact matches in the global text search. In the base system, this property is blank, which means it will use the default view. Add view name if you want to show specific fields on form. For example, I added "ess" to this property:

(Image 8: Text Search View System Property)

Then I did a global search for the incident number for an exact match, and it is showing in the ess view now. Please note that if ess view is not available for any form, that means the system will show it in default view:

(Image 9: Exact Match in ESS View)

The glide.ui.text_search.view property controls the list and form view for global text search results. In the base system, this property is text_search. If the text_search view is not available for list or form, it will be default view. For example, the value for the glide.ui.text_search.view property is text_search, but there is no list view defined for an incident, hence it will be default view for list:

I added incident list for text_search:

(Image 10: List Element for Text Search view)

Then I cleared the cache 'https://<instance_name>.service-now.com/cache.do' after adding the view. The global search list is now in the new text_search view I created:

For more information on configuring global text search results with a text search view, check out Text search views format search results.

4. How do security rules affect search results?

Search results are already filtered out by security rules. Only users who meet the ACL requirements of the search results will actually see them. For example, if a property table is added for text indexing, and ITIL users try to search anything on the property table, they will not get any results since ITIL users do not have read access on the property table. Users may see different results for the similar search string depending on the roles and security rules they pass.

In order to see the search result a user should have read access to on the table, here is one example:

As an admin, here are my search results for the search string "windows." I see results for Business rules, UI actions, Client scripts, UI Policies, and system properties:

(Image 11: Search Result for Admin)

Here are the results that an ITIL user sees for that same search string:

(Image 12: Search Result for ITIL)

All these system process tables are hidden from ITIL users since they have no read access on these tables.

5. Why are users unable to search under certain groups and tables?

Here are two reasons this can happen:

1. The Search Group is restricted for the user and is controlled by an admin. The admin has the ability to add roles to Search Groups to make it available for the users with that role. Here is one example:

User with ITIL role is unable to search on task group:

(Image 13: No search on task Group)

Notice that the user does not even know search on task exists, because it is not displayed in the search groups at the top. Check the search group to see what roles this group has:

(Image 14: Search Group Roles)

It just has the admin role, which means only an admin can search on the task group. Add the ITIL role to the search to make it available for ITIL users.

2. The user accidentally changed the user preference for search table. Now that we added ITIL role to the search group, an ITIL user should be able to search on the Task group:

(Image 15: Search Table User Preference)

ITIL users can see the Task search group now and it is checked, but there are still no search results for task. Click on the Tasks search group to see whether the tables are checked:

(Image 16: Tables in Search Groups)

(Image 17: Search Tables not selected)

Oops! The search tables were not selected by the user (or the user accidentally removed it). Select search tables and ITIL users should now be able to search the Task search group:

(Image 18: User can search under task Group)

6. Can attachments be included in search results?

Attachments can be included in search results. By default, only knowledge base records include attachments. Admins have the ability to enable attachments in searches for other tables as well.

To enable attachment indexing on other tables, add the attribute attachment_index=true to the collection record of a table. It is important to remember that attachment indexing only applies to the table you added the attachment_index attribute to. It does not extend to child tables. For example, adding the attachment_index attribute to a task table does not index attachments on the incident table. For index attachments on the incident table, you must add it to the incident table.

Adding attachment_index attribute will trigger a regenerate index for the table. Reindexing larger tables like task and cmdb may take several hours and may slow down the system. We highly recommended doing this after business hours. used

7. How do you search for an exact phrase?

To search for an exact phrase match, make sure to include your search in quotation marks. For example, if you want to do an exact match for Windows Vista, include the entire phrase in quotation marks ("Windows Vista") so that Zing will consider it as one phrase. If you do not include the quotation marks, it will consider them as two separate words and will return search results for "windows," "vista," and "windows vista" and in any sequence.

Additional resources for global text search and indexing

Searching for more information on global text search, indexing, and stop words, here are some great posts you can read:

• Configuring auto stop words and regenerate text indexes by nisha.narayanakurup
• Global Text Search Suggestions by chuck.warner
• Debugging Search by britt.champeau
  
• Enhance your custom applications with Zing text search by tom.cullen@servicenow.com